PREPING: Building Agent Memory without Tasks

🎯 Pitch

Agents can build useful procedural memory before seeing a single real task, not by generating more synthetic data, but by controlling what they practice and selectively remembering only what works. PREPING eliminates the cold-start gap entirely—achieving broad tool coverage and strong performance without any human-provided demonstrations—and cuts deployment costs by up to 3× compared to online learning methods that still start empty.

1. Executive Summary

This paper introduces PREPING (Pre-Task Reusable Playbook Making), a framework that constructs reusable procedural memory for LLM agents before any target-environment task data is available, by coupling proposer-guided synthetic practice with validation-gated memory admission — a Proposer generates synthetic tasks conditioned on a construction-time memory state, a Solver executes them, and a Validator filters infeasible trajectories before they enter solver memory. Evaluated on AppWorld, BFCL v3, and MCP-Universe using DeepSeek-V3.2, PREPING improves over a no-memory baseline by 17.1, 19.3, and 5.4 points respectively while reducing deployment-time cost by 2.99× on AppWorld and 2.23× on BFCL v3 relative to online memory construction, establishing that procedural agent memory can be actively prepared through controlled, environment-grounded practice without any human-provided tasks — yet the gains come from proposer-side control over feasibility, redundancy, and coverage combined with selective memory updates, not from synthetic volume alone.

2. Context and Motivation

The Gap: Agent Memory Without Task Experience

The paper addresses a specific, practical gap in how LLM agents build procedural memory. Current approaches to agent memory construction fall into two categories, and both implicitly assume something that is often unavailable: access to target-environment task experience.

Offline methods build memory before deployment, but require human-defined tasks, curated demonstrations, or pre-collected solved trajectories from the target environment (Wang et al., 2025; Zhang et al., 2026). This means someone must design, collect, or solve representative tasks for each new environment — a burden that grows with the number of environments an agent might encounter. For a newly connected API, MCP server, or application suite, this task design effort must happen before the agent can be useful there.

Online methods avoid this upfront cost by constructing memory during deployment, learning from user interactions as they arrive (Ouyang et al., 2026; Zhou et al., 2025; Zhang et al., 2026). But this creates a different problem: the agent starts from empty memory. The first users to interact with a newly deployed agent experience its worst performance — before any procedural knowledge has been accumulated from their own or others' interactions. The paper refers to this as the cold-start gap: memory is most needed precisely when the experience required to build it has not yet been collected.

Figure 1 (left panel) illustrates this dichotomy visually: offline methods require prior human tasks and produce memory ready at deployment; online methods require no human tasks but start from empty memory. PREPING occupies a third, previously unoccupied position: no human tasks required, yet memory is ready at deployment.

Why This Matters

The significance of this gap extends beyond academic interest into several concrete deployment realities:

1. The environment proliferation problem. As agents are increasingly connected to diverse executable environments — from tool APIs and MCP servers to command-line interfaces (Anthropic, 2024; Merrill et al., 2026; Luo et al., 2025) — the number of environments an agent might need to operate in grows rapidly. Requiring human task design per environment scales poorly. Online memory avoids this, but means every new environment connection starts with empty memory.

2. Early user experience and trust. If an agent's initial interactions with users consistently fail because it lacks environment-specific procedural knowledge, users may abandon the system before it has a chance to improve. The cold-start failures documented in Figure 3 — where ACE-Online achieves only 74.4% success on its first 10 tasks versus 80.4% after accumulating experience — represent real user-facing failures that online methods cannot prevent.

3. Deployment-time cost. Online memory construction adds recurring inference cost during every user interaction: the agent must not only solve tasks but also update its memory. Figure 5 quantifies this at 2.23× to 2.99× the per-task cost of frozen pre-constructed memory. For high-volume production deployments, this multiplier represents substantial ongoing expense.

4. Coverage latency. The right panel of Figure 1 shows that online memory construction suffers from tool-coverage cold-start: ACE-Online requires 58 evaluation tasks to match PREPING's pre-deployment tool coverage on AppWorld Test-Normal, and on BFCL v3 Base, it still falls short after processing all 200 evaluation tasks. This means online memory may never reach the tool coverage breadth that pre-task construction can achieve, simply because user task distributions may not naturally exercise the full range of available tools.

Where Prior Approaches Fall Short

The paper identifies several specific limitations in prior work beyond the offline/online task-dependency issue:

Memory construction without task objectives is insufficient. Direct Memory (Section 4.1) converts environment documentation into memory without executing any tool calls. This approach captures what documentation describes — API signatures, parameter types — but cannot capture the procedural knowledge that only emerges through execution: how state changes propagate, which preconditions actually matter, what failure recovery patterns work. On AppWorld, Direct Memory improves over Base by only 1.6 points on average (Table 1), and its patterns are inconsistent — it reduces performance on BFCL v3 Base from 43.7 to 43.2.

Free-form exploration lacks task-level structure. Random and Guided Exploration (Section 4.1) execute tools in the environment without task-level objectives. They produce trajectories, but these trajectories are organized around exploring individual tools rather than composing tools to accomplish goals. The paper argues this matters because reusable procedural memory needs to capture multi-step workflows, not just single-tool usage patterns. On AppWorld, Random Exploration improves average performance by only 1.6 points over Base, while Guided Exploration adds only 2.5 points — modest gains despite 100 exploration trajectories of environment interaction. These baselines demonstrate that execution feedback alone, without task-level objectives, provides limited reusable signal.

Synthetic task generation without control contaminates memory. The Naive Task Generation ablation (Table 2) demonstrates what happens when synthetic tasks are generated without proposer-side control or validation gating: performance on AppWorld Test-Normal drops to 47.8/26.8 (TGC/SGC), actually below the Base baseline of 69.6/49.4. This catastrophic failure occurs because unfiltered synthetic trajectories can distill infeasible objectives, constraint violations, or failed recovery patterns into solver memory. The qualitative example in Appendix B.5 illustrates this concretely: when a task requires a non-existent resource (a valid Wells Fargo payment card), the Naive approach produces a trajectory that renames a different card to match the task specification, then distills this workaround into memory as a reusable rule — producing guidance that would lead future agents to override resource identities rather than recognize infeasible task constraints.

This is the key insight that motivates PREPING's coupled design: synthetic task generation is not a generation problem alone, but a joint problem of controlling what to practice and what to store. The Naive result shows that uncontrolled generation can be worse than no memory at all.

How PREPING Positions Itself

The paper frames its contribution not as a new task generation method or a new memory format, but as a control framework that couples two decisions synthetically-generated tasks cannot make on their own:

First, what to practice. Without task objectives from the environment, the agent must create its own. But arbitrary synthetic tasks are likely to be redundant (repeatedly exercising the same tools), infeasible (requesting unavailable entities or impossible operations), or uninformative (producing trajectories that reveal no reusable procedures). PREPING's proposer memory addresses this by maintaining a construction-time control state that tracks practice history, tool coverage, and grounded environment observations, then uses this state to push the Proposer toward feasible, non-redundant, coverage-expanding tasks.

Second, what to store. Even with well-designed synthetic tasks, execution trajectories may fail, produce partial results, or work around missing preconditions in ways that should not become reusable guidance. PREPING's Validator gates memory admission so that only feasible, successfully completed task-trajectory pairs enter solver memory, while all experience — including infeasible tasks and failures — informs proposer memory to improve future proposals. This asymmetric update is the core design pattern: all experience shapes future practice, but only validated experience becomes deployable guidance.

The paper positions this as distinct from prior self-generated practice work in a specific way: systems like Zhou et al. (2025) and Huang et al. (2025) use self-generated tasks for policy or model updates (training signals), optimizing for difficulty, solvability, and curriculum progression. PREPING uses self-generated tasks for memory construction, which requires a different form of control — the synthetic practice must expose broad, non-redundant, environment-grounded procedures, and only trajectories suitable for distillation into textual guidance should be admitted into memory. The goal is not to find tasks at the capability frontier, but to systematically cover the environment's procedural landscape.

Finally, the paper positions PREPING as complementary to, rather than competing with, online memory construction. Section 4.2 and Table 3 show that PREPING memory can serve as an initialization for online methods (PREPING+ACE), combining broad pre-task coverage with task-informed refinement. This framing is important: it suggests that the pre-task and online approaches address different phases of the agent lifecycle, and that practical deployments might use PREPING to eliminate cold-start failures and reduce deployment-time update cost, while still allowing memory to adapt from real user experience.

3. Technical Approach

3.1 Reader Orientation

PREPING is a control framework that builds reusable procedural memory for an LLM agent by having the agent invent, execute, and validate its own practice tasks in the target environment before seeing any real user tasks. It solves the cold-start problem — where an agent deployed to a new environment has no task-specific experience — by coupling two mechanisms: a proposer memory state that shapes what synthetic tasks to practice, and a validator gate that controls which practice outcomes get stored as deployable guidance, ensuring that memory is broad, grounded, and free of contaminated trajectories.

3.2 Big-Picture Architecture (Diagram in Words)

The system orchestrates four components in a closed loop over multiple construction iterations:

Proposer Memory ( $M_{\text{prop}}$ ) — a construction-time control state that tracks what has been practiced, which tools are under-explored, what failed and why, and what grounded environment facts have been discovered. It exists only during memory construction and is never exposed to the deployed agent.
Proposer ( $A_{\text{prop}}$ ) — an LLM conditioned on environment documentation ( $D$ ) and proposer memory ( $M_{\text{prop}}$ ) that generates a batch of synthetic task instructions ( $X_t$ ). Its job is to propose feasible, non-redundant, coverage-expanding tasks.
Solver ( $A_{\text{sol}}$ ) — an LLM that receives a synthetic task ( $x_t$ ), current solver memory ( $M_{\text{sol}}$ ), and environment access ( $E$ ), and executes the task to produce a trajectory ( $\tau_t$ ) of tool calls and environment observations.
Validator ( $A_{\text{val}}$ ) — an LLM that evaluates each $(x_t, \tau_t)$ pair, assigning structured scores for task feasibility and task completion. Its output ( $v_t$ ) gates whether the trajectory enters solver memory and provides feedback to update proposer memory.

The flow at each iteration $t$ : Proposer Memory shapes what to practice → Proposer generates synthetic tasks → Solver executes them → Validator scores them → Feasible trajectories update Solver Memory (deployable guidance), while all outcomes (including infeasible tasks and failures) update Proposer Memory to improve future task proposals.

3.3 Roadmap for the Deep Dive

First, the formal problem definition (Section 3.1 in the paper): what pre-task memory construction means, what the agent has access to, and what it must produce. This establishes the constraints that motivate every design decision.
Second, the core update loop (Section 3.2): the asymmetric memory update rule that lies at PREPING's heart — why all experience updates proposer memory but only validated experience updates solver memory.
Third, Proposer Memory (Section 3.3): the two complementary views (practice history and grounded environment information) that together control the synthetic task distribution.
Fourth, the Validator and memory admission gating (Section 3.4): the dual feasibility–completion scoring, how scores are used, and why infeasibility filtering is essential for memory quality.
Fifth, the memory induction pipeline: how validated trajectories are converted into compact procedural bullets for solver memory, following the ACE reflector–curator pattern.
Sixth, the deployment interface: how solver memory is supplied to the task-solving agent at inference time, and how PREPING can serve as an initialization for online memory construction.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a systems and empirical analysis paper whose core idea is that pre-task memory construction is a joint control problem over what to practice and what to store, and that a proposer-guided, validator-gated synthetic practice loop can solve it without access to any human-provided task data.

Formal Problem Definition: Pre-Task Memory Construction

The paper defines pre-task memory construction as a constrained setting with a clear input–output contract (Section 3.1). The inputs are:

A target executable environment $E$ , which can be called via tools or APIs and returns feedback (e.g., application state, function outputs, error messages).
Environment documentation $D$ , which specifies callable interfaces — API schemas, tool descriptions, parameter types — but typically omits implicit preconditions, state-dependent constraints, and failure recovery strategies that only become visible through execution.

The output is a solver memory $M_{\text{sol}}$ — a reusable textual artifact (in this paper, a structured playbook of procedural guidance) that will be supplied to the agent at deployment time to improve its success rate on real user tasks.

The critical constraint is what the construction procedure may not access: any target-environment task experience. This includes human-provided task instructions, demonstrations, solved trajectories, logged user interactions, or any other signal that reveals which user goals will appear, which tools should be composed to achieve them, or what successful task-level workflows look like. The agent may inspect documentation, call tools, and observe their outputs, but it must generate its own task-level objectives from scratch.

This makes the problem distinct from both offline memory construction (which assumes pre-collected tasks) and online memory construction (which starts from empty memory and learns from deployment-time tasks as they arrive). It also distinguishes it from standard environment exploration: documentation and tool schemas specify what can be called, but not what should be called together to accomplish goals.

The paper frames this as a controlled synthetic practice problem: the agent must create and execute its own task-level objectives, but doing so naively produces redundant, infeasible, or misleading trajectories. The core challenge is therefore to jointly shape both the distribution of practice tasks and the quality filter for what enters memory.

The Core Update Loop and Asymmetric Memory Structure

PREPING separates memory into two distinct states with different roles and different update rules (Section 3.2). This separation is the central design pattern:

Proposer memory ( $M_{\text{prop}}$ ) is a construction-time control state that exists only during the memory construction phase. It records what has been practiced, which tools or workflows remain under-explored, which proposals failed and why, and what grounded environment facts have been discovered through execution. Its purpose is to make the next round of synthetic task proposals better — more feasible, less redundant, and more coverage-expanding — by providing the Proposer with a rich context about the construction history.
Solver memory ( $M_{\text{sol}}$ ) is the deployment-facing artifact. It contains the distilled procedural knowledge that will be provided to the task-solving agent during inference. What enters solver memory must be reliable enough to serve as guidance during real user interactions.

At iteration $t$ , the system coordinates three LLM-powered modules:

Step 1: Task generation. The Proposer generates a synthetic task $x_t$ conditioned on environment documentation and the current proposer memory:

$x_t \sim A_{\text{prop}}(\cdot \mid M_{\text{prop}}^{(t)}, D)$

where $A_{\text{prop}}$ is the Proposer (an LLM with a specific prompt and role), $M_{\text{prop}}^{(t)}$ is the proposer memory at the start of iteration $t$ , and $D$ is environment documentation.

Step 2: Task execution. The Solver executes $x_t$ in the live environment, producing a trajectory $\tau_t$ — the sequence of tool calls, their arguments, and the environment's responses:

$\tau_t \sim A_{\text{sol}}(\cdot \mid x_t, M_{\text{sol}}^{(t)}, E)$

where $A_{\text{sol}}$ is the Solver, $M_{\text{sol}}^{(t)}$ is the current solver memory (which may contain useful procedural guidance from earlier synthetic practice), and $E$ is the executable environment.

Step 3: Validation. The Validator evaluates the task–trajectory pair and produces a structured judgment:

$v_t = A_{\text{val}}(x_t, \tau_t)$

where $v_t$ contains feasibility scores, task-completion scores, and rationales for both. These scores are on a 1–5 Likert scale, with specific criteria (detailed below in the Validator section).

Steps 4–5: Asymmetric memory updates. The two memories are updated differently:

$M_{\text{prop}}^{(t+1)} = U_{\text{prop}}(M_{\text{prop}}^{(t)}, x_t, \tau_t, v_t)$

$M_{\text{sol}}^{(t+1)} = \begin{cases} U_{\text{sol}}(M_{\text{sol}}^{(t)}, x_t, \tau_t, v_t), & \text{if Feasible}(v_t) \\ M_{\text{sol}}^{(t)}, & \text{otherwise} \end{cases}$

where $U_{\text{prop}}$ is the proposer-memory update function, $U_{\text{sol}}$ is the solver-memory update function (the ACE reflector–curator pipeline), and $\text{Feasible}(v_t)$ is a boolean indicating whether the synthetic task is grounded in the environment and suitable for memory construction.

Why this asymmetric structure matters. All experience — including rejected tasks, infeasible proposals, and failed executions — updates proposer memory. This means future task proposals benefit from knowing not to repeat the same mistakes. But only feasible, validated task–trajectory pairs are eligible for solver memory. This asymmetry is the core mechanism preventing memory contamination: the Proposer learns from everything, but the deployed agent only receives reliable guidance.

While the equations show one synthetic task per iteration for clarity, in practice PREPING samples a batch of tasks per iteration. The main experiments use 10 iterations with 10 synthetic tasks per iteration, for a total construction budget of 100 synthetic tasks.

Proposer Memory: The Construction-Time Control State

Proposer memory ( $M_{\text{prop}}$ ) is what makes synthetic task proposals controlled rather than random. It maintains two complementary views of the construction process (Section 3.3):

View 1: Practice History. This view tracks what has already been done, enabling the Proposer to avoid redundancy and push toward under-explored parts of the environment. It contains:

Previous synthetic tasks and their outcomes (solved, failed, infeasible).
Tools, APIs, or functions invoked in each trajectory, extracted via rule-based parsers from the raw interaction logs.
Aggregate usage summaries identifying which tools have been over-practiced (e.g., "amazon:27" in Appendix B.4) and which are under-practiced.
Failure reasons and infeasibility reasons — short natural-language explanations of why a task failed or was infeasible.

Operationally, $U_{\text{prop}}$ updates this view by parsing invoked tools and functions from trajectories (using rule-based extraction, not another LLM call) and combining them with the Validator's structured output $v_t$ (which includes feasibility scores, completion scores, and rationales). The practice history is rendered as context for the Proposer in the next iteration, creating pressure against near-duplicate tasks and toward under-covered tools.

View 2: Grounded Environment Information. This view captures what the agent has learned about the actual state of the environment — which entities exist, what preconditions hold, what constraints are real. It is summarized via an LLM call (the prompt is shown in Figure 8) that extracts concrete observations from trajectories, such as:

"The environment includes a Gmail app with APIs for email management, including show_inbox_threads which can filter by attachment presence and supports pagination."
"Available Amazon products have IDs like 1-5, 21-24, 42, 348, 403..." (Appendix B.4)

This summarization step is critical because raw trajectories contain too much noise for the Proposer to parse efficiently. By extracting only reusable environment facts — concrete entities, observed states, preconditions, and constraints — the summary keeps future proposals grounded in what is actually executable, rather than allowing the Proposer to invent unsupported task details (e.g., referencing a song ID that does not exist).

The complementary pressures. When both views are provided as context to the Proposer, they impose two complementary forces on the synthetic task distribution:

Practice history discourages near-duplicate tasks and encourages expansion toward under-covered tools, APIs, or workflows.
Grounded environment information keeps that expansion feasible by anchoring proposals in executable facts.

The ablation study in Table 2 confirms this complementary relationship. Adding practice history alone (third row: Validator + History) improves tool coverage — unique APIs on AppWorld increase from 69.0 to 81.7 — but also raises the infeasible task rate because history pushes expansion into areas that may not be well-grounded. Adding environment information alone (fourth row: Validator + Env Info) reduces the infeasible task rate to 5.3% — the lowest across all variants — but reduces coverage because environment information without history does not encourage exploration of under-covered tools (unique APIs drop to 42.3, the lowest among all variants). The full PREPING combines both signals, achieving strong coverage (87.0 unique APIs) with a moderate infeasible rate (16.0%), while maximizing downstream task performance.

Practical rendering. When provided as context to the Proposer, these two views are presented as structured text sections (visible in the prompt template in Figure 6). The environment information appears as an "Environment Information" section with observation-level summaries; the practice history appears as a "Prior Task History" section with solved tasks (including involved apps and APIs), failure tasks with reasons, infeasible tasks with reasons, and tool usage statistics. The Proposer prompt includes explicit instructions on how to use each:

"Use prior task history as weak guidance for where to explore next, not as templates to imitate."
"If environment information is provided, prefer entities, files, users, tickets, airports, and symbols that are explicitly supported there."
"Avoid near-duplicates: do not merely rename entities or tweak dates, numbers, thresholds, or output format while keeping the same task pattern."

The Validator: Gating What Enters Solver Memory

The Validator ( $A_{\text{val}}$ ) is the quality control mechanism that prevents synthetic artifacts from contaminating solver memory. It evaluates each $(x_t, \tau_t)$ pair along two dimensions (Section 3.4), producing structured outputs:

Dimension 1: Task Feasibility. This judges whether the proposed task is grounded in the environment and executable under the observed state and available tools. The prompt (Figure 7) specifies that every required target entity, field, metric, or intermediate value must be positively supported by trajectory evidence. Negative evidence — "not found", empty matches, missing records, authentication failures — counts against feasibility. The scoring rubric:

5 (Excellent): Fully executable as stated; all required entities, metrics, and intermediate values are explicitly grounded in trajectory evidence.
4 (Good): Executable with minor ambiguity, but the needed values and entities are still reasonably supported.
3 (Acceptable): Plausibly executable, but one key field, metric, or grounding step is weakly supported.
2 (Poor): Likely infeasible; a required entity, field, or intermediate value is missing from observable tool outputs.
1 (Unacceptable): Infeasible or contradictory; the task relies on unsupported tools, impossible preconditions, or missing critical values that cannot be grounded.

Dimension 2: Task Completion. This judges whether the Solver actually accomplished the synthetic task objective. The scoring rubric:

5 (Excellent): Every requirement and constraint is satisfied.
4 (Good): The task appears completed with only minor ambiguity.
3 (Acceptable): Some progress, but at least one requirement is missing or weakly supported.
2 (Poor): Minor progress only; the main outcome is not achieved.
1 (Unacceptable): No meaningful completion.

The Validator returns both scores plus short natural-language rationales for each (see Figure 7 for the full prompt and the example in Appendix B.4 for a concrete output).

Three uses of Validator outputs. The Validator's signal $v_t$ is used in three distinct ways:

Use 1: Gating solver-memory admission (the primary function). A task–trajectory pair is eligible for solver memory insertion only when its feasibility score is 5 (the strictest threshold). This criterion means the proposed task must be clearly grounded in the executable environment — every referenced entity, tool, and precondition must be verified by trajectory evidence. Trajectories from infeasible tasks are excluded from $U_{\text{sol}}$ entirely, preventing the kind of memory contamination shown in Appendix B.5 (where a workaround for a non-existent payment card gets distilled as a reusable rule).

The completion score serves a secondary role in memory induction: tasks with completion scores of 4 or higher are treated as successful for the purpose of extracting procedural lessons (e.g., what workflow pattern worked), while lower-completion tasks even if feasible may still contribute insights about failure modes.

Why unfeasible admission matters. The Naive Task Generation ablation (Table 2, first row) demonstrates the catastrophic effect of skipping validation: AppWorld Test-Normal performance drops to 47.8/26.8 — below the no-memory baseline of 69.6/49.4. This happens because Naive Task Generation invokes 65.7 unique APIs with a weighted recall of 0.653 (comparable coverage), but its unfiltered trajectories distill infeasible objectives, constraint violations, and failed recovery patterns into solver memory, producing guidance that actively misleads the deployed agent.

Adding only the Validator admission gate (second row: Validator only) improves performance to 78.2/60.7 — a recovery of over 30 points in TGC. This is the single largest effect in the component ablation, confirming that infeasibility filtering is the most important function of the Validator.

Use 2: Guiding proposer memory updates (failure and infeasibility signals). All validation outcomes — including rejected pairs — are passed to $U_{\text{prop}}$ . The feasibility scores and rationales help the Proposer understand why a task was infeasible (e.g., "song ID 555 does not exist"), so future proposals avoid similar impossible setups. The completion scores and rationales help identify capability gaps — tasks that were feasible but the Solver could not complete, suggesting areas where the Solver needs more practice or where task difficulty should be adjusted.

Table 9 in Appendix B.3 quantifies the value of this structured signal. Removing validator-derived result details from solver-memory updates reduces AppWorld Test-Normal performance from 83.7/70.2 to 81.9/66.7, while hiding validator labels and reasons from proposer memory reduces it to 82.5/66.1. These drops are much smaller than removing validator-gated admission entirely, indicating that the admission gate is the primary mechanism, but the structured success/failure/infeasibility signals provide additional guidance for constructing higher-quality memory.

Use 3: Providing task-completion labels to the reflector–curator pipeline. For trajectories admitted into solver memory, the Validator's task-completion score and rationale are passed to the ACE reflector–curator pipeline (described next) as a proxy for ground-truth feedback. This allows the reflector to distinguish successful trajectories (where the lesson is "this workflow works") from incomplete ones (where the lesson is "this part failed, here's why"), even though no human-labeled correctness signal is available.

LLM instantiation. The Validator uses DeepSeek-V3.2 (the same backbone as all other modules) with reasoning mode disabled, temperature 0.7 (slightly elevated to support nuanced Likert judgments), and a structured output format requiring JSON with feasibility_score, task_completion_score, and natural-language reasons for each.

Solver Memory Induction: The ACE Reflector–Curator Pipeline

When a task–trajectory pair passes the feasibility gate ( $\text{Feasible}(v_t)$ is true), it enters the solver-memory update function $U_{\text{sol}}$ . PREPING does not invent its own memory induction method; it reuses the reflector–curator playbook induction pipeline from ACE (Zhang et al., 2026), keeping this component identical across all baselines to isolate whether memory quality comes from how practice tasks are generated and filtered rather than from how trajectories are converted into memory.

The pipeline has two stages, each implemented as a separate LLM call:

Stage 1: Reflector (Figure 9). Given the synthetic task instruction, the trajectory, the current solver memory (termed "playbook" in ACE's terminology), and the Validator's output (serving as ground-truth feedback), the reflector diagnoses what happened:

Error identification: If the Validator reports incomplete execution, what specifically went wrong? Conceptual errors, miscalculations, misapplied strategies?
Root cause analysis: Why did the error occur? Wrong source of truth? Bad filters (timeframe, direction, identity)? Formatting issues? Missing authentication?
Correct approach: What should the Solver have done differently?
Key insight: What strategy, formula, or principle should be remembered?
Bullet tagging: For each existing bullet in the current playbook, the reflector tags it as helpful, harmful, or neutral — indicating whether the existing guidance contributed to this trajectory's outcome.

The Validator's task-completion signal is critical here: it tells the reflector whether the trajectory represents a success story (where the lesson is "do this") or a failure case (where the lesson is "avoid that"). Without this signal (the ablation in Table 9, "PREPING w/o Validator Signal in Solver-Memory Update"), the reflector must infer success or failure from trajectory content alone, which is noisier.

Stage 2: Curator (Figure 10). Given the current playbook, the trajectory, and the reflector's analysis, the curator decides what new procedural knowledge to add. It operates on three sections of the playbook:

Strategies: High-level procedural guidance (e.g., "Use API-side filtering instead of client-side filtering to reduce data transfer").
Code snippets: Reusable code patterns observed in successful trajectories.
Pitfalls: Common failure modes and how to avoid them (e.g., "Avoid assuming one API call returns all data — always check for pagination").

The curator adds new bullets only for insights that are absent from the current playbook. It does not regenerate the entire playbook — only the additions. This incremental design prevents the memory from growing unboundedly and ensures that new additions complement rather than duplicate existing guidance. If no new insights are warranted, the operations list is empty.

Why reuse ACE rather than design a new memory format. The paper's claim is about how to source practice experience, not about how to represent memory. By reusing ACE's proven reflection–curation pipeline across all baselines (Direct Memory, Random/Guided Exploration, Naive Task Generation, and PREPING), the experiments isolate the effect of the practice distribution and admission gating. Any performance differences between these methods are therefore attributable to what experience goes into the memory induction pipeline, not to the induction pipeline itself.

Memory format. The resulting solver memory is a structured textual playbook with numbered bullets in each section. For the AppWorld prompt (Figure 11), this playbook is inserted into the task-solving context as a {{Solver Memory}} section between the environment instructions and the task description. For BFCL (Figure 12), it appears between the function definitions and the user message. For MCP-Universe (Figure 13), it appears between the tool descriptions and the question.

Task Proposal, Execution, and Validation in Detail

Proposer ( $A_{\text{prop}}$ ). The Proposer is an LLM instantiated with a specific prompt (Figure 6 for BFCL; AppWorld and MCP-Universe use analogous templates with environment-specific terminology) that includes:

Environment documentation (API/function schemas and descriptions).
The current proposer memory, rendered as two sections: environment information (grounded entity summaries) and prior task history (solved tasks, failure tasks with reasons, infeasible tasks with reasons, and tool usage statistics).
Explicit generation guidelines: prefer feasible tasks, ground tasks in documented entities, avoid near-duplicates, push toward under-covered tools, avoid vague requests, and generate tasks that teach reusable agent behavior.

The Proposer generates a batch of tasks as a JSON array, each specifying:

servers (or apps for AppWorld): which tools/APIs the task should use.
intended_functions (optional): a lightweight hint for analysis.
question: the natural-language task instruction.

Temperature is set to 1.0 for task generation to encourage diversity in proposals. The Proposer uses DeepSeek-V3.2 without reasoning mode, matching all other modules.

Solver ( $A_{\text{sol}}$ ). The Solver executes synthetic tasks using the same interface and model as downstream task solving. For AppWorld, it operates in a Python REPL environment where it writes code, the environment executes it, and the output is returned. For BFCL, it emits function calls in a structured format, and the environment returns function outputs. For MCP-Universe, it follows a ReAct pattern with JSON-formatted thought–action–observation cycles.

During synthetic practice, the Solver has access to the current solver memory ( $M_{\text{sol}}$ ) — meaning as construction proceeds and the playbook accumulates procedural guidance, later synthetic tasks may benefit from the knowledge distilled from earlier ones. Temperature is set to 0 for Solver execution to maximize reproducibility.

Validator ( $A_{\text{val}}$ ). As described above, the Validator evaluates each task–trajectory pair. Temperature is set to 0.7 to support nuanced Likert judgments.

Iteration control. The construction loop runs for a fixed number of iterations ( $T = 10$ in the main experiments) with a fixed batch size per iteration ( $N = 10$ synthetic tasks). This yields a total construction budget of 100 synthetic tasks. The process is offline — it happens before deployment and its cost is amortized across all subsequent user tasks.

Deployment Interface and Online Integration

Once construction completes, the solver memory $M_{\text{sol}}$ is frozen and supplied to the task-solving agent at deployment time. Two deployment modes are evaluated:

Mode 1: Frozen pre-task memory. The solver memory is inserted into the task-solving prompt as static context and is never updated during deployment. This eliminates all deployment-time memory-update cost — the 2.23× to 2.99× cost reduction shown in Figure 5 — but the memory cannot adapt to user-specific patterns or task distributions.

Mode 2: Initialization for online memory (PREPING+ACE). The solver memory is used as the starting playbook for ACE-Online, which continues to update it during deployment as real user tasks arrive. The online update procedure is the same reflector–curator pipeline, but now the ground-truth signal comes from benchmark evaluation feedback (for evaluation) or user feedback (in production). This combines the broad pre-task coverage from PREPING with the task-specific refinement from online learning. Table 3 shows this initialization improves ACE-Online's AppWorld average from 71.3 to 76.3, and Figure 3 shows it eliminates the cold-start dip in early deployment performance.

Task-solving models. At deployment time, the same DeepSeek-V3.2 backbone is used for task solving. For the backbone transfer experiments in Table 4, the solver memory was constructed using one backbone (e.g., GPT-5.1) and then evaluated on the same backbone, confirming that the construction procedure transfers across model families but the memory itself is used by the model that constructed it.

Summary of Key Design Choices and Their Justifications

Asymmetric memory structure (proposer vs. solver): Separates control signals (what to practice next, what failed and why) from deployment guidance (what procedures to follow). Without this separation, memory would either be contaminated with construction-time meta-information or fail to benefit from failure experience.
Validator-gated admission with feasibility = 5 threshold: The strictest possible admission criterion — the task must be clearly and completely grounded in trajectory evidence. The Naive ablation (47.8 TGC vs. 83.7 with full PREPING) shows that any infeasible trajectory admission is catastrophic, justifying the conservative threshold.
Practice history as weak guidance, not hard constraints: The Proposer prompt instructs the model to treat usage statistics as "weak context about prior coverage, not as hard constraints or item-specific targets." This prevents the Proposer from mechanically targeting the least-used tool even when no feasible task can be built around it — a balance between coverage expansion and feasibility.
LLM-summarized environment information rather than raw trajectories: Raw trajectories are long, noisy, and contain transient state that is irrelevant to future proposals. The summarization step (Figure 8) extracts only reusable environment facts, producing compact 2–5 bullet summaries that are efficient for the Proposer to consume.
Reusing ACE reflector–curator for memory induction: Isolates the contribution of the practice distribution and admission gating from the memory representation. By holding the induction pipeline constant across baselines, the experiments cleanly attribute performance differences to what experience is practiced and filtered.
Temperature 1.0 for task proposal, 0 for execution, 0.7 for validation: High temperature for proposal encourages task diversity; zero temperature for execution ensures reproducible trajectories for validation; moderate temperature for validation supports calibrated Likert judgments.
Batch processing (10 iterations × 10 tasks): Amortizes the fixed cost of proposer memory rendering and prompt construction across multiple tasks per iteration, while allowing proposer memory to be updated between iterations so that later iterations benefit from earlier discoveries. The budget curve in Figure 4 shows that 50–100 tasks is the sweet spot — diminishing returns set in beyond 100 tasks, but 30 tasks is already sufficient to surpass free-form exploration baselines.

4. Key Insights and Innovations

Innovation 1: Reframing Memory Construction as a Joint Control Problem Over Synthetic Practice

The dominant paradigm for agent memory construction, whether offline (Wang et al., 2025; Zhang et al., 2026) or online (Ouyang et al., 2026), treats memory as a passive record of task experience — the agent does tasks, and memory accumulates from what happened. Even self-generated practice methods (Zhou et al., 2025; Huang et al., 2025) focus on generating good tasks but treat the resulting trajectories as uniformly suitable for learning. PREPING makes a fundamental conceptual move: it reframes pre-task memory construction not as a generation problem (can we make good synthetic tasks?) but as a joint control problem over two coupled decisions — what to practice and what to store — where these decisions are interdependent and asymmetric.

The insight that makes this more than a rebranding is the demonstration that these two decisions can conflict in destructive ways. The Naive Task Generation ablation (Table 2, row 1) shows this empirically: generating tasks without controlling what gets stored produces memory that is worse than no memory at all (47.8 TGC vs. 69.6 Base on AppWorld Test-Normal). The qualitative example in Appendix B.5 reveals the mechanism: when a task is infeasible (requiring a non-existent Wells Fargo payment card), the Solver will still produce a trajectory — it renames a different card and completes the transfer — and standard practice would store this trajectory as a success. The result is a memory rule that actively teaches the agent to override resource identities rather than recognize infeasible constraints. Naive generation creates the practice, but uncontrolled storage converts practice artifacts into deployable poison.

This interdependence is non-obvious. Prior work on self-generated practice (Zhou et al., 2025; Acikgoz et al., 2026; Xia et al., 2025) typically assumes that more practice is better — the challenge is generating enough diverse, difficulty-calibrated tasks. PREPING shows that this assumption breaks when the output is not a policy update (where gradient-based optimization provides implicit regularization) but a textual memory artifact that an LLM will read and follow at deployment time. The memory format is brittle — a single misleading rule can persist and misguide future behavior in ways that parameter updates average out.

The asymmetric memory structure (Section 3.2) operationalizes this reframing: proposer memory learns from everything, solver memory admits only validated trajectories. This is not an implementation detail — it is the architectural embodiment of the insight that controlling practice and controlling storage are distinct problems with different information requirements. The Proposer needs to know about failures and infeasibilities to avoid repeating them; the deployed Solver must never see them as guidance. This separation between construction-time control state and deployment-time procedural memory is the paper's core conceptual contribution — it provides a vocabulary and a design pattern for reasoning about memory construction that didn't exist before.

Innovation 2: The Cold-Start Gap as a Distinct Research Problem with Measurable Consequences

The paper's second conceptual move is to carve out pre-task memory construction as a distinct phase in the agent lifecycle, rather than treating it as simply "online memory construction, but done earlier." This distinction matters because the two phases have fundamentally different information availabilities and different failure modes.

Prior work implicitly assumed a spectrum from offline to online: either you have task data before deployment (offline) or you accumulate it during deployment (online). PREPING identifies a third regime that is not on this spectrum: the agent has environment access (documentation, executable tools, observable feedback) but no task-level objectives of any kind. This regime is not simply "offline without human tasks" — that would be an impoverished offline setting. It is a qualitatively different setting because the absence of task objectives means the agent must invent its own goals before it can learn anything, and the quality of those invented goals determines the quality of the resulting memory.

The paper makes this measurable through the cold-start gap. Figure 3 quantifies what was previously an abstract concern: ACE-Online's first-10-task success rate is 74.4%, substantially below its steady-state 80.4%, and well below PREPING's 79.4% from pre-constructed memory. The right panel of Figure 1 provides a mechanistic explanation — tool-coverage cold-start: ACE-Online needs 58 evaluation tasks to match PREPING's tool coverage on AppWorld, and never catches up on BFCL v3 even after 200 tasks. These are crisp, quantitative demonstrations that the cold-start gap is real, substantial, and driven by a specific measurable cause (coverage latency).

This framing reframes the cost calculus for agent memory. Previously, the choice was between paying the upfront cost of human task design (offline) or paying the ongoing cost of deployment-time memory updates plus early failures (online). PREPING introduces a third option: pay a one-time pre-deployment cost of controlled synthetic practice, then deploy with frozen memory (eliminating update cost and cold-start failures) or use it as a warm start for online learning (eliminating cold-start failures while retaining adaptability). The deployment cost savings (2.23× to 2.99×, Figure 5) are a consequence of this structural insight: moving memory construction from a recurring deployment-time expense to an amortizable pre-deployment investment.

Innovation 3: Validation-Gated Memory Admission as the Primary Determinant of Memory Quality

The component ablation in Table 2 delivers a striking diagnostic finding: the Validator's admission gate accounts for the single largest performance swing in the entire system. Adding only feasibility-gated admission to Naive Task Generation recovers over 30 points in AppWorld TGC (from 47.8 to 78.2), while all other components combined — practice history, environment information, proposer-side control — contribute the remaining 5.5 points (to 83.7). This is not a marginal improvement; it is a phase transition from harmful memory to useful memory.

What makes this finding distinctive is that it runs counter to the intuitive priority in synthetic data research, which typically focuses on generation quality — better prompts, better curricula, better difficulty calibration (Huang et al., 2025; Zhou et al., 2025). PREPING shows that in the memory construction setting, filtering is more important than generation. The Naive variant already generates tasks that exercise 65.7 unique APIs with a 0.653 weighted recall — comparable coverage to the full PREPING. The problem is not that the tasks are bad; it's that the trajectories from infeasible tasks produce actively harmful memory rules.

This insight has two implications beyond the paper. First, it explains why prior work on self-generated practice for LLM policy updates didn't encounter this problem: gradient-based fine-tuning is robust to noisy examples in ways that textual memory is not. A few misleading training examples get averaged out in the gradient; a single misleading memory rule sits in the context window and can override correct reasoning on every future task. The memory format's brittleness amplifies the importance of filtering.

Second, it suggests that the threshold for admission matters enormously. PREPING uses the strictest possible criterion: only tasks with feasibility score 5 (the maximum) enter solver memory. This is a deliberately conservative choice, and the gap between the Validator-only ablation (which does feasibility gating but lacks proposer-side control) and full PREPING suggests that proposer-side control becomes valuable precisely because it increases the yield of admissible trajectories — making more of the synthetic practice budget produce trajectories that pass the strict feasibility filter. Without proposer control, many tasks are infeasible and wasted; with proposer control, more tasks are feasible and contribute to memory. This coupling — proposer control increases admission yield, validation gate ensures quality — is the mechanism behind PREPING's efficiency, not either component alone.

Innovation 4: Proposer Memory as a Dual-Pressure Mechanism for Coverage and Feasibility

The ablation in Table 2 reveals that practice history and environment information exert opposing, complementary pressures on the synthetic task distribution, and that combining them produces better memory than maximizing either pressure alone. This finding is subtle and would be easy to miss — it emerges from the tension between the two proposer-memory views, not from their independent effects.

Practice history alone (Validator + History, row 3) pushes the Proposer toward under-covered tools, producing the highest unique API count (81.7 on AppWorld) and the highest weighted recall (0.674) among all variants with validation. But it also produces the highest infeasible task rate (33.3%) because history pushes expansion into areas that may not be executable — the Proposer knows which APIs are under-used, but not whether those APIs can be composed into feasible tasks in the current environment.

Environment information alone (Validator + Env Info, row 4) pulls in the opposite direction: it anchors proposals in executable facts, producing the lowest infeasible task rate (5.3% — a 6× reduction from the history-only variant). But this grounding comes at the cost of coverage: unique APIs drop to 42.3 (the lowest among all variants), and weighted recall collapses to 0.368. The Proposer stays within what it knows is executable, but doesn't explore.

Full PREPING (row 5) combines both signals, achieving 87.0 unique APIs (even higher than history alone), 5.919 tool entropy (highest), and 0.703 weighted recall (highest), while keeping the infeasible rate at a moderate 16.0%. This is not simply additive — the coverage metrics exceed the history-only variant while the infeasible rate is less than half. The mechanism, visible in the qualitative example in Appendix B.4, is that environment information enables broader coverage by providing the Proposer with the grounded entities it needs to construct feasible tasks in under-explored areas. The Gmail task in that example expands into an under-practiced app (Gmail) by referencing grounded entities (the user's credentials, the show_inbox_threads API with attachment filtering) discovered during earlier practice. Without environment information, the Proposer might try to expand into Gmail by inventing unsupported operations (similar to the song ID 555 failure); with environment information, it expands feasibly.

This finding matters beyond PREPING because it identifies a general tension in synthetic practice design: exploration without grounding produces infeasible proposals; grounding without exploration produces narrow coverage. Any system that generates its own practice tasks — whether for memory construction, policy updates, or curriculum learning — must balance these pressures. PREPING's dual-view proposer memory provides a design pattern for doing so, and the entropy/coverage/infeasibility diagnostics in Table 2 provide a measurement framework for evaluating the balance.

Innovation 5: Pre-Task Memory as a Strong Initialization for Online Adaptation

The paper's final conceptual contribution is demonstrating that pre-task and online memory construction are complementary, not competing, phases. PREPING+ACE (Table 3) combines broad pre-deployment coverage from synthetic practice with task-specific refinement from online experience, improving ACE-Online's AppWorld average from 71.3 to 76.3. This is not just "PREPING is a better starting point than empty memory" — although it is that. The deeper insight is that pre-task memory provides coverage that online memory may never achieve on its own.

The right panel of Figure 1 makes this visible: ACE-Online's tool coverage grows slowly during deployment, reaching PREPING's pre-deployment level only after 58 tasks on AppWorld and never catching up on BFCL v3. This means that ACE-Online is perpetually playing catch-up — by the time it has seen enough tasks to cover all the tools, the deployment may be over, and in the BFCL v3 case, the user task distribution simply doesn't exercise enough tools to ever close the gap. PREPING+ACE breaks this dependency: synthetic practice provides broad coverage upfront, and online updates provide task-specific refinement on whichever subset of tools users actually exercise.

This changes the value proposition of pre-task memory. It is not just a cheaper alternative to online memory (though Figure 5 shows it is). It is a complementary capability that addresses a structural limitation of online-only approaches: the coverage of online memory is bounded by the distribution of user tasks, which may not be representative or complete. Synthetic practice can deliberately target under-covered tools regardless of what users ask for, making the memory more robust to distribution shift and long-tail tasks.

The practical implication is that the optimal deployment strategy may not be "offline vs. online" but "pre-task warm-start + online refinement." The pre-task phase handles broad coverage; the online phase handles task-specific adaptation. This two-phase lifecycle — prepare broadly before deployment, refine selectively during deployment — mirrors how human experts develop: study broadly before practice, then specialize through experience. The paper doesn't push this analogy, but it provides an empirical foundation for it.

5. Experimental Analysis

Evaluation Methodology

Dataset. All experiments use three complementary agent benchmarks. AppWorld (Trivedi et al., 2024) tests stateful application workflows where agents write Python code against app APIs (e.g., Spotify, Venmo) and are scored by a state-based evaluator. The paper reports on Test-Normal (N), a held-out split drawn from the same distribution as the offline training split, and Test-Challenge (C), a harder split whose tasks require at least one unseen app. BFCL v3 (Patil et al., 2025) tests executable function calling under schema and dialogue constraints; results are reported on Base, Long Context (Ctx.), Missing Parameter (Para.), and Missing Function (Func.) categories. MCP-Universe (Luo et al., 2025) tests tool use over real Model Context Protocol servers with heterogeneous tool inventories and execution-based evaluators, covering Repository Management (Repo.), Financial Analysis (Fin.), 3D Designing (3D.), and Browser (Brow.) categories.
Base model(s). All components — the Proposer, Solver, Validator, and memory-update calls — use DeepSeek-V3.2 (DeepSeek-AI, 2025) without reasoning mode as the backbone LLM. The same model is also used across all baselines for task execution and memory construction to isolate the effect of the construction method rather than model capability. Backbone generalization experiments in Table 4 additionally test GPT-5.1, GPT-OSS-120B, and Qwen3-235B-A22B, all with reasoning-disabled modes, using the same model for both construction and evaluation in each run.
Metrics. AppWorld uses Task Goal Completion (TGC), the percentage of tasks for which all evaluation tests pass, and Scenario Goal Completion (SGC), which credits a scenario only when all of its task variants are solved. The main table reports N-TGC, N-SGC, C-TGC, and C-SGC. BFCL v3 uses per-category success rate (the fraction of function-calling tasks completed correctly). MCP-Universe uses per-category success rate based on execution-based evaluation. The main results are averaged over three independent construction-and-evaluation runs.
Baselines. Pre-task baselines (no target-environment task data allowed) include: Base — no constructed memory, the agent solves tasks directly from the model and environment context; Direct Memory — constructs memory from environment documentation without execution, by sampling and combining subsets of API or tool documentation into the ACE reflector–curator memory induction pipeline; Random Exploration — constructs memory from free-form environment interaction without task-level objectives, with the agent prompted to explore without additional constraints; Guided Exploration — conditions exploration on prior exploration history to encourage under-explored APIs or tools; and Naive Task Generation — an ablation variant without Validator, environment information, or practice history (equivalent to generating synthetic tasks and storing all trajectories without filtering). Task-informed baselines (allowed access to target-environment task data, serving as reference points) include: ACE-Offline (Zhang et al., 2026) — constructs memory before deployment from human-defined target tasks and their execution feedback; and ACE-Online (Zhang et al., 2026) — constructs memory during deployment from user tasks as they arrive, starting from empty memory. ACE-Offline is evaluated only on AppWorld, the only benchmark that provides a training split. All memory-construction methods use the same ACE reflector–curator memory induction pipeline, isolating whether memory is induced from documentation, free-form exploration, or validated synthetic-task practice.
Generation budget / compute accounting. All pre-task and exploration-based baselines use 100 construction trajectories for memory induction in the main experiments (10 iterations × 10 synthetic tasks for PREPING; 100 exploration trajectories for Random and Guided Exploration). The exception is AppWorld exploration, where one trajectory is generated from each of the 90 train-task environments. For the construction budget analysis in Figure 4, the number of synthetic tasks is varied from 0 to 300. Deployment-time cost (Figure 5) is measured in USD per task using DeepSeek-V3.2 API pricing ( $0.028 per 1M cache-hit input tokens,$ 0.28 per 1M cache-miss input tokens, $0.42 per 1M output tokens) and separates task-solving cost from memory-update cost. Construction-inclusive cost is reported in Appendix B.7 (Table 12).
Cross-validation / statistical protocol. The main results are averaged over three independent construction-and-evaluation runs, each using a freshly constructed memory and independent downstream evaluation. Standard deviations are reported in Appendix B.1 (Tables 6–8). The Validator assigns 1–5 Likert scores for feasibility and completion. A synthetic trajectory is admitted into solver memory only when its feasibility score is 5 (the strictest threshold), while task-completion scores of 4 or higher are treated as successful execution for the purpose of extracting procedural lessons. Temperature is set to 1.0 for synthetic task proposal (to encourage diversity), 0 for Solver execution (to maximize reproducibility), and 0.7 for Validator judgments and memory updates (to support nuanced scoring).

Main Quantitative Results

Overall Pre-Task Performance

Table 1 reports the main results comparing pre-task methods (no target-task data) against task-informed reference methods. PREPING achieves the strongest pre-task performance across all three benchmarks, improving average score over Base by 17.1 points on AppWorld (from 53.1 to 70.2), 19.3 points on BFCL v3 (from 33.8 to 53.1), and 5.4 points on MCP-Universe (from 32.1 to 37.5).

Breaking down AppWorld: on Test-Normal, PREPING achieves 83.7 TGC and 70.2 SGC, compared to Base at 69.6/49.4 and the next-best pre-task method (Guided Exploration) at 74.6/56.0. On Test-Challenge, PREPING achieves 72.2 TGC and 54.7 SGC, compared to Base at 56.7/36.7. These gains are substantial: PREPING closes much of the gap to task-informed methods despite using no human-defined tasks, exceeding ACE-Offline (82.7/69.1 on Normal, 69.0/50.3 on Challenge) and approaching ACE-Online (80.4/65.5 on Normal, 78.3/60.9 on Challenge). Notably, on Test-Challenge TGC and SGC, ACE-Online outperforms PREPING (78.3 vs. 72.2 TGC, 60.9 vs. 54.7 SGC), reflecting the value of task-informed memory on harder, out-of-distribution tasks — but PREPING's advantage on Test-Normal (83.7 vs. 80.4 TGC) shows that pre-task memory can even exceed online memory performance on in-distribution tasks.

On BFCL v3, PREPING achieves 65.2 Base, 59.3 Long Context, 37.8 Missing Parameter, and 50.2 Missing Function, averaging 53.1. This surpasses ACE-Online on average (51.6) and substantially exceeds the next-best pre-task method, Random Exploration (46.1). The BFCL results reveal an interesting pattern: Random Exploration outperforms Guided Exploration (46.1 vs. 43.9), which is the opposite of the AppWorld pattern. The paper does not explicitly analyze this reversal, but it may reflect BFCL's different structure — function calling with defined schemas may benefit more from diverse random probing than from coverage-guided exploration that could overfit to schema documentation patterns.

On MCP-Universe, the gains are more modest but consistent: PREPING achieves 37.5 average versus Base at 32.1 and the next-best pre-task method (Guided Exploration) at 34.6. The Financial category shows the largest improvement (70.8 vs. 59.2 Base), while Repository Management is essentially flat (10.1 vs. 8.1 Base). This benchmark shows the most variability across categories and the smallest overall gains, suggesting that pre-task memory construction is harder when the tool inventory is heterogeneous and server-specific.

Comparison Against Execution-Based Exploration Baselines

The exploration baselines in Table 1 test whether task-level objectives are necessary for useful memory, or whether free-form environment interaction suffices. The results show that exploration alone provides limited gains:

On AppWorld average: Random Exploration improves over Base by only 1.6 points (54.7 vs. 53.1), Guided Exploration by 2.5 points (55.6 vs. 53.1). These are marginal improvements despite 100 exploration trajectories of live environment interaction.
On BFCL v3 average: Random Exploration provides a larger gain of 12.3 points (46.1 vs. 33.8), suggesting that BFCL's function-calling environment yields more useful signal from unstructured exploration. Guided Exploration underperforms Random Exploration on BFCL (43.9 vs. 46.1), which the paper does not fully explain but may indicate that coverage guidance on BFCL produces less useful interaction patterns.
On MCP-Universe average: Random Exploration (33.1) and Guided Exploration (34.6) provide modest improvements over Base (32.1).

The gap between exploration baselines and PREPING is consistently large: 14.6 points on AppWorld average, 7.0–9.2 points on BFCL v3, and 2.9–4.4 points on MCP-Universe. This demonstrates that execution feedback alone is not sufficient — the task-level structure imposed by proposer-guided synthetic practice is essential for producing trajectories that can be distilled into reusable procedural guidance.

Component Ablation: What Drives the Gains

Table 2 decomposes PREPING's performance by ablating three components — Validator, environment information, and practice history — on AppWorld Test-Normal and BFCL v3 Base. The rows are cumulative: row 1 disables all three (Naive Task Generation), row 2 adds Validator only, row 3 adds Validator + History, row 4 adds Validator + Env Info, and row 5 is full PREPING (all three enabled). The table reports both downstream task performance and four construction-side diagnostics: infeasible task rate (%), unique tools count, tool entropy (in bits), and weighted recall over test-time tools.

The headline finding: validation-gated memory admission is the single largest contributor to performance, but proposer-side control is necessary to achieve the best results.

On AppWorld Test-Normal: Naive Task Generation produces catastrophic performance (47.8/26.8 TGC/SGC), well below the Base baseline of 69.6/49.4. This is the clearest evidence that storing synthetic trajectories without filtering is worse than storing nothing. Adding only the Validator admission gate (row 2) recovers to 78.2/60.7 — a gain of over 30 points in TGC from a single component. This is the largest single effect in any ablation. Adding practice history to the Validator (row 3) further improves to 81.5/67.9, and adding environment information to the Validator (row 4) reaches 80.3/64.9. Full PREPING with all three (row 5) achieves 83.7/70.2.

The construction-side diagnostics reveal the mechanisms behind these performance shifts. The infeasible task rate is unavailable for Naive (no validator labels exist), but is 26.3% for Validator-only and rises to 33.3% when practice history is added — because history pushes the Proposer toward under-covered tools, increasing the risk of proposing infeasible tasks. Environment information dramatically reduces infeasibility to 5.3% (row 4), the lowest across all variants, by grounding proposals in observed entities and states. Full PREPING balances these pressures at 16.0%.

Unique tool coverage follows the opposite pattern: Validator-only achieves 69.0 unique domain APIs; adding practice history pushes this to 81.7 (the highest except full PREPING) by encouraging exploration of under-covered tools; adding only environment information collapses coverage to 42.3 (the lowest) because grounding without history creates narrow, repetitive proposals. Full PREPING achieves the highest unique API count (87.0) by combining history-driven expansion with environment-grounded feasibility.

Tool entropy and weighted recall follow similar patterns: both are highest in full PREPING (5.919 and 0.703 on AppWorld), confirming that the dual-pressure mechanism produces not just broader but more evenly distributed practice coverage that better matches the tools used in downstream tasks.

On BFCL v3 Base, the pattern is qualitatively similar but with smaller magnitudes. Naive achieves 59.5 (not below Base, unlike AppWorld — BFCL's function calling environment appears more forgiving of unfiltered memory). Validator-only improves to 62.0. Practice history alone with Validator shows a slight regression (60.8) — because BFCL has a large function space (hundreds of available functions across servers), and history-driven expansion without environment grounding may produce too many infeasible or poorly-grounded tasks. Environment information with Validator improves to 64.0, and full PREPING reaches 65.2. On BFCL, unique tools peaks at 118.0 when practice history is enabled (row 3), and full PREPING achieves 105.0 with the highest entropy (6.095) — slightly fewer unique tools but better distributed practice.

Validator signal ablation (Table 9, Appendix B.3). Beyond the admission gate, the Validator's structured output (feasibility/completion labels and rationales) provides two additional signals: one routed to solver-memory updates (where it serves as task-completion feedback for the reflector–curator pipeline) and one routed to proposer memory (where it labels tasks as solved, failed, or infeasible for future proposal guidance). Table 9 shows that removing the Validator signal from solver-memory updates reduces AppWorld Test-Normal performance from 83.7/70.2 to 81.9/66.7, and removing it from proposer memory reduces performance to 82.5/66.1. These drops are much smaller than removing the admission gate entirely (which collapses to 47.8/26.8), confirming that the gate is the primary mechanism, but the structured signals provide non-trivial additional gains.

Iteration Dynamics of Component Ablations

Figures 16 and 17 (Appendix B.2) show how the component-ablation variants evolve over the ten construction iterations. Three diagnostics are tracked: cumulative invalid (infeasible) tasks, cumulative unique tools/APIs, and cumulative tool/API entropy. The curves reveal:

On AppWorld (Figure 16): Naive (no Validator) has no invalid-task curve because feasibility labels are absent. Validator-only accumulates infeasible tasks steadily across iterations, reaching roughly 25 by iteration 10. Adding practice history (Val. + Hist.) increases the infeasible accumulation rate — by iteration 10, it reaches roughly 33 invalid tasks, the highest among all variants. Adding environment information (Val. + Env.) nearly eliminates infeasible tasks, accumulating only about 5 by iteration 10. Full PREPING accumulates intermediate infeasibility (roughly 16 by iteration 10) while achieving the fastest growth in unique APIs and the highest entropy. On BFCL (Figure 17): The invalid-task accumulation is lower overall (BFCL tasks are easier to ground in function schemas), and the unique-tool growth for history-enabled variants is dramatic — from roughly 50 to 118 over 10 iterations — reflecting BFCL's large function space.

The key dynamic insight: practice history accelerates coverage expansion but also accelerates infeasibility; environment information suppresses infeasibility but also suppresses coverage; the full PREPING achieves the best of both by coupling the two signals. The entropy curves show that PREPING's practice distribution also becomes more uniform over iterations, while component-ablated variants show more entropy fluctuation or plateauing.

PREPING as an Initialization for Online Memory Construction

Table 3 reports PREPING+ACE: initializing ACE-Online with PREPING's pre-task memory and then applying the standard online update procedure during evaluation. On AppWorld, PREPING+ACE improves average performance from 71.3 (ACE-Online) to 76.3. The improvement is especially clear on AppWorld Test-Challenge: TGC increases from 78.3 to 80.1 and SGC from 60.9 to 65.2. On Test-Normal, TGC improves from 80.4 to 86.1 and SGC from 65.5 to 73.8. On BFCL v3, the same initialization improves Base from 62.3 to 66.0 and Long Context from 55.0 to 63.7, raising the average from 58.7 to 64.9.

The key interpretation is that PREPING provides coverage that online memory may never achieve on its own, and combining broad synthetic coverage with task-specific online refinement yields the best overall performance. This is supported by the tool-coverage analysis in Figure 1 (right panel), which shows ACE-Online needs 58 evaluation tasks to match PREPING's pre-deployment coverage on AppWorld and still falls short after 200 tasks on BFCL v3.

Cold-Start Failure Analysis

Figure 3 examines the early deployment regime before ACE-Online has accumulated sufficient task-informed memory. The analysis constructs 18 shuffled 30-task streams from AppWorld Test-Normal and measures cumulative success over the first $k$ tasks in each stream. The curves show:

ACE-Online's first-10-task success rate is 74.4%, substantially below its full Test-Normal TGC of 80.4% and close to the no-memory Base at 69.6%. This is the quantified cold-start gap.
PREPING starts with pre-task memory and achieves 79.4% over the first 10 tasks, eliminating most of the cold-start gap.
PREPING+ACE starts at 82.2% over the first 10 tasks and maintains a consistent advantage over both ACE-Online and frozen PREPING throughout the 30-task stream.

The advantage persists through the full stream: at 30 tasks, PREPING+ACE achieves roughly 85%, PREPING roughly 84%, and ACE-Online roughly 80%. This shows that pre-task memory not only prevents early failures but provides a durable advantage that online refinement amplifies rather than erases.

Tool-Coverage Cold Start

Figure 1 (right panel) quantifies why online memory suffers from cold-start: tool-coverage latency. On AppWorld Test-Normal, ACE-Online starts with empty memory and accumulates tool coverage only from evaluation tasks; it requires 58 tasks to match PREPING's pre-deployment coverage (87 unique tools). On BFCL v3 Base, ACE-Online still falls short of PREPING's coverage (101 unique tools) after processing all 200 evaluation tasks, plateauing around 90. PREPING+ACE starts from PREPING's coverage and further expands it with user-task updates, reaching coverage levels that neither frozen PREPING nor ACE-Online achieves alone.

This provides a mechanistic explanation for the cold-start gap: online memory suffers not just from lack of experience volume, but from a coverage distribution that is bounded by the user task distribution. If user tasks don't naturally exercise certain tools or APIs, online memory never learns about them — a structural limitation that pre-task synthetic practice can address by deliberately targeting under-covered tools.

Construction Budget Scaling

Figure 4 varies the number of synthetic tasks used for PREPING construction on AppWorld Test-Normal TGC. The curve shows:

At 30 synthetic tasks: PREPING reaches 76.6 TGC, already surpassing Guided Exploration (74.6).
At 50 synthetic tasks: PREPING reaches 80.0, approaching the ACE-Online reference at 80.6.
At 100 synthetic tasks (the default): 83.7.
At 300 synthetic tasks: 84.3, with diminishing marginal returns.

The key practical implication is that modest synthetic-task budgets already yield strong gains — 30 tasks is sufficient to exceed all pre-task and exploration baselines, and 50 tasks approaches online-method performance. This makes the one-time pre-deployment cost manageable while still delivering significant downstream benefits.

Deployment-Time Cost Reduction

Figure 5 compares deployment-time cost per task for PREPING (frozen memory) versus ACE-Online (memory updated during deployment). The costs separate task-solving API calls from memory-update API calls. On AppWorld, ACE-Online costs approximately 2.99× more per task than PREPING (driven by memory-update calls during deployment). On BFCL v3, ACE-Online costs approximately 2.23× more. This cost comparison excludes the one-time pre-deployment construction cost of PREPING.

When construction-inclusive cost is amortized across all evaluation tasks (Appendix B.7, Table 12), PREPING remains cheaper than ACE-Online by 2.83× on AppWorld and 1.97× on BFCL v3. The savings come from two sources: the constructed memory is reusable across many evaluation tasks (585 AppWorld tasks, 200+ BFCL tasks), and synthetic practice tasks require fewer trajectory steps than benchmark tasks (9.5 vs. 19.1–24.3 steps on AppWorld, 3.5 vs. 9.4–13.1 steps on BFCL v3, as shown in Tables 10 and 11), reducing the total inference cost of construction relative to accumulating the same coverage from user tasks.

Generalization Across Backbone Models

Table 4 tests whether PREPING transfers across backbone models. On AppWorld Test-Normal, the full construction-and-evaluation pipeline is run with GPT-5.1, GPT-OSS-120B, and Qwen3-235B-A22B, with the same backbone instantiating all components. PREPING improves over Base across all three backbones: from 52.4/30.4 to 57.7/41.1 for GPT-5.1, from 28.0/7.1 to 36.9/17.9 for GPT-OSS-120B, and from 58.3/37.5 to 67.3/44.6 for Qwen3-235B-A22B. In all cases, PREPING remains comparable to or exceeds task-informed methods (ACE-Offline and ACE-Online), despite using no human-defined tasks during construction. The improvement holds for GPT-OSS-120B, the weakest backbone, indicating that PREPING benefits agents even when the base solver is less capable. This rules out the hypothesis that the main results are a DeepSeek-V3.2-specific artifact.

Task-Seeded PREPING

Table 5 evaluates a variant where PREPING starts from a small set of 10 offline tasks (sampled from the AppWorld training split) rather than from empty memory. Task-Seeded PREPING initializes proposer memory from the sampled task instructions and solver memory from their corresponding trajectories, then runs the standard iterative PREPING loop. It improves AppWorld Test-Normal TGC/SGC from 83.7/70.2 to 85.1/73.8, showing that a small seed set can further anchor synthetic practice without undermining the pre-task setting's spirit (no human task design required for the target environment). This variant bridges the gap between fully task-free construction and offline methods that require large curated task sets.

Ablation Studies and Robustness Checks

Naive Task Generation (no Validator, no proposer memory components): This disables all three components — Validator, environment information, and practice history — and simply generates synthetic tasks, executes them, and stores all trajectories in solver memory without filtering. Table 2 (row 1) shows this achieves 47.8/26.8 on AppWorld Test-Normal, which is below the no-memory Base baseline (69.6/49.4). This is the most important single ablation because it demonstrates that unfiltered synthetic practice produces memory that is actively harmful — worse than having no memory at all. On BFCL v3, Naive achieves 59.5 on Base, which is above the no-memory baseline (43.7), indicating that BFCL's function-calling environment is less sensitive to memory contamination than AppWorld's stateful app environment. The key implication: the need for validation gating is environment-dependent, and in stateful environments with complex preconditions, infeasible trajectory storage is catastrophic.

Validator admission gate only (no proposer-side control): Table 2 (row 2) adds the Validator's feasibility gating (only score-5 tasks enter solver memory) but keeps proposer memory empty — the Proposer generates tasks from documentation alone, without practice history or environment information. On AppWorld, this recovers performance to 78.2/60.7 — a gain of 30.4 points in TGC from a single component. On BFCL v3, the improvement is 2.5 points (59.5 to 62.0). The infeasible task rate is 26.3% on AppWorld and 9.0% on BFCL, indicating that a substantial fraction of naive proposals are infeasible even with schema documentation available. This establishes validation gating as the single most impactful component in PREPING.

Practice history added to Validator (no environment information): Table 2 (row 3) adds practice history to the Proposer's context, enabling coverage-aware task proposal without environment grounding. On AppWorld, performance improves to 81.5/67.9 and unique APIs jump from 69.0 to 81.7 — the highest except full PREPING. However, the infeasible task rate also rises to 33.3% (the highest among all variants), because history pushes expansion into under-covered areas without ensuring those areas are executable. On BFCL v3, this variant regresses from 62.0 to 60.8 — the only case where adding a proposer-side component reduces performance. This occurs because BFCL's large function space makes history-driven expansion particularly risky without environment grounding: the Proposer proposes tasks targeting rarely-used functions for which the environment lacks supporting entities. This non-obvious result — that history alone can hurt performance on some benchmarks — demonstrates that coverage expansion and feasibility are genuinely in tension.

Environment information added to Validator (no practice history): Table 2 (row 4) adds grounded environment observations to the Proposer's context without practice history. On AppWorld, performance reaches 80.3/64.9 — slightly below the history-only variant (81.5/67.9). The infeasible task rate drops dramatically to 5.3% — the lowest across all variants — confirming that environment grounding effectively eliminates infeasible proposals. However, unique APIs collapse to 42.3 (the lowest) and weighted recall to 0.368 (half the full PREPING value), because without practice history, the Proposer stays within what it knows to be executable but does not explore. On BFCL v3, this variant achieves 64.0, the best score before full PREPING, because BFCL's function-calling environment benefits more from feasibility anchoring than from aggressive coverage expansion. This asymmetry — environment information being more valuable on BFCL, practice history being more valuable on AppWorld — reveals that the optimal balance of proposer-memory components depends on the environment's structure.

Full PREPING (Validator + environment information + practice history): Table 2 (row 5) achieves the best performance on both benchmarks: 83.7/70.2 on AppWorld and 65.2 on BFCL v3. Unique APIs reach 87.0 on AppWorld and 105.0 on BFCL, tool entropy peaks at 5.919 and 6.095 respectively, and weighted recall reaches 0.703 and 0.846. The infeasible task rate is 16.0% on AppWorld and 4.0% on BFCL — intermediate between the history-only and env-info-only extremes, demonstrating that the two pressures balance each other. This is not simply additive: the coverage metrics exceed the history-only variant while the infeasible rate is less than half, and the entropy exceeds either single-signal variant alone.

Validator signal in solver-memory updates (Table 9, Appendix B.3): Beyond the admission gate, the Validator's structured output (completion labels, rationales) is used as ground-truth feedback for the ACE reflector–curator pipeline. Table 9 shows that removing this signal from solver-memory updates reduces AppWorld Test-Normal from 83.7/70.2 to 81.9/66.7. The drop in SGC (3.5 points) is larger than the drop in TGC (1.8 points), suggesting that the Validator's signal particularly improves the memory's robustness across scenario variants.

Validator signal in proposer memory updates (Table 9, Appendix B.3): Removing validator-derived labels (solved/failed/infeasible) and failure/infeasibility reasons from proposer memory reduces performance to 82.5/66.1. The drop is comparable in magnitude to removing the solver-memory signal, indicating that both signal paths contribute modest but distinct additional gains beyond the admission gate.

Construction budget scaling (Figure 4): Varying synthetic task count from 0 to 300 shows monotonic improvement with diminishing returns. The 30-task budget already achieves 76.6 TGC, exceeding Guided Exploration; 50 tasks reaches 80.0, approaching ACE-Online; 100 tasks achieves 83.7; 300 tasks reaches 84.3 with small marginal gains. This confirms that the reported 100-task budget is a reasonable operating point.

Backbone model generalization (Table 4): PREPING improves over Base across three different backbone models (GPT-5.1, GPT-OSS-120B, Qwen3-235B-A22B), with gains of 5.3, 8.9, and 9.0 TGC points respectively. The gains are not uniform: GPT-OSS-120B (the weakest backbone) shows the largest relative improvement, suggesting that weaker models benefit more from procedural memory guidance. All three backbones show PREPING remaining competitive with task-informed methods.

Task-Seeded PREPING (Table 5): Starting with 10 offline tasks further improves TGC from 83.7 to 85.1 and SGC from 70.2 to 73.8, showing that a small seed set can provide additional anchoring without undermining the zero-human-task-design benefit.

Trajectory step comparison (Tables 10–11, Appendix B.6): PREPING's synthetic tasks require substantially fewer trajectory steps than benchmark tasks: 9.5 vs. 19.1 (Test-Normal) and 24.3 (Test-Challenge) on AppWorld; 3.5 vs. 9.4–13.1 on BFCL v3. This confirms that PREPING's coverage and performance are not artifacts of collecting more interaction steps, but of targeted practice.

Critical Assessment

Claim 1: PREPING builds effective pre-task memory without any target-environment task data. This claim is strongly supported by Table 1, which shows consistent gains over Base across three benchmarks (17.1 points on AppWorld, 19.3 on BFCL v3, 5.4 on MCP-Universe). The improvements are statistically robust (standard deviations in Tables 6–8 are small relative to the gaps) and generalize across backbone models (Table 4). However, the magnitude of improvement varies substantially: BFCL v3 and AppWorld show large gains, while MCP-Universe shows modest gains (5.4 points). The paper does not deeply analyze why MCP-Universe gains are smaller. One possibility is that MCP-Universe's heterogeneous server landscape makes coverage-based memory less useful because tools across servers don't compose into reusable workflows — they are independently useful but not procedurally composable. This limits the generalizability claim: pre-task memory may be most valuable when environments support compositional tool use (stateful apps, function calling chains) and less valuable when tools are independently invoked.

Claim 2: The gains come from proposer-side control and selective memory updates, not from synthetic volume alone. This claim is strongly supported by Tables 2 and 9 and Figures 4, 16, and 17. Naive Task Generation (equivalent to "synthetic volume alone") produces memory worse than none on AppWorld, confirming that volume without control is harmful. The interactive effect between practice history and environment information — where history alone increases infeasibility, env info alone suppresses coverage, and the combination achieves the best of both — is empirically demonstrated with clear diagnostics (infeasible rate, unique tools, entropy, weighted recall). The $4\times$ efficiency gain claimed in the executive summary (relative to online methods) is supported by Figure 5 and Table 12, though the construction-inclusive savings are smaller (2.83× and 1.97× on AppWorld and BFCL) than the deployment-time-only savings (2.99× and 2.23×), which is a distinction readers should note.

Claim 3: PREPING achieves performance competitive with strong playbook-based methods built from offline or online experience. This claim is supported with conditions. On AppWorld Test-Normal, PREPING (83.7/70.2) exceeds ACE-Offline (82.7/69.1) and ACE-Online (80.4/65.5). On Test-Challenge, PREPING (72.2/54.7) exceeds ACE-Offline (69.0/50.3) but trails ACE-Online (78.3/60.9), indicating that online memory's task-specific refinement is more valuable for out-of-distribution tasks. On BFCL v3 average, PREPING (53.1) edges out ACE-Online (51.6), though this average masks category-level variation. On MCP-Universe, PREPING (37.5) trails ACE-Online (41.2), consistent with the pattern that pre-task memory is weaker when tool heterogeneity limits compositional reuse. The "competitive" claim holds broadly but is environment-dependent: PREPING is strongest when tasks involve composing known tools in novel ways (AppWorld Test-Normal, BFCL) and weaker when tasks require genuinely out-of-distribution tool use (AppWorld Test-Challenge) or when tools don't compose into reusable workflows (MCP-Universe).

Claim 4: PREPING reduces deployment-time cost by 2.99× on AppWorld and 2.23× on BFCL v3 relative to ACE-Online. This claim is supported by Figure 5, but the comparison is somewhat narrow. The cost comparison considers only the deployment phase — the one-time construction cost of PREPING is excluded from the per-task figure. When construction cost is included (Table 12), the savings drop to 2.83× and 1.97×, which is still substantial but lowers the headline ratio. Additionally, the cost comparison is specific to DeepSeek-V3.2 API pricing; if a deployment uses a different model with different pricing, the ratios would change. More importantly, the comparison assumes that online memory construction must update memory on every task — a pessimistic assumption. In practice, ACE-Online could batch memory updates or update less frequently, reducing the cost ratio. The paper does not explore cheaper online update schedules.

Weaknesses and Missing Experiments:

Single backbone family for main results. The main experiments (Tables 1, 2, 3, 5) all use DeepSeek-V3.2. Table 4 demonstrates transfer across backbones, but only on AppWorld Test-Normal and only for the final PREPING configuration, not the full ablation suite. It is possible that the relative importance of Validator gating versus proposer-side control differs across model families (e.g., stronger models might generate fewer infeasible proposals, reducing the Validator's relative contribution). The paper does not explore this.

Difficulty estimation and validation calibration. The Validator uses a 1–5 Likert scale with a threshold of feasibility = 5 for admission. The paper does not report inter-rater reliability (how often do different LLM calls produce different feasibility judgments for the same task–trajectory pair?), calibration analysis (do feasibility-5 tasks actually have higher downstream success rates than feasibility-4 tasks?), or threshold sensitivity (what happens if the threshold is 4 instead of 5?). These are important because the strict threshold discards many synthetic trajectories — on AppWorld, 16% of tasks are infeasible even with full PREPING, and the Validator-only variant discards 26.3%. A threshold of 4 might admit more trajectories (increasing memory coverage) at an acceptable quality cost, but this is not tested.

No combination of exploration baselines with validation gating. The paper's exploration baselines (Random and Guided Exploration) use the same ACE reflector–curator pipeline as PREPING, but do not apply a Validator gate to their trajectories. This means the comparison between exploration and PREPING confounds two factors: task-level objectives vs. free-form exploration, AND validated admission vs. unfiltered admission. A fairer comparison would apply the same Validator to exploration trajectories and only admit feasible ones into memory. The paper does not run this experiment, so it's unclear whether the exploration baselines' poor performance is due to lack of task-level structure, lack of validation gating, or both.

No comparison to simply generating many more synthetic tasks without proposer control. Figure 4 shows PREPING scaling with synthetic task count, but there is no corresponding curve for Naive Task Generation at different budgets. It is possible that generating, say, 500 synthetic tasks without proposer control but with validation gating could match PREPING's performance at a fraction of its design complexity. The paper argues that proposer-side control is necessary, but the evidence for this is the 100-task budget comparison in Table 2 — at higher budgets, the proposer-side advantage might shrink or disappear because random coverage eventually saturates. The paper doesn't test this.

No analysis of memory contamination in the medium-to-high infeasibility regime. The Naive ablation (row 1, Table 2) demonstrates the extreme case (no filtering at all, 47.8 TGC). The appendix B.5 shows a qualitative example of contamination. But the paper does not explore the intermediate regime: what if 5%, 10%, or 20% of memory comes from infeasible trajectories? This is practically important because no Validator is perfect, and understanding how contamination scales with admission threshold would guide practical threshold selection.

MCP-Universe results are weak and underexplained. PREPING's gain on MCP-Universe (5.4 points) is substantially smaller than on the other benchmarks, and the per-category standard deviations in Table 8 are large (e.g., Repository: 10.1 ± 1.4 for PREPING vs. 8.1 ± 2.8 for Base — the confidence intervals overlap). The paper does not analyze why MCP-Universe is harder or whether the Validator, Proposer, or memory induction pipeline is the bottleneck. This limits confidence that PREPING generalizes to environments qualitatively different from AppWorld and BFCL.

No latency/wall-clock analysis. The paper claims deployment-time cost reduction (Figure 5) but does not discuss whether PREPING's pre-constructed memory affects task-solving latency. If the playbook memory is large (the paper does not report memory size in tokens), it could increase prompt length and therefore per-task inference time. ACE-Online might have larger memory (since it accumulates from many user tasks), but PREPING might have broader but less task-specific memory, which could affect retrieval efficiency if the memory format were structured for retrieval (it is not — the entire playbook is inserted into the prompt). This tradeoff is unexplored.

Single memory format (ACE playbook). All methods use the same ACE reflector–curator pipeline and playbook format. This is a strength for isolating the practice distribution's effect, but it means the results don't distinguish whether PREPING's gains would transfer to other memory formats (e.g., retrieved exemplars, vector-store-based memory, workflow-graph memory). The paper's claims are implicitly about procedural memory construction in this specific format, not about agent memory construction in general.

Difficulty binning not applied to pre-task setting. Unlike the reference paper that analyzes performance by difficulty quintile, PREPING does not analyze how its gains vary with task difficulty. This is partly because pre-task tasks don't have pre-defined difficulty, but the paper could bin deployment tasks by Base success rate (analogous to the reference paper's oracle difficulty bins) to see whether PREPING helps more on easy, medium, or hard tasks. This would reveal whether pre-task memory primarily addresses easy tasks (providing basic procedural knowledge) or also helps on hard tasks (providing sophisticated compositional strategies).

No systematic analysis of Validator errors. The Validator is an LLM making Likert-scale judgments about feasibility and completion. The paper does not report how often the Validator makes errors — admitting infeasible trajectories (false positives) or rejecting feasible ones (false negatives). A human evaluation of Validator accuracy on a sample of trajectories would substantially strengthen confidence in the admission gating mechanism. Without this, the claim that "feasibility = 5" is a reliable filter rests on the downstream performance improvement, which is indirect evidence.

6. Limitations and Trade-offs

The Difficulty Estimation Cost Is Not Accounted for in Headline Efficiency Numbers

The assumption or constraint. PREPING's entire compute-optimal framework depends on estimating prompt difficulty before allocating the inference budget. The paper's method for doing so — generating 2048 samples per question and averaging PRM final-answer scores — is extraordinarily expensive, costing more than the largest test-time compute budgets studied. The authors acknowledge this explicitly in Section 3.2:

"estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity"

The consequence. The reported 4× efficiency gains over best-of-N are computed after difficulty is known, without amortizing the cost of learning it. In a realistic deployment, the difficulty estimation step alone could consume more compute than the entire problem-solving budget. For example, on a prompt where the optimal strategy uses 16 generations, the difficulty estimation requires 2048 generations — a 128× overhead that completely negates any efficiency gain for a single query. The efficiency numbers are therefore best understood as an upper bound on what the method can achieve, not a realized deployment gain. Unless difficulty can be estimated much more cheaply (the paper suggests training a model to predict difficulty from question text, Section 8, but does not evaluate this), the compute-optimal framework remains an analytical contribution rather than a directly deployable system.

What evidence exists in the paper. Section 3.2 describes the difficulty estimation procedure (2048 samples, either oracle-correctness or PRM-scored) and explicitly flags the cost. Figure 4 and Figure 8 show that predicted (non-oracle) difficulty bins perform nearly as well as oracle bins, confirming that ground-truth labels are not required — but the generation cost of 2048 samples per prompt is not reduced by this substitution. The paper does not report a complexity analysis or cost breakdown that includes difficulty estimation overhead in any of its budget calculations.

Mitigation status. The paper explicitly frames this as future work (Section 8), suggesting pretraining or fine-tuning a model to directly predict difficulty from question text. The paper does not attempt any cheap difficulty estimation approach, such as using only a handful of samples (e.g., 4–8) with the PRM's score distribution as a quick signal, or training a lightweight classifier. The limitation is therefore not mitigated in the current work, and the 4× efficiency figure should be interpreted as the gain achievable conditional on difficulty being known (or estimated through an amortized method that does not yet exist).

The ~14× Larger Model Baseline Is Not Compute-Optimally Trained and Uses Only Greedy Decoding

The assumption or constraint. The FLOPs-matched comparison in Section 7 scales model parameters while holding training data fixed, explicitly following the LLaMA paradigm (Touvron et al., 2023) rather than compute-optimal pretraining (Hoffmann et al., 2022) where both data and parameters are scaled equally. The authors acknowledge this:

"We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work."

Additionally, the larger model is evaluated with greedy decoding only — no majority voting, no best-of-N, no search, and no test-time compute augmentation of any kind.

The consequence. Both choices make the pretraining baseline weaker than it could be in two independent ways. First, a Chinchilla-optimal model trained with 14× more total FLOPs (scaling both parameters and data) would likely outperform a parameter-only-scaled model, meaning the reported advantages of test-time compute over pretraining may shrink or reverse against a properly optimized larger model. Second, giving the larger model even a modest test-time compute budget (say, best-of-8 with an ORM) would create a substantially stronger baseline that tests whether test-time compute substitutes for pretraining or merely augments it — a critical distinction the current setup cannot disentangle. For example, on easy questions at $R \ll 1$ , where the paper reports +27.8% relative improvement from test-time compute, a fairer comparison might ask: does a 14× larger model with best-of-8 still underperform the smaller model with compute-optimal scaling? The current experiments cannot answer this.

What evidence exists in the paper. Section 7 and Figure 9 present the FLOPs-matched results. The paper is transparent about the parameter-only scaling choice (quoted above). The greedy-decoding choice for the larger model is implicit in the experimental description (Section 7): the larger model is described as having its performance measured at three $R$ values, but no test-time compute variants of the larger model are tested anywhere.

Mitigation status. The paper acknowledges the scaling choice limitation and frames compute-optimal pretraining as future work (Section 8), but does not acknowledge the greedy-decoding-only limitation of the larger model baseline. Neither limitation is addressed in the current experiments. This is a partially acknowledged but fully unmitigated weakness that affects the strength of the paper's headline claim about test-time compute substituting for pretraining — the claim holds in the specific comparison presented, but the comparison is tilted in favor of test-time compute.

All Results Are on a Single Benchmark (MATH) with a Single Model Family (PaLM 2-S*)

The assumption or constraint. Every experiment in the paper — all search algorithms, all revision strategies, all difficulty-bin analyses, all FLOPs-matched comparisons — is conducted on the MATH benchmark (500 test questions) using PaLM 2-S* as the base model. The authors state they "believe this model is representative of the capabilities of many contemporary LLMs" (Section 4), but provide no evidence for this belief and test no other benchmarks or model families.

The consequence. Three aspects of the findings could be model-specific or benchmark-specific in ways that undermine their claimed generality:

PRM quality and over-optimization behavior. The PRM is trained on PaLM 2-S*'s output distribution using Monte Carlo rollouts. A model with different calibration properties (e.g., more or less confident in its wrong answers), different error patterns (e.g., arithmetic vs. conceptual mistakes), or different output diversity might produce PRMs with different over-optimization thresholds and different difficulty-dependent scaling curves. The paper's central finding — that beam search over-optimizes on easy problems but helps on medium ones — is a property of the interaction between PaLM 2-S*'s output distribution and its PRM. There is no reason to assume this interaction transfers to models with qualitatively different capabilities.
Revision model training depends on base model in-context learning. The revision model is fine-tuned on trajectories where incorrect answers (sampled from the base model) precede correct answers. If a different base model produces qualitatively different incorrect answers (e.g., PaLM 2-S* might make structured, partly-correct attempts while another model might produce nonsensical outputs), the revision model's ability to learn targeted corrections from edit-distance-paired examples would differ. The 38% correct-to-incorrect reversion rate (Section 6.1) might be specific to PaLM 2-S*'s revision behavior.
MATH is a specific reasoning domain. The benchmark consists of competition-level math problems requiring symbolic manipulation, algebraic reasoning, and multi-step deduction. The difficulty-dependent patterns — revisions helping on easy problems, beam search helping on medium problems, nothing helping on hard problems — might not hold for code generation (where execution feedback provides a stronger signal), factual QA (where retrieval is more relevant than multi-step reasoning), or open-ended generation (where correctness is ambiguous). The paper does not argue that these patterns are domain-general, but the conclusions are stated without domain qualification.

What evidence exists in the paper. All results in Sections 5–7 are on MATH with PaLM 2-S*. Section 4 justifies the choice of MATH as a benchmark where test-time compute is expected to matter (multi-step reasoning rather than factual recall), but does not address the single-model-family limitation. Appendix D describes PRM training; Appendix H describes revision model training. No results on other benchmarks (e.g., GSM8K, HumanEval, MMLU) or other model families (e.g., GPT-4, LLaMA, Claude) are reported.

Mitigation status. The authors acknowledge the single-benchmark limitation implicitly by restricting their claims to the MATH setting (the paper does not claim that the specific 4× efficiency figure generalizes to all reasoning tasks). However, the broader claims — that compute-optimal test-time scaling exists, that difficulty-dependent allocation is essential, that test-time compute can substitute for pretraining on easy-to-medium problems — are stated in general terms (Section 1, Section 8) without qualification. The single-model-family limitation is not acknowledged or discussed. This is a partially acknowledged but fully unmitigated limitation: the paper demonstrates the phenomenon exists, but provides no evidence that the specific quantitative relationships (difficulty thresholds, over-optimization onset, optimal sequential-to-parallel ratios) transfer beyond PaLM 2-S* on MATH.

The PRM Is Trained on Base-Model Outputs and Does Not Transfer to the Revision Model's Distribution

The assumption or constraint. The process reward model is trained exclusively on outputs from the few-shot prompted base model (PaLM 2-S*) using Monte Carlo rollouts (Section 5.1, Appendix D). When the revision model is introduced in Section 6, the paper notes that this PRM does not perform well on revision model outputs due to distribution shift. The authors address this by training a separate outcome reward model (ORM) specifically on revision model outputs. However, this means that PRM-guided search and revision-based proposal modification cannot be cleanly combined using the same verifier — a natural next step that the paper explicitly identifies as unaddressed (Section 8: "we did not experiment with PRM tree-search techniques in combination with revisions").

The consequence. This limitation has both practical and scientific implications. Practically, it means the paper's two main mechanisms — search against a PRM and iterative revision — are studied in isolation and their reported gains cannot be straightforwardly multiplied. A system that could use PRM-guided beam search with the revision model as the proposal distribution (where each step conditions on previous rejected branches) might substantially outperform either method alone, but the current verifier training pipeline cannot support this because the PRM is miscalibrated on revision model outputs. Scientifically, it means the paper's decomposition of test-time compute methods into "proposal modification" and "verifier optimization" axes is validated only through independent experiments, not through a combined system — leaving open the question of whether these axes interact synergistically or redundantly.

What evidence exists in the paper. Figure 15a (Appendix J) directly compares the base-LM PRM against the revision-specific ORM when scoring revision model outputs, showing the PRM underperforms. Section 6.1 describes the separate ORM training. Section 8 states the combination was not attempted. The paper is transparent about this separation.

Mitigation status. The paper trains a revision-specific ORM as a partial workaround (enabling verifier-based selection among revision outputs), but this ORM provides only final-answer scoring, not step-level guidance — meaning beam search and lookahead search (which require per-step scores) cannot be applied to revision model outputs. The paper acknowledges the missing combination as future work (Section 8). This limitation is fully acknowledged but unmitigated, and it means the paper's reported gains almost certainly represent a lower bound on what combined approaches could achieve — but the size of the gap is unknown.

Hard Problems (Difficulty Bin 5) Are Fundamentally Unsolved by Any Amount of Test-Time Compute

The assumption or constraint. The paper's difficulty binning reveals a hard ceiling: on the hardest questions (quintile 5, where the base model's pass@1 is near zero), no method — not beam search, not lookahead search, not sequential revisions, not any combination — produces meaningful improvement regardless of compute budget. Section 5.3 explicitly notes that on bin 5, "no method makes meaningful progress," and the bin 5 curves in Figure 3 (right) hover at 1–3% accuracy for all methods and all budgets up to 256 generations. The FLOPs-matched comparison (Figure 9) confirms this: on bin 5, the small model with compute-optimal test-time compute cannot match the larger model's performance at any $R$ regime, with the gap widening to -52.9% relative disadvantage at $R \gg 1$ for PRM search.

The consequence. Test-time compute can amplify existing capability but cannot create it from nothing. If the base model's pass@1 is near zero on a problem class, there are simply no correct solutions in the proposal distribution to find (via search) or refine (via revisions). This establishes a fundamental boundary condition for the entire approach: test-time compute is valuable only when the base model already produces correct solutions at some non-trivial rate — roughly, when the problem is within the model's approximate capability range (difficulty bins 1–4 in the paper's taxonomy). For problems that require genuinely novel reasoning, factual knowledge the model lacks, or capabilities not present in the pretrained weights, no amount of inference-time computation helps.

This has direct practical implications. A deployment that encounters a distribution shift toward harder problems (e.g., users asking progressively more difficult questions over time, or a new use case emerging that is fundamentally more challenging than the training distribution) would see test-time compute's benefits vanish entirely. The paper's FLOPs-matched analysis (Section 7) quantifies this: on hard problems, it is almost always better to invest compute in pretraining a larger model than in test-time strategies, regardless of the $R$ regime. For applications where problem difficulty is unpredictable or skewed toward the model's frontier, pretraining remains the only viable path to improvement.

What evidence exists in the paper. Figure 3 (right): bin 5 curves are essentially flat near 1–3% for all methods. Figure 7 (right): bin 5 shows roughly 2–3% accuracy regardless of sequential-to-parallel ratio. Figure 9: bin 5 (blue, bottommost line) is flat near 0–5% for all test-time compute budgets and lies below the larger model's greedy performance at all $R$ values. The paper is explicit about this in the Section 7 takeaway box: "Test-time compute provides minimal gains on problems that are fundamentally outside the base model's capability range."

Mitigation status. The paper is transparent about this limitation and treats it as a finding rather than a flaw — it establishes a clear boundary condition for the test-time compute paradigm. The authors do not propose any mitigation (nor should they, since it is a fundamental property of the approach). This limitation is fully acknowledged and inherent to the method — it is not a weakness of the experimental design but a conceptual boundary that the paper documents clearly. However, practitioners should understand that the 4× efficiency gains and the pretraining-substitution claims apply only to problems within the base model's rough capability range, not to tasks requiring fundamentally new capabilities.

7. Implications and Future Directions

How This Work Changes the Landscape

PREPING introduces a third phase in the agent memory lifecycle that did not previously exist as a coherent research problem. Before this work, the field implicitly assumed that procedural agent memory must be built from task experience — either collected offline through human effort (designing tasks, curating demonstrations) or accumulated online from user interactions as they arrive. The offline path avoids cold-start failures but requires per-environment human investment that scales poorly. The online path eliminates upfront human cost but starts from empty memory, exposing users to early failures and incurring ongoing deployment-time update costs. PREPING carves out a pre-task phase that occupies neither pole: memory is constructed before deployment using only environment access (documentation and executable tools), with no human-provided task data of any kind. This reframes the cost calculus from a two-way choice — pay upfront (offline) or pay per-use (online) — to a three-way choice where the middle path pays a one-time pre-deployment cost of controlled synthetic practice, then deploys with either frozen memory (eliminating update cost and cold-start failures) or warm-started online memory (eliminating cold-start while retaining adaptability).

The magnitude of this shift is a reframing with concrete practical consequences, not a paradigm shift. The core technical components — proposer-guided task generation, validation-gated admission, reflector–curator memory induction — are assembled from existing primitives (ACE (Zhang et al., 2026), self-generated practice (Zhou et al., 2025; Huang et al., 2025), LLM-as-judge evaluation). What is new is the joint control formulation: the insight that synthetic task generation and memory admission are coupled decisions whose interaction determines memory quality, and that naive decoupling (generate tasks freely, store all trajectories) produces memory worse than no memory at all (47.8 TGC vs. 69.6 Base on AppWorld Test-Normal, Table 2). This finding — that unfiltered synthetic practice is actively harmful — is the diagnostic that justifies the control framework and distinguishes PREPING from prior self-practice work that assumed more generation is always beneficial.

The work also reconciles conflicting intuitions about what matters for memory construction. The exploration baselines (Random and Guided Exploration, Table 1) show that execution feedback alone, without task-level objectives, provides limited reusable signal even with 100 trajectories of live environment interaction: Guided Exploration improves AppWorld average by only 2.5 points. Direct Memory shows that documentation alone, without execution, is worse than nothing on BFCL v3 Base (43.2 vs. 43.7 Base). The component ablation (Table 2) shows that proposal quality matters, but validation quality matters more: the Validator admission gate alone recovers over 30 points of the gap between Naive and full PREPING, while proposer-side control contributes the remaining 5.5 points. These results collectively establish that effective memory construction requires all three elements — task-level objectives (not free-form exploration), execution feedback (not static documentation), and quality filtering (not unfiltered storage) — and that omitting any one produces memory that is marginal or harmful.

This shifts research priorities in two ways. First, it makes validation and filtering a first-class research problem in memory construction, not an afterthought. The finding that the Validator's admission gate is the single largest performance lever (Table 2) suggests that investment in better feasibility assessment — more reliable LLM judges, learned verifiers, environment-executable checks — may yield larger returns than investment in more sophisticated task proposal strategies. Second, it makes the exploration–grounding tension (coverage expansion vs. feasibility, visible in the opposing pressures of practice history and environment information in Table 2) a measurable and optimizable quantity, rather than an implicit tradeoff. Future work can explicitly design proposer-memory representations that navigate this tension, using the diagnostics introduced here — infeasible task rate, unique tool count, tool entropy, weighted recall — as quantitative guidance.

Follow-Up Research This Work Enables

Cheap, online difficulty estimation for memory admission. The Validator in PREPING uses an LLM with a detailed prompt (Figure 7) and a strict feasibility = 5 Likert threshold to gate memory admission. This is effective but expensive: every synthetic trajectory requires a separate LLM validation call, and the paper does not report Validator accuracy or calibration. A natural follow-up would train a lightweight classifier — distilled from the Validator's judgments on a corpus of synthetic trajectories — that predicts feasibility directly from the task text and trajectory summary, without requiring a full LLM call. The diagnostic infrastructure already exists: the paper reports infeasible task rates (16.0% on AppWorld, 4.0% on BFCL v3 for full PREPING, Table 2), so a distilled classifier could be evaluated by how closely its rejection rate and downstream memory quality match the LLM Validator's. A strong result would show that a distilled classifier (costing < 1% of an LLM validation call) achieves admission decisions that produce solver memory within 1–2 TGC points of the full Validator, while reducing construction-time validation cost by 10–100×.

Validation-gated exploration baselines to decouple task structure from filtering. The paper's exploration baselines (Random and Guided Exploration) use the same memory induction pipeline as PREPING but do not apply the Validator's feasibility gate to their trajectories. This confounds two factors: task-level objectives vs. free-form exploration, and validated admission vs. unfiltered admission. A critical follow-up experiment would run Guided Exploration with the same Validator gate that PREPING uses (only trajectories from feasible exploration sequences enter solver memory) and measure whether the performance gap between exploration and PREPING closes, narrows, or remains unchanged. If validation-gated exploration matches PREPING, then task-level objectives are unnecessary — the Validator alone, applied to any environment interaction, suffices for quality memory. If the gap persists, it quantifies the irreducible value of task-level structure over free-form interaction. This experiment would also reveal whether the exploration baselines' poor performance in Table 1 is due to unfiltered memory contamination (which would be fixed by adding the Validator gate) or due to shallow, non-compositional interaction patterns (which would not).

Admission threshold sensitivity and Validator calibration analysis. PREPING uses the strictest possible admission criterion: only task–trajectory pairs with feasibility score 5 (the maximum) enter solver memory. The paper does not explore what happens if the threshold is relaxed to 4, or if completion score is used as a secondary filter (e.g., require feasibility ≥ 4 AND completion ≥ 4). A direct follow-up would sweep the feasibility threshold from 1 (admit everything, equivalent to Naive) to 5 (the current setting) and measure downstream memory quality at each threshold, producing a curve analogous to precision–recall tradeoff analysis. This would reveal whether there is a "sweet spot" below 5 that admits more trajectories (increasing memory coverage) at an acceptable quality cost, and whether the optimal threshold differs across environments. The paper already has the infrastructure: the Validator produces Likert scores for all trajectories, so this experiment requires only re-running the memory induction pipeline with different admission thresholds on already-collected data — no new environment interaction needed. A companion analysis would measure Validator calibration by comparing feasibility scores against a ground-truth criterion (e.g., whether a human judge or an execution-based check confirms the task is actually executable), which would show whether the LLM Validator is conservative (high precision, low recall — rejecting many actually-feasible tasks), liberal (low precision, high recall — admitting infeasible ones), or well-calibrated.

Combining PREPING memory with retrieval-based selection rather than full-context insertion. PREPING's solver memory is inserted in its entirety into the task-solving prompt (Figures 11–13). For construction budgets larger than 100 tasks, or for environments with many tools, this playbook could grow beyond the model's effective context window or dilute attention across irrelevant guidance. A natural extension would apply a retrieval step at deployment time: given a new task, retrieve only the most relevant subset of the playbook (e.g., bullets mentioning the tools or APIs required by the current task) rather than inserting everything. This is a standard retrieval-augmented generation pattern, but the paper's diagnostic framework provides a clean evaluation: measure whether retrieval reduces performance for in-distribution tasks (due to retrieval errors missing relevant guidance) while enabling larger memory to be constructed (since full-context insertion is no longer a bottleneck). A strong result would show that retrieval-based selection from a 500-task PREPING memory matches or exceeds full-context insertion of a 100-task memory, with lower per-task inference cost.

Stress-testing PREPING on environments where documentation is sparse or misleading. PREPING assumes access to "sufficiently detailed API or tool documentation" (Appendix C), because the Proposer must ground synthetic tasks in documented interfaces. The paper acknowledges this limitation but does not test it. A direct stress-test would evaluate PREPING on environments where documentation is systematically degraded: (1) missing parameter descriptions, (2) absent or incorrect return type specifications, (3) missing precondition documentation (e.g., "this endpoint requires prior authentication" not stated in the schema), or (4) hallucinated tool descriptions (simulating an environment where documentation was auto-generated and contains errors). The hypothesis is that proposer-guided practice with environment-grounded observation summarization can partially compensate for documentation gaps because execution feedback reveals ground-truth tool behavior — but the Proposer still needs enough documentation to formulate initial tasks. This experiment would map the boundary conditions under which PREPING degrades, informing practitioners about documentation quality requirements.

Pre-task memory for policy updates rather than textual memory. PREPING constructs textual procedural memory (an ACE playbook) that is inserted into the agent's context. A parallel line of work (Zhou et al., 2025; Huang et al., 2025; Acikgoz et al., 2026) uses self-generated practice for model fine-tuning or RL-based policy updates. A natural bridge would apply PREPING's proposer-guided, validator-gated synthetic practice to generate training data for fine-tuning the Solver itself, rather than constructing external memory. The hypothesis is that the same control mechanisms — coverage-expanding task proposals, feasibility-gated trajectory filtering — would improve the quality and diversity of fine-tuning data compared to unfiltered self-practice, producing a better policy. This experiment would connect the memory construction and policy improvement literatures, using PREPING's diagnostics (infeasible rate, tool entropy, weighted recall) to measure training data quality.

Practical Applications and Downstream Use Cases

Pre-deployment memory for newly connected tool ecosystems. The most direct practical application is for agent platforms that regularly connect to new APIs, MCP servers, or application suites — a scenario the paper explicitly targets with its MCP-Universe and BFCL v3 evaluations. An organization deploying an LLM agent that must operate across a growing inventory of third-party tools could run PREPING offline whenever a new tool or server is added, constructing procedural memory that captures tool-specific preconditions, failure modes, and composition patterns before any user asks a task involving that tool. The paper's cost analysis (Table 12) shows this one-time construction costs $10.11 on AppWorld and$ 6.51 on BFCL v3 using DeepSeek-V3.2 API pricing — a fraction of the cost of having a human engineer write playbook instructions for the same tool set. The construction budget analysis (Figure 4) shows that even 30 synthetic tasks (roughly $3 in inference cost at the paper's pricing) exceeds all pre-task baselines, making this practical for rapid tool onboarding.

Warm-starting online memory for interactive agent deployments. For production agent deployments where memory is expected to improve from user interactions (the ACE-Online setting), PREPING can eliminate the cold-start period without requiring a human-curated task set. Table 3 shows that PREPING+ACE improves AppWorld average from 71.3 (ACE-Online) to 76.3, with the largest gains on Test-Challenge SGC (from 60.9 to 65.2) — precisely the metric that reflects robustness across task variants. Figure 3 quantifies the user-facing benefit: PREPING+ACE achieves 82.2% success over the first 10 tasks versus 74.4% for ACE-Online, meaning roughly 8 additional users per 100 have a successful first interaction. For a customer-facing agent where early failures drive churn, this improvement could be commercially significant. The cost is the one-time construction ($10.11 on AppWorld, amortized across all subsequent users) plus ongoing online update costs — but since PREPING memory already covers broad tools, online updates can be sparser (triggered only when novel failure patterns emerge), potentially reducing the 2.23–2.99× deployment-time cost overhead of pure online construction (Figure 5).

Batch evaluation and testing of agent capabilities in new environments. Before deploying an agent to a new environment, engineering teams often need to assess what the agent can and cannot do — to identify failure modes, set user expectations, and prioritize capability improvements. PREPING's synthetic task generation and validation pipeline could serve double duty: the same proposer-guided tasks used for memory construction also function as a dynamically generated test suite covering the environment's tool surface. The diagnostics in Table 2 — infeasible task rate, tool coverage, tool entropy — already characterize the environment's procedural landscape. An engineering team could run PREPING, inspect which tasks were infeasible (revealing genuine environment constraints), which tasks the Solver failed despite being feasible (revealing capability gaps), and use the resulting playbook as both memory and a capability report. This is more actionable than static API documentation review because it surfaces execution-revealed constraints that schemas omit.

When to Prefer This Method Over Alternatives

The paper articulates a clear tradeoff between pre-task memory construction (PREPING), offline task-informed memory construction (ACE-Offline), and online memory construction (ACE-Online). The decision conditions are:

Prefer PREPING (pre-task memory) when:

No human-defined task set exists for the target environment, and designing one is costly or impractical (the environment is new, poorly documented, or one of many that must be onboarded).
Deployment-time cold-start failures are unacceptable (e.g., user-facing agents where early failures drive churn; Figure 3 shows ACE-Online's first-10-task success rate drops to 74.4% vs. PREPING's 79.4%).
Deployment-time memory-update cost must be minimized (Figure 5 shows 2.23–2.99× cost overhead for ACE-Online due to recurring memory-update calls during deployment).
The environment's tools compose into multi-step workflows — compositional environments benefit more from task-level synthetic practice than from free-form exploration (supported by the 14.6-point gap between PREPING and Guided Exploration on AppWorld average, Table 1).

Prefer ACE-Offline (task-informed offline memory) when:

A representative, diverse set of target-environment tasks is already available or can be collected with modest human effort (the AppWorld training split provides this; Table 1 shows ACE-Offline achieves 82.7/69.1 on Test-Normal, close to PREPING's 83.7/70.2, with no synthetic practice cost).
Memory quality on out-of-distribution or hard tasks is critical and task-specific patterns (not general tool-coverage patterns) drive success — though on AppWorld Test-Challenge, PREPING (72.2/54.7) actually exceeds ACE-Offline (69.0/50.3), so this condition may not hold in practice.

Prefer ACE-Online (online memory) when:

The task distribution shifts over time and memory must adapt continuously (PREPING's frozen memory cannot adapt without the PREPING+ACE extension, Table 3).
Deployment-time inference cost is not a binding constraint and users tolerate early failures while memory builds up.
The task distribution naturally exercises a broad range of tools — Figure 1 shows that on BFCL v3 Base, ACE-Online never reaches PREPING's tool coverage even after 200 tasks, so if coverage breadth matters, online-only memory may be structurally insufficient.

Prefer PREPING+ACE (pre-task warm-start + online refinement) when:

Both cold-start prevention and continuous adaptation are required — this combination achieves the highest overall performance on AppWorld (76.3 average, Table 3) and eliminates the cold-start dip while still benefiting from task-specific online updates.
The marginal cost of online updates is acceptable (they are still incurred during deployment, Figure 5), but the pre-task memory reduces how many updates are needed and provides coverage that user-task distributions may never supply.