ArXiv: 2507.21046

🎯 Pitch

LLM agents that can learn from their own mistakes remain largely aspirational—this survey reveals that while systems like WebRL can boost performance from 4.8% to 42.4% through self-generated training, catastrophic forgetting and reward hacking cripple autonomous evolution at scale. The paper provides the first systematic framework to diagnose why, organizing agent self-improvement into three axes—what component to update, when to adapt, and how to guide change—to chart the road from static chatbots to genuinely self-evolving intelligence.

1. Executive Summary

This survey establishes the first systematic framework for understanding self-evolving agents—LLM-based systems that autonomously modify their internal parameters, contextual state, tools, or architectural topology based on their own trajectories and feedback signals—by organizing the field around three foundational questions: what to evolve (model parameters, memory/prompts, tools, architecture), when to evolve (intra-test-time vs. inter-test-time), and how to evolve (reward-based, imitation-based, or population-based mechanisms). Analyzing representative methods across domains like coding (SWE-bench, WebArena), GUI interaction (OSWorld), mathematical reasoning (MATH, GSM8K), and medical diagnosis, the survey taxonomizes approaches spanning from lightweight textual reflection (e.g., Reflexion's verbal reinforcement learning) through reinforcement learning with process rewards (e.g., RAGEN's multi-turn tool-use optimization) to population-based architectural search (e.g., ADAS's Monte Carlo Tree Search over agent topologies). The survey identifies verifier over-optimization and catastrophic forgetting as central scaling bottlenecks, while establishing that fully autonomous self-evolution without human intervention remains aspirational—current systems operate across a spectrum from "proto-evolution" (iterative bootstrapping) to "strong self-evolution" (autonomous diagnosis and reconfiguration), with self-directed curriculum generation yielding substantial gains (e.g., WebRL improving from 4.8% to 42.4% through autonomously generated tasks) only when verifiable feedback signals are available.

2. Context and Motivation

The Core Problem: We Don't Have a Systematic Understanding of How LLM Agents Should Autonomously Improve

The fundamental gap this survey addresses is deceptively simple to state but profound in its implications: the field lacks a unified conceptual framework for understanding, designing, and comparing self-evolving agent systems. While individual papers have demonstrated that LLM-based agents can improve through various mechanisms—self-reflection on failures (Shinn et al., 2023), reinforcement learning from environmental rewards (Wang et al., 2025q; Bai et al., 2024), evolutionary search over agent architectures (Hu et al., 2024c; Zhang et al., 2024c)—these advances have accumulated in an ad hoc manner, with each work proposing its own terminology, its own decomposition of the problem, and its own evaluation methodology. There has been no attempt to step back and ask: what are the essential dimensions along which any self-evolving agent system must make design choices?

This gap is not merely taxonomic. Without a shared framework, three critical problems emerge:

  • Fragmented terminology prevents comparison. Different papers use different language for conceptually similar mechanisms. Is "verbal reinforcement learning" in Reflexion (Shinn et al., 2023) the same thing as "self-reflection" in Self-Refine (Madaan et al., 2023b), or "textual feedback" in TextGrad (Yellamraju et al., 2024)? Without a common vocabulary, researchers cannot easily determine whether apparent conflicts in results reflect genuine differences in method efficacy or simply different instantiations of the same underlying principle.

  • Design choices are made implicitly rather than systematically. Any self-evolving agent must answer at least three questions: what part of the agent should change, when should that change occur, and what signal should drive it. In current practice, these choices are often bundled together in monolithic system descriptions, making it difficult to isolate which design decisions actually matter. If a system improves agent performance, is it because of the architectural search space, the evolutionary algorithm, the reward function design, or the specific tasks used for evaluation? The field lacks the conceptual machinery to cleanly ablate these factors.

  • Evaluation conflates capability with adaptability. Standard agent benchmarks (SWE-bench, WebArena, GAIA) measure what an agent can do at a single point in time, not whether it improves through experience. Yet the defining claim of self-evolving agents is precisely that they get better over time. As the survey documents in Section 7, most existing benchmarks adopt episodic evaluation where agent state resets between tasks, "fundamentally precluding measurement of knowledge accumulation or degradation—precisely the phenomena that distinguish self-evolving agents from static systems." This means we cannot currently answer basic questions like: does an agent that performs well after 1000 iterations of self-training actually retain its original capabilities, or has it catastrophically forgotten them?

Why This Problem Matters: The Shift from Scaling Models to Scaling Adaptation

The urgency of this taxonomic work derives from a broader paradigm shift in AI research that the survey explicitly positions itself within. As the authors state in Section 1:

"This emerging need for dynamic, continual adaptation signals a conceptual shift in artificial intelligence: from scaling up static models to developing self-evolving agents."

This statement is rich with implications that deserve unpacking. For the past several years, the dominant narrative in LLM research has been that capability improvements come primarily from scaling—more parameters, more data, more pretraining compute (Hoffmann et al., 2022). But this narrative has increasingly encountered hard limits, which the survey identifies through several concrete bottlenecks:

The static model bottleneck. LLMs, once trained, are frozen. Their knowledge is fixed at the cutoff date of their training data. Their capabilities are determined by their pretraining and fine-tuning regimen. When deployed in open-ended environments—interacting with users over months, facing novel tasks, encountering knowledge that postdates their training—they have no native mechanism for adaptation. As the survey notes, "conventional knowledge retrieval mechanisms prove inadequate, giving rise to agents capable of dynamically adapting their perception, reasoning, and actions in real time."

The human-in-the-loop bottleneck. Traditional approaches to improving deployed models require human engineers to curate new training data, design new fine-tuning procedures, validate results, and schedule updates. This pipeline is slow, expensive, and fundamentally cannot scale to the diversity of tasks and environments that agents face in the wild. A self-evolving agent that can learn from its own interactions—generating its own training signals from environmental feedback, execution traces, or self-reflection—offers a path to improvement that does not depend on constant human intervention.

The capability distribution bottleneck. Different tasks require different capabilities, and no single model—no matter how large—is optimal for all of them. The survey catalogues evidence that agents can improve substantially through test-time or inter-test-time adaptation: WebRL improves from 4.8% to 42.4% on web navigation through a self-evolving curriculum (Qi et al., 2024), while SEAgent achieves a 23.2 percentage point improvement through autonomous software exploration (Sun et al., 2025c). These gains suggest that adaptation can be more cost-effective than scaling model size, echoing findings from the test-time compute scaling literature (Snell et al., 2024) but applied to a broader class of agent capabilities.

The autonomy aspiration. Beyond immediate practical concerns, the survey explicitly frames self-evolving agents as a step toward Artificial Super Intelligence (ASI)—systems that "not only can learn and evolve from experience with an unpredictable speed but also perform at or above human-level intelligence across a wide array of tasks" (Section 1). While the authors are careful to note that "fully autonomous self-evolution without human intervention represents an aspirational goal rather than the current norm," this framing establishes the stakes: understanding self-evolution is not just about building better chatbots, but about understanding the fundamental principles by which intelligent systems can autonomously improve.

Where Prior Approaches Fall Short

The survey identifies specific limitations in prior work along multiple axes:

Prior surveys treat evolution as a subsidiary concern. The authors carefully differentiate their contribution from existing surveys. Luo et al. (2025a) "discuss several ways to evolve, such as self-learning and multi-agent co-evolution," but do so as one component within a broader agent taxonomy rather than as the central organizing principle. Liu et al. (2025a) "explicitly introduce the evolution in terms of different components of agents, such as tools and prompts," but again within a comprehensive agent overview. Tao et al. (2024) focus specifically on self-evolution of language models themselves, "rather than on the broader concept of agents." The cumulative result is that self-evolution has been treated as a feature of agents rather than as a first-class research paradigm requiring its own conceptual framework.

No unified vocabulary exists across the field. The survey highlights this fragmentation explicitly. When one paper describes "verbal reinforcement learning" (Shinn et al., 2023), another describes "iterative refinement with self-feedback" (Madaan et al., 2023b), and a third describes "textual differentiation" (Yellamraju et al., 2024), are these fundamentally different mechanisms or different names for the same underlying idea? The absence of a shared vocabulary means that research progress is difficult to track, contradictory findings are hard to reconcile, and best practices cannot propagate efficiently across the community.

Evaluation is fundamentally misaligned with the phenomenon being studied. This is perhaps the survey's most important critique of existing work. Self-evolving agents are defined by their capacity to improve over time, yet they are evaluated primarily using static benchmarks designed for fixed-capability systems. Section 7 systematically documents this mismatch:

  • Current benchmarks measure instantaneous performance, not learning trajectories
  • Most evaluation protocols reset agent state between tasks, precluding measurement of knowledge accumulation or forgetting
  • Efficiency metrics (tokens consumed, wall-clock time, memory growth) are reported inconsistently or not at all
  • Long-horizon safety drift—whether an agent becomes more or less safe as it evolves—is essentially unmeasured

The survey further identifies specific "coverage gaps" in evaluation across five dimensions (Table 8): adaptivity benchmarks lack open-ended exploration without predetermined objectives; retention assessment is undermined by episodic designs that reset state; generalization evaluation fails to capture temporal robustness under distribution drift; efficiency evaluation imposes no enforced budgets during evolution; and safety evaluation remains static rather than tracking long-horizon drift.

The relationship between self-evolution and other learning paradigms is muddled. The survey identifies conceptual confusion between self-evolving agents and related but distinct paradigms: curriculum learning, lifelong learning, model editing, and unlearning. As Table 1 establishes, self-evolving agents are distinguished by a unique combination of properties—runtime adaptation, dynamic test-time behavior, active exploration, structural change capability, and self-reflection—that no other paradigm fully captures. But prior work has often used these terms interchangeably or failed to delineate their boundaries, making it difficult to understand what unique challenges self-evolution poses that existing techniques cannot address.

How This Paper Positions Itself

The survey positions itself not as proposing a new method, but as providing the conceptual infrastructure that the field has been missing. The key intellectual move is the decomposition of self-evolution into three orthogonal dimensions—what, when, and how—each of which can be analyzed independently, and whose cross-product defines the design space of possible self-evolving systems.

The what dimension (Section 3) specifies the evolutionary loci within an agent system: the model parameters that can be updated through self-generated supervision, the context (prompts and memory) that shapes behavior without weight modification, the tools that extend capability through creation and refinement, and the architecture that governs information flow and collaboration patterns. This decomposition matters because it recognizes that self-evolution is not one thing—adjusting a prompt based on reflection feedback is fundamentally different from retraining model weights on self-generated trajectories, and different again from evolving a multi-agent topology through genetic algorithms. The taxonomy makes these differences legible and comparable.

The when dimension (Section 4) distinguishes intra-test-time evolution (adaptation during task execution, coupled to the immediate problem) from inter-test-time evolution (learning between tasks, aimed at improving future performance across the distribution). This distinction captures a fundamental tension in agent design: should the agent improve its answer to this specific question through reflection and revision, or should it extract generalizable lessons from its experience to perform better on future questions? The survey shows that different learning paradigms (in-context learning, supervised fine-tuning, reinforcement learning) are instantiated differently depending on when they occur, and that the choice of timing has profound implications for computational cost, latency, and the type of improvement achievable.

The how dimension (Section 5) categorizes the mechanisms by which evolution is driven: reward-based methods that use scalar or textual feedback signals, imitation and demonstration methods that learn from exemplar trajectories, and population-based methods that maintain multiple agent variants and apply evolutionary operators. This categorization reveals that the choice of mechanism is not independent of the other dimensions—textual feedback is naturally suited to intra-test-time adaptation of context (since it doesn't require weight updates), while population-based methods are naturally suited to inter-test-time optimization of architecture (since evaluating multiple topologies is expensive and best done offline).

The survey explicitly frames itself as a "guiding synthesis" rather than a review of a fully established paradigm. The authors acknowledge the field's immaturity:

"Given that this represents a rapidly forming research area where conceptual boundaries are still being actively negotiated within the community, we frame this survey as a guiding synthesis rather than a review of a fully established paradigm. Instead of enforcing rigid boundaries, we aim to structure the heterogeneous mechanisms emerging in the community into a coherent framework."

This intellectual humility is important. The survey is not claiming to have discovered the "correct" taxonomy of self-evolving agents, but rather to provide a lens through which existing work can be understood, compared, and built upon. The operational definition of self-evolving agents—modifying internal parameters, contextual state, tools, or architecture based on trajectories and feedback to improve future performance—establishes inclusion criteria, but the authors explicitly include a spectrum from "proto-evolution" to "strong self-evolution," allowing the framework to accommodate both early-stage systems that require significant human intervention and more autonomous systems that approach the aspirational goal.

The survey also positions itself relative to the broader AI research trajectory. Figure 1 establishes a conceptual progression: from LLMs (static language understanding and generation), to foundation agents (execution via tools and planning), to self-evolving agents (learning from feedback and experience), and ultimately toward ASI (open exploration). Self-evolving agents occupy a critical position in this trajectory—they represent the point at which agentic systems begin to take responsibility for their own improvement, rather than depending on external engineering. The survey thus frames its contribution not as an endpoint but as establishing a "roadmap for advancing more adaptive, capable, robust, and versatile agentic systems in both research and real-world deployments" (Section 1).

In sum, the survey addresses a gap that has become increasingly apparent as the number of self-evolving agent papers has grown: the field needs a shared language, a systematic decomposition of the design space, and evaluation methodologies matched to the phenomenon under study. By organizing the literature around the what-when-how framework, the survey provides the intellectual scaffolding for the next phase of research, where self-evolution is treated not as a feature to be bolted onto existing agents, but as a fundamental organizing principle around which entire agent architectures are designed.

3. Technical Approach

3.1 Reader Orientation

This paper is a survey and taxonomic framework, not a single system—it synthesizes and organizes the heterogeneous landscape of self-evolving agent research into a coherent conceptual structure. The core problem it solves is conceptual fragmentation: without a shared vocabulary and systematic decomposition of the design space, researchers cannot effectively compare methods, isolate which design choices matter, or build cumulatively on each other's work. The solution takes the shape of a three-dimensional orthogonal taxonomy—what to evolve, when to evolve, and how to evolve—where any concrete self-evolving agent system can be positioned as a point in this design space by specifying its choices along each independent dimension.

3.2 Big-Picture Architecture (Diagram in Words)

The survey constructs its framework through five major conceptual components that map to its section structure:

  1. Agent System Formalization (Section 2): A mathematical model defining what an agent is before it evolves—a tuple $\Pi = (\Gamma, \{\psi_i\}, \{C_i\}, \{W_i\})$ comprising architecture, models, context, and tools—and a transformation function $f$ that maps the agent from its current state to a new state based on trajectories and feedback. This establishes the universe of possible evolutionary modifications.

  2. The What Dimension (Section 3): An enumeration of the four evolutionary loci within an agent—Model parameters, Context (prompts and memory), Tools, and Architecture—each representing a component whose state can be autonomously rewritten in an experience-driven and persistent manner. This dimension specifies which parts of the agent can change.

  3. The When Dimension (Section 4): A temporal decomposition of evolution into two modes—intra-test-time (adaptation during task execution, coupled to the immediate problem) and inter-test-time (learning between tasks, aimed at improving future performance). This dimension specifies at what stage the evolution occurs.

  4. The How Dimension (Section 5): A categorization of evolutionary mechanisms into three families—reward-based (using scalar or textual feedback), imitation/demonstration-based (learning from exemplar trajectories), and population-based (maintaining multiple agent variants with evolutionary operators). This dimension specifies what signal and algorithm drives the evolution.

  5. Evaluation Framework (Section 7): A five-goal evaluation taxonomy—adaptivity, retention, generalization, efficiency, and safety—mapped to three temporal paradigms (static, short-horizon, long-horizon). This provides the measurement infrastructure for assessing whether evolution is actually occurring.

Information flows through the survey as follows: the formal definitions in Section 2 establish what constitutes a valid self-evolving agent and delineate boundaries with related paradigms; Sections 3–5 populate the three dimensions of the taxonomy with concrete methods, showing how each design choice manifests in practice; Section 6 maps where evolution occurs (general vs. specialized domains); Section 7 provides the evaluation lens for assessing evolutionary outcomes.

3.3 Roadmap for the Deep Dive

  • First, the formal definition of agent systems and self-evolution (Section 2.1)—this establishes the mathematical vocabulary (POMDP, agent tuple, transformation function, utility objective) that the entire taxonomy depends on, and defines the inclusion criteria for what counts as a self-evolving agent.

  • Second, the operational definition and spectrum from proto-evolution to strong self-evolution (Section 2.1)—this resolves the boundary problem of what to include in the survey and makes explicit the aspirational nature of full autonomy.

  • Third, the what dimension (Section 3)—walking through each of the four evolutionary loci (model, context, tools, architecture) with concrete examples, showing how the same underlying evolutionary principle manifests differently depending on which component is being modified.

  • Fourth, the when dimension (Section 4)—explaining the temporal decomposition and how the three learning paradigms (ICL, SFT, RL) are instantiated differently in intra-test-time vs. inter-test-time contexts.

  • Fifth, the how dimension (Section 5)—detailing the three families of evolutionary mechanisms, their cross-cutting dimensions (online/offline, on/off-policy, reward granularity), and how they interact with the what and when choices.

  • Sixth, the evaluation framework (Section 7)—since evaluation methodology must be matched to the phenomenon under study, and the survey identifies this mismatch as a central gap.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a survey and taxonomic synthesis paper whose core idea is that self-evolving agent research can be systematically understood, compared, and advanced by decomposing the design space into three orthogonal dimensions—what to evolve, when to evolve, and how to evolve—and that this decomposition reveals previously hidden patterns, trade-offs, and research gaps.

Formal Definition of Agent Systems and Self-Evolution

The survey begins by establishing a mathematical model that makes precise what changes during evolution and what constitutes a valid evolutionary transformation. This formalism serves as the foundation upon which the entire taxonomy is built—it defines the vocabulary through which "what to evolve" is expressed, and the transformation function through which "how to evolve" is operationalized.

Environment as POMDP. The environment in which an agent operates is modeled as a partially observable Markov Decision Process, represented as an 8-tuple:

E=(G,S,A,T,R,Ω,O,γ)E = (G, S, A, T, R, \Omega, O, \gamma)

where $G$ is a set of potential goals (task objectives like user queries), $S$ is a set of environment states, $A$ is a set of actions (each being a combination of textual reasoning, retrieval of external knowledge, and tool calls), $T(s'|s, a)$ is the state transition probability function, $R : S \times A \times G \to \mathbb{R}$ is the feedback/reward function conditioned on a specific goal, $\Omega$ is a set of observations accessible to the agent, $O(o'|s, a)$ is the observation probability function, and $\gamma$ is the discount factor.

What it models: The environment is the external world (including users and execution environments like Linux shells) that the agent interacts with. It is partially observable because the agent never has direct access to the true state $s$—it only receives observations $o \sim O(\cdot|s, a)$ that provide incomplete information.

Why this form: The POMDP formalism is chosen because it naturally captures several essential features of agent environments: (1) sequential decision-making under uncertainty (through the state transition function and partial observability), (2) goal-conditioned behavior (through the goal set $G$ that parameterizes the reward function), and (3) the mixture of discrete tool calls and continuous language generation (through the action space $A$ being the union of the natural language space and the tool space). A simpler MDP formulation would fail to capture partial observability; a stateless formulation would fail to capture how actions affect future states.

Agent system as tuple. The (multi-)agent system is defined as:

Π=(Γ,{ψi},{Ci},{Wi})\Pi = (\Gamma, \{\psi_i\}, \{C_i\}, \{W_i\})

where $\Gamma$ is the architecture that determines control flow and collaborative structures between multiple agents (represented as a sequence of nodes organized by graph or code structures), $\{\psi_i\}$ is the set of underlying LLMs/MLLMs at each node, $\{C_i\}$ is the set of context information at each node (including prompt $P_i$ and memory $M_i$), and $\{W_i\}$ is the set of available tools/APIs at each node.

What it models: The agent is decomposed into four orthogonal components—its structure (architecture), its cognitive engine (models), its immediate behavioral context (prompts and memory), and its action capabilities (tools). Each component corresponds to one of the evolutionary loci discussed in Section 3.

Why this form: This decomposition is the survey's central analytical move. By explicitly separating these four components, the survey makes legible which parts of an agent can evolve. Architecture evolution ($\Gamma \to \Gamma'$) is a different design choice from model evolution ($\psi_i \to \psi_i'$), which is different from tool evolution ($W_i \to W_i'$). This decomposition also reveals dependencies: a change to tools may require changes to prompts, and a change to architecture may require changes to how models are invoked. The tuple structure enables the what-when-how taxonomy by defining the universe of things that can evolve.

Agent policy. At each node $i$, the agent policy is a function:

πθi(⋅∣o)\pi_{\theta_i}(\cdot|o)

that takes an observation and outputs the probability distribution of the next action, where $\theta_i = (\psi_i, C_i)$ bundles the model and its context. The action space is the union of the natural language space and the tool space $W_i$.

What it defines: The policy is the behavior-generating mechanism of the agent—given what it observes, what does it do? The parameters $\theta_i$ are exactly the parts of the agent that can be modified through evolution: the model weights $\psi_i$ and the context $C_i$ (which includes prompts and memory).

Why this parameterization: Bundling model and context into $\theta_i$ reflects the reality that behavior can be changed either by updating model weights (through SFT or RL) or by modifying the context (through prompt optimization or memory updates). This unification is important because it means the survey's framework applies equally to weight-based evolution (like RAGEN's policy gradient updates) and context-based evolution (like Reflexion's verbal reinforcement learning).

Self-evolving strategy as transformation. The central mathematical object of the survey is the self-evolving strategy:

f(Π,τ,r)=Π′=(Γ′,{ψi′},{Ci′},{Wi′})f(\Pi, \tau, r) = \Pi' = (\Gamma', \{\psi_i'\}, \{C_i'\}, \{W_i'\})

where $\Pi$ is the current agent system, $\tau$ is the trajectory generated on a task, $r$ is the external or internal feedback, and $\Pi'$ is the new agent system. The function $f$ takes the agent's current state, its experience, and the resulting feedback, and produces a modified agent.

What it computes: Given what the agent did ($\tau$) and how well it did ($r$), what should the agent become ($\Pi'$)? The transformation can modify any subset of the four components—it might update model weights, revise prompts, add new tools, or restructure the workflow topology.

Why this form: The transformation function $f$ is the instantiation of "how to evolve." Different methods propose different $f$ functions: Reflexion's $f$ modifies context $C_i$ by appending verbal reflections to memory; RAGEN's $f$ modifies model weights $\psi_i$ through policy gradient updates; ADAS's $f$ modifies architecture $\Gamma$ through Monte Carlo Tree Search. The function signature $f(\Pi, \tau, r)$ captures the three essential inputs to any evolutionary process: the current system state, the behavioral data, and the evaluation signal.

Objective of self-evolving agents. The overarching objective is defined as maximizing cumulative utility over a sequence of tasks:

max⁡f∑j=0nU(Πj,Tj)\max_f \sum_{j=0}^n U(\Pi_j, T_j)

where $U$ is a utility function measuring performance on task $T_j$, and the agent evolves through successive applications of $f$:

Πj+1=f(Πj,τj,rj)\Pi_{j+1} = f(\Pi_j, \tau_j, r_j)

What it defines: This is the optimization problem that any self-evolving agent system is trying to solve: find a transformation strategy $f$ such that the agent's performance, summed over all tasks it encounters, is maximized. Each application of $f$ produces a new agent $\Pi_{j+1}$ from the previous agent $\Pi_j$, its trajectory $\tau_j$, and its feedback $r_j$.

Why this form: This formulation makes explicit the sequential, cumulative nature of self-evolution. An agent is not evaluated on a single task but on its trajectory of improvement across tasks. This is what distinguishes self-evolution from one-shot fine-tuning: the objective is not to maximize performance on a fixed dataset but to maximize the rate of improvement over an open-ended sequence of tasks. The formulation also reveals why evaluation is challenging (as Section 7 discusses)—measuring cumulative utility requires tracking performance over time, not at a single point.

Operational Definition and the Spectrum from Proto-Evolution to Strong Self-Evolution

The survey recognizes that the field is nascent and that enforcing rigid inclusion criteria would exclude valuable early-stage work. It therefore provides both a conceptual boundary and a spectrum.

Operational definition. A self-evolving agent is defined as one that "modifies its internal parameters, contextual state, toolset, or architectural topology based on its own trajectories or feedback signals, with the explicit objective of improving future performance" (Section 2.1).

Three inclusion criteria. This definition entails three requirements: (i) updates must be experience-dependent, driven by trajectories, self-generated data, or environment feedback, not by generic data synthesis; (ii) updates must produce a persistent, policy-changing effect—the agent behaves differently after evolution than before, and this difference persists across tasks; (iii) the system must possess mechanisms for autonomous exploration or self-initiated learning, even if it also uses pre-collected data.

Why these criteria matter. The experience-dependence criterion distinguishes self-evolution from standard fine-tuning on externally curated datasets. The persistence criterion distinguishes self-evolution from transient in-context learning that does not carry forward to future tasks (a single ICL example that is forgotten after the episode ends does not constitute evolution). The autonomy criterion distinguishes self-evolution from human-scheduled retraining—the agent must have some capacity to initiate its own learning, even if humans also provide guidance.

Proto-evolution vs. strong self-evolution. The survey explicitly includes a spectrum:

"In this survey, we do not impose a rigid exclusion threshold that would disregard early-stage developments. Instead, we analyze the mechanisms contributing to the self-evolving paradigm ranging from proto-evolution (e.g., iterative bootstrapping or feedback-driven prompting) to strong self-evolution (fully autonomous diagnosis and reconfiguration), allowing us to provide a comprehensive view of how diverse methods contribute to the 'What, When, and How' of the paradigm's progression toward full autonomy."

What this means operationally. "Proto-evolution" includes systems like Reflexion, where the agent records natural-language reflections in its episodic memory and uses these to guide future decisions—the evolution is real (the agent behaves differently on future tasks because of its accumulated reflections) but the mechanism is relatively simple (text appended to context). "Strong self-evolution" includes systems like the Darwin Gödel Machine, where the agent recursively rewrites its own Python codebase, with evolution driven by empirical performance and novelty rewards—the evolution is deep (the agent's fundamental capabilities change) and highly autonomous.

Why this spectrum is important. The spectrum framework addresses a tension in the field: if the survey only included fully autonomous systems, it would cover almost no existing work, but if it included everything that merely uses feedback, it would lose coherence. By making the spectrum explicit, the survey can include early-stage methods while transparently acknowledging where they fall short of full autonomy, and can show how different methods contribute different pieces of the eventual autonomy puzzle.

Relationships with Other Paradigms

The survey carefully distinguishes self-evolving agents from four related but distinct paradigms: curriculum learning, lifelong learning, model editing, and unlearning. Table 1 provides a structured comparison across seven properties: runtime evolving, dynamic test-time adaptation, active exploration, structural change, self-reflection and evaluation, context modification, and toolset modification. Self-evolving agents are the only paradigm that checks all seven boxes.

Curriculum learning. This is a training strategy where data are presented in order of increasing difficulty (Bengio et al., 2009). It operates on a static dataset and updates only model parameters. The key differences from self-evolving agents are: (1) curriculum learning does not involve runtime adaptation—the curriculum is designed offline and fixed before training; (2) it does not enable active exploration or structural change; (3) it is fundamentally a single-training-run strategy, not an ongoing evolutionary process. The survey notes that some self-evolving agents use curriculum learning as a component (e.g., WebRL's self-evolving curriculum), but the curriculum is generated autonomously by the agent rather than designed by humans.

Lifelong learning. This refers to the ability of models to continuously learn from new tasks while retaining previously acquired knowledge (Parisi et al., 2019). The survey identifies two fundamental differences from self-evolving agents:

  1. Memory functionality and usage timing: Lifelong learning methods employ memory mechanisms (experience replay buffers, episodic memory) primarily as training-time tools for gradient computation and parameter optimization. Self-evolving agents leverage runtime context (prompts, working memory, conversation history) that directly influences action generation at test-time without requiring parameter updates. The distinction is "training-time replay vs. test-time state adaptation."

  2. Learning initiative: Lifelong learning primarily acquires knowledge passively through externally provided task sequences. Self-evolving agents actively explore their environment and incorporate internal reflection or self-evaluation mechanisms to guide their learning trajectory. They do not just learn from what is given to them—they generate their own learning opportunities.

Model editing and unlearning. These aim to efficiently modify specific knowledge in models while preserving irrelevant knowledge (Wang et al., 2024f). Compared to self-evolving agents, model editing (1) cannot modify non-parametric components such as memory or tools, and (2) relies on a pre-defined pipeline from the algorithm designer. Self-evolving agents can spontaneously employ more diverse and flexible strategies based on environmental observation or internal feedback signals.

Two-lens framework. The survey introduces a useful distinction between a "problem-setting view" and a "solution-paradigm view" (Section 2.2). Curriculum learning and lifelong learning arise from concrete learning problems (how to structure training data, how to avoid catastrophic forgetting)—they are defined by the problems they solve. Model editing and self-evolving agents, in contrast, originate as solutions—they propose mechanisms for updating or modifying a system. Self-evolving agents generalize model editing by treating adaptation as a first-class capability, allowing not only parameter updates but also changes to runtime context, memory, tools, and workflow structures.

Why this distinction matters. The problem-setting vs. solution-paradigm distinction clarifies why self-evolving agents resist comparison with established paradigms: they are not defined by a specific problem (like catastrophic forgetting) but by a specific capability (the ability to autonomously improve across multiple dimensions). This means they can incorporate elements of curriculum learning, lifelong learning, and model editing as sub-components of a broader evolutionary strategy. The survey's taxonomy provides the language for describing when and how these sub-components are employed.

The What Dimension: Four Evolutionary Loci

The what dimension decomposes the agent system $\Pi = (\Gamma, \{\psi_i\}, \{C_i\}, \{W_i\})$ into four pillars, each representing a category of internal state that can be autonomously modified. Table 2 maps representative methods to these pillars, showing which dimensions each actively evolves.

Model Evolution

Model evolution encompasses two sub-dimensions: policy evolution (refining model parameters for better task performance) and experience evolution (learning from and constructing environments to generate learning signals).

Policy evolution. The key innovation in this sub-area is self-generated training data—agents autonomously creating the problems they train on, rather than depending on human-curated datasets. The Self-Challenging Agent (SCA) (Zhou et al., 2025e) exemplifies this: a language model alternates between a "challenger" role that generates executable Code-as-Task problems and an "executor" role that solves them. Successful solution trajectories are then used for fine-tuning. This creates a closed loop where the agent generates its own curriculum.

Why this approach over alternatives. Traditional agent training data collection (human annotation of tool-use trajectories) is expensive and yields limited coverage. Purely synthetic data-generation pipelines without self-challenge suffer from inadequate quality—the problems are too easy or too uniform. The self-challenging mechanism ensures that generated problems are at the frontier of the agent's current capability because the challenger role is played by the same model that will attempt to solve them. If the solver can already solve all generated problems, the challenger must generate harder ones.

Experience evolution. This extends policy evolution by having agents actively interact with or construct their environments to capture experiences that drive iterative improvement. AgentGen (Hu et al., 2024b) synthesizes diverse simulation worlds (in PDDL or Gym-style formats) from an initial corpus and implements a bidirectional evolution loop that progressively adjusts task difficulty. Reflexion (Shinn et al., 2023) introduces self-reflective mechanisms where agents record natural-language critiques of their previous actions, guiding future behavior. AdaPlanner (Sun et al., 2023) enables closed-loop adaptive planning where agents refine strategies on-the-fly based on environmental feedback.

The environment loop. The critical insight is that self-evolving models do not merely consume fixed datasets—they are embedded in an environmental loop: the agent generates actions, the environment provides feedback (execution traces, success/failure signals, natural-language critiques), and this feedback is transformed into learning signals (preference pairs for DPO, reward values for RL, or training targets for SFT). RAGEN (Wang et al., 2025q) formalizes this by modeling multi-step tool-use tasks as Markov Decision Processes and using dense environmental feedback for policy gradient updates. DYSTIL (Wang et al., 2025c) takes a different approach, using high-level strategy advice generated by language models to progressively internalize complex decision-making skills into RL agents.

Context Evolution

Context evolution addresses the question: what information should be present in the agent's context window to shape its behavior optimally? The survey distinguishes two sub-components: memory evolution (what past information to store, forget, and retrieve) and prompt optimization (how to phrase and structure instructions).

Memory evolution. The key mechanism is an evolving long-term memory that grows and adapts as the agent solves tasks and interacts with its environment. Different systems operationalize this differently:

  • SAGE (Liang et al., 2024) uses the Ebbinghaus forgetting curve—a psychological model of how human memory decays over time—to decide what to remember or forget. When a memory's "strength" falls below a threshold based on this curve, it is forgotten.

  • Mem0 (Chhikara et al., 2025) implements a two-phase pipeline: first, the agent extracts salient facts from recent dialogue; second, it decides how to update long-term memory through three operations—ADD new facts, MERGE/UPDATE redundant ones, or DELETE contradictions. This ensures the memory remains coherent rather than accumulating contradictory information.

  • Memory-R1 (Yan et al., 2025) trains a dedicated Memory Manager agent using reinforcement learning to select structured operations (ADD, UPDATE, DELETE) based on a reward signal that balances memory coherence, retrieval accuracy, and storage efficiency.

  • MemGen (Zhang et al., 2025f) introduces a fundamentally different approach: instead of storing explicit text in memory, it operates in a latent space. A learned memory trigger decides when to invoke memory, and a "weaver" constructs latent token sequences, enabling fluid interweaving of reasoning and memory without explicit retrieval.

Experience distillation. Beyond storing raw experiences, advanced memory systems distill experiences into generalizable guidance. Expel (Zhao et al., 2024a) processes past trajectories to generate insights and rules that guide future interactions—transforming specific episodes ("I tried approach X on problem Y and failed because of Z") into general heuristics ("when encountering problems with property Y, avoid approach X because it tends to fail due to Z"). Agent Workflow Memory (Wang et al., 2024j) records common sub-task sequences (workflows) so the agent can retrieve and reuse proven action sequences rather than planning from scratch.

Prompt optimization. This transforms prompt text into a learnable component that co-evolves with the agent's experience. The survey organizes methods along a spectrum of increasing autonomy:

  • Search-based methods treat instruction design as an optimization problem. APE (Zhou et al., 2022) generates candidate prompts, scores them on validation examples, and selects the best. PromptAgent (Wang et al., 2023d) casts prompt discovery as Monte-Carlo Tree Search, exploring instruction space strategically. ProTeGi (Pryzant et al., 2023) generates natural-language "corrections" that are applied as edits to the prompt, forming a textual analogue of gradient descent.

  • Evolutionary methods maintain populations of prompts and apply mutation and selection. PromptBreeder (Fernando et al., 2023) discovers increasingly effective instructions through this population-based approach.

  • Fully autonomous methods eliminate the need for external validation data. SPO (Xiang et al., 2025) creates a self-contained loop where the model generates its own training data and uses pairwise preference comparison on its outputs to refine the prompt, requiring no external labeled data or human feedback.

Multi-node prompt optimization. In complex agent systems that orchestrate sequences of LLM calls, prompt optimization becomes a multi-node problem. DSPy (Khattab et al., 2023) represents an entire workflow as a graph whose sub-prompts are jointly tuned for a global objective. TextGrad (Yellamraju et al., 2024) and LLM-AutoDiff (Yin & Wang, 2025) treat each prompt as a parameter in a differentiable program and propagate natural-language "gradients" backward through the workflow to refine every step simultaneously.

Why prompt optimization matters for self-evolution. Prompt optimization is a form of evolution that requires no weight updates, making it computationally cheap and safe (it cannot catastrophically forget pre-trained knowledge). It directly alters the model's behavior by changing its inputs rather than its parameters, which means it can be applied at test-time without any training infrastructure.

Tool Evolution

Tool evolution represents one of the most ambitious forms of self-evolution because it requires the agent to create, refine, and manage executable code—capabilities that directly expand the agent's action space.

Autonomous tool creation. The primary motivation is overcoming the limitations of a fixed toolset. The survey identifies three approaches along a spectrum:

  • Emergent creation through exploration: Voyager (Wang et al., 2023a) builds an ever-expanding library of skills in Minecraft through trial-and-error, driven by intrinsic motivation to explore. When the agent encounters a situation it cannot handle with existing skills, it attempts to write new code, tests it in the environment, and adds successful implementations to a persistent skill library. The skill library grows automatically as the agent explores more of the game world.

  • Reactive creation on capability gaps: ATLASS (Haque et al., 2025), Alita (Qiu et al., 2025b), and Live-SWE-Agent (Xia et al., 2025a) create new tools from scratch or employ retrieval-augmented generation to search open-source code repositories the moment a capability gap is identified. When the agent realizes it lacks a tool for a specific subtask, it immediately attempts to create or find one.

  • Structured creation through formal synthesis: CREATOR (Qian et al., 2023b) disentangles abstract tool creation (reasoning about the general structure of a reusable function) from concrete tool usage (deciding how to apply that function to a specific instance). SkillWeaver (Zheng et al., 2025a) analyzes successful human or agent task trajectories to propose, synthesize, and hone new skills into robust, reusable APIs.

Iterative tool refinement. A newly generated tool is typically a brittle script, not a reliable function. The mastery process involves a self-correction loop where the agent learns from its own experience. This requires solving the credit assignment problem: determining precisely which line of code or which parameter was responsible for a failure. The agent analyzes multiple feedback signals—compiler errors, unexpected API return values, environmental state changes, implicit signals from user actions—and uses these to debug not only the tool's code but also its documentation (docstrings, argument descriptions), which is crucial for correct future usage.

Tool management and selection. As the skill library grows into hundreds or thousands of tools, the challenge shifts from creation to efficient management. ToolGen (Wang et al., 2025k) represents a paradigm shift: it encodes tools as unique tokens within the language model's vocabulary, reframing tool retrieval as a generation problem—the model predicts the most appropriate tool as a natural continuation of its thought process, leveraging the transformer's pattern recognition capabilities. TOOLMEM (Xiao et al., 2025b) enables agents to learn and store the strengths and weaknesses of different tools in a dedicated memory, optimizing selection for specific task requirements.

The closed evolutionary cycle. The survey frames tool evolution as an aspirational closed cycle: an autonomous agent that can perceive gaps in its capabilities, create novel solutions, master them through practice, and seamlessly integrate them into a coherently managed and ever-expanding repertoire. This vision blurs the distinction between the agent and its tools—in the limit, as in the Darwin Gödel Machine, the agent can rewrite its own core code, making tool evolution indistinguishable from agent evolution.

Architecture Evolution

Architecture evolution is the most structural form of self-evolution—it modifies how the agent's components are organized and how they communicate, rather than modifying the components themselves.

Single-agent architecture optimization. This operates at two levels:

  • Node-level optimization refines individual LLM-invoking nodes within a fixed topology. TextGrad (Yellamraju et al., 2024) propagates "textual gradients" (natural-language critiques) from the final output backward through the workflow, refining each node's prompt without altering the overall structure.

  • Holistic architecture search treats the agent's entire design as an optimizable object. AgentSquare (Shang et al., 2025) defines a modular design space of components (planners, memory modules, tool-use components) and uses an evolutionary algorithm to find the most effective combination. EvoFlow (Zhang et al., 2025c) uses evolutionary algorithms to construct heterogeneous workflows by selecting the most suitable LLM for each task from a diverse pool, optimizing for both performance and cost through a multi-objective fitness function.

  • Recursive self-modification represents the most radical form. The Darwin Gödel Machine (Zhang et al., 2025h) recursively modifies its own Python codebase across generations. AlphaEvolve (Novikov et al., 2025) uses evolutionary coding to improve specific algorithms. Gödel Agent (Yin et al., 2025) provides a self-referential framework where agents analyze and alter their own logic. MemEvolve (Zhang et al., 2025g) evolves not just the agent's experiential memory but the memory system's architecture itself through a bilevel optimization process that adapts encoding, storage, and retrieval mechanisms.

Multi-agent architecture optimization. This addresses the problem of discovering optimal collaboration patterns among multiple agents:

  • Workflow optimization treats agent collaboration as a search problem. ADAS (Hu et al., 2024c) frames system design as search through a Turing-complete space of code-based configurations. AFlow (Zhang et al., 2024c) makes this practical by introducing reusable operators representing common agentic patterns and employing Monte Carlo Tree Search (MCTS) to efficiently navigate the enormous design space. The key insight is that the space of possible multi-agent workflows is combinatorially vast, and systematic search discovers configurations that human designers would not anticipate.

  • Query-specific generation creates customized agent systems for each specific query. MaAS (Zhang et al., 2025d) creates a "supernet" of potential architectures and samples a specialized system for each query. ScoreFlow (Wang et al., 2025n) trains a generator using preference optimization to produce effective multi-agent topologies. FlowReasoner (Gao et al., 2025) uses reinforcement learning to train a meta-agent that constructs bespoke workflows on the fly.

  • Multi-agent co-evolution focuses on how multiple autonomous agents evolve their internal behavioral policies through interaction, enabling emergent coordination, task delegation, and beneficial competition. ReMA (Wan et al., 2025) uses multi-agent reinforcement learning (MARL) to collaboratively train a high-level meta-thinker and a low-level executor. GiGPO (Feng et al., 2025b) enhances MARL training by aggregating trajectories for more precise credit assignment.

The cost challenge. A key practical challenge is the computational cost of evaluating each potential workflow. Agentic Predictor (Trirat et al., 2025) addresses this by training a lightweight model to estimate workflow performance based on structural and semantic features without full execution, significantly accelerating the search process.

The When Dimension: Temporal Decomposition

The when dimension distinguishes two temporal modes of self-evolution based on their relationship to task execution. This distinction is captured in Figure 5, which shows intra-test-time evolution as a pathway where adaptation occurs within task execution (variant generation, verification, policy update) and inter-test-time evolution as a pathway where learning happens retrospectively (rollout, trajectory analysis, policy update).

Intra-test-time self-evolution refers to adaptive processes that occur during task execution. The defining characteristic is synchronous coupling: feedback signals are generated and processed while the agent is actively working on a problem, and optimization directly targets performance on the current problem instance. The agent recognizes its limitations on a specific problem and initiates targeted learning to enhance capabilities in real-time.

Inter-test-time self-evolution refers to learning processes that occur between task completions, leveraging accumulated experiences to improve future performance. The defining characteristic is retrospective decoupling: learning operates on historical data (curated datasets or accumulated behavioral trajectories), with optimization oriented toward improving expected performance across the task distribution.

How learning paradigms differ across temporal modes. The survey shows that ICL, SFT, and RL are instantiated differently depending on timing:

  • ICL in intra-test-time uses the context window as a dynamic memory system for immediate adaptation. The agent analyzes its performance, generates verbal critiques, and maintains reflections in episodic memory buffers to guide subsequent decisions within the same task (Shinn et al., 2023; Madaan et al., 2023a). In inter-test-time, ICL leverages execution results from previous tasks as contextual information for future problem-solving, with the agent progressively improving as task-relevant information accumulates in the context (Wang et al., 2024j; Laskin et al., 2022).

  • SFT in intra-test-time represents a paradigm shift where models perform immediate self-modification through learned meta-adaptation strategies. Self-adaptive language modeling (Zweiger et al., 2025) generates "self-edits"—meta-level instructions that can restructure information representations, specify optimization hyperparameters, or invoke tools for data augmentation and gradient computation—triggering immediate fine-tuning. In inter-test-time, SFT methods establish iterative self-improvement through synthetic data generation and self-evaluation: models generate responses to unlabeled instructions, enhance them through self-critique, and fine-tune on the improved outputs (Lu et al., 2023; Zelikman et al., 2022).

  • RL in intra-test-time enables models to develop new capabilities on-demand when encountering problems beyond their current competence. LADDER (Simonds & Yoshiyama, 2025) implements test-time reinforcement learning (TTRL): upon identifying a challenging problem, the system generates focused problem variants and conducts intensive, targeted RL specifically for that problem class. In inter-test-time, RL leverages unconstrained computational resources for extensive environmental interaction: RAGEN (Wang et al., 2025q) employs online RL for multi-turn tool-use tasks, WebRL (Qi et al., 2024) develops web navigation agents through self-evolving curricula that automatically adjust task complexity, and DigiRL (Bai et al., 2024) enables device-control agents to master interactions through autonomous RL.

Why the temporal dimension matters. The choice of timing has profound implications: intra-test-time evolution is constrained by latency requirements (the agent must still complete the task) and can only invest compute proportional to task difficulty; inter-test-time evolution can use arbitrarily large compute budgets but cannot help with the current task. Systems that combine both (e.g., learning meta-strategies offline and applying them online) may achieve the best of both worlds.

The How Dimension: Evolutionary Mechanisms

The how dimension categorizes the mechanisms by which evolution is driven, progressing through three major paradigms that emerged to address limitations of previous ones.

Reward-Based Self-Evolution

Reward-based methods center on the design of feedback signals to guide iterative self-improvement. The survey categorizes these by the nature of the feedback:

Textual feedback leverages natural language to provide detailed, interpretable instructions for refinement. Unlike scalar rewards, textual feedback encapsulates nuanced critiques and actionable suggestions. Reflexion (Shinn et al., 2023) proposes "verbal reinforcement learning" where agents reflect in natural language on past trials and store these reflections as episodic memory. Self-Refine (Madaan et al., 2023a) and SELF (Lu et al., 2023) demonstrate that even state-of-the-art models can be improved via multi-turn, language-based self-critique without additional supervised data or external reinforcement. TextGrad (Yellamraju et al., 2024) treats unstructured textual feedback as a differentiable signal capable of directly influencing both prompt design and model parameters.

Internal rewards exploit intrinsic metrics such as the model's probability estimates or certainty. Confidence-Informed Self-Consistency (CISC) (Taubenfeld et al., 2025) weights reasoning paths by confidence scores to improve accuracy and efficiency. Self-Rewarding Language Models (Yuan et al., 2025b) demonstrate that models can act as their own reward function, generating training data through self-instruction and self-evaluation cycles. AgentEvolver (Zhai et al., 2025) proposes a self-attributing mechanism that uses the LLM's reasoning to retrospectively assign step-wise rewards that are dense and semantically grounded for policy optimization.

External rewards derive from sources outside the model—the environment, majority voting, or explicit rules. Majority voting (Shafayat et al., 2025; Wei et al., 2025c) uses consensus among multiple model outputs as a proxy for correctness. Environment feedback is central to agentic LLM research (e.g., SWE-Dev, SICA, DYSTIL) where agents learn through direct interaction with real-world environments and tools. Rule-based rewards (Wang et al., 2025h; Wang & Xiong, 2025) use explicit constraints or logical rules as verifiable signals, particularly effective in mathematical reasoning and structured problem solving.

Implicit rewards hypothesize that LLMs can learn from feedback signals even when not explicitly labeled as rewards. "Reward Is Enough" (Song et al., 2025) demonstrates that LLMs can perform in-context reinforcement learning using simple scalar signals embedded in the context window. Endogenous reward (Li et al., 2025e) reveals that standard next-token prediction implicitly learns a generalist reward function that can be extracted from model logits without additional training.

Why reward design matters. The choice of reward signal determines the nature, efficiency, and effectiveness of learning. Textual feedback provides rich, interpretable guidance but may be noisy or inconsistent. Scalar rewards are precise but sparse, requiring sophisticated credit assignment. Internal rewards enable fully autonomous improvement but may suffer from confirmation bias. External rewards provide objective supervision but may require significant engineering.

Imitation and Demonstration Learning

Imitation learning in self-evolving agents generalizes the traditional paradigm: the "expert" is not necessarily a fixed human but any source of high-quality demonstration—a past successful trajectory, a more capable agent, or synthesized environmental interactions.

Self-generated demonstration learning involves agents creating their own training data through iterative refinement. STaR (Zelikman et al., 2022) bootstraps reasoning by generating reasoning chains, fine-tuning on correct solutions, and repeating this cycle. V-STaR (Hosseini et al., 2024) adds verifier-guided self-training, where separate verifier models assess the quality of generated reasoning chains before they enter training data. AdaSTaR (Koh et al., 2025) introduces adaptive data sampling that dynamically adjusts training data composition based on model performance across reasoning tasks.

Cross-agent demonstration learning involves agents learning from demonstrations provided by other agents. SiriuS (Zhao et al., 2025b) maintains an experience library containing successful interaction trajectories generated by different agents, enabling efficient knowledge sharing. Each agent can leverage the collective experience of the entire system.

Hybrid demonstration learning combines self-generated and external demonstrations. RISE (Qu et al., 2024b) enables agents to introspect on their reasoning, identify weaknesses, and generate corrective demonstrations. Confidence-based approaches (Li et al., 2024b) use the model's uncertainty estimates to select high-quality demonstrations from both sources.

Why imitation learning over reward-based methods. Imitation learning is prescriptive and exemplar-based: the agent receives a complete, successful guide and learns to reproduce this behavior. This is more sample-efficient than trial-and-error reward learning when high-quality demonstrations are available, but it constrains the agent to behaviors within the demonstration distribution and may limit exploration of novel strategies.

Population-Based and Evolutionary Methods

Population-based methods maintain multiple agent variants simultaneously, enabling parallel exploration through selection, mutation, crossover, and competitive interaction.

Single-agent evolution uses population mechanisms applied to variants of a single agent:

  • Learning from evolution applies biological operators directly. The Darwin Gödel Machine (DGM) (Zhang et al., 2025h) maintains an archive of all historical versions, enabling branching from any past "species." GENOME (Zhang et al., 2025r) directly applies genetic algorithms to language model parameter evolution, using crossover, mutation, and selection operators on model weights. GENOME+ extends this with particle swarm optimization, adding inheritance mechanisms and ensemble methods.

  • Self-play is a paradigm where agents improve through iterative interaction with versions of themselves. SPIN (Chen et al., 2024f) has current models compete against previous versions, creating evolutionary pressure. SPC (Chen et al., 2025c) features adversarial co-evolution where a "sneaky generator" creates deceptive errors and a "step critic" learns to detect them. Absolute Zero (Zhao et al., 2025a) co-evolves a reasoning agent and its internal self-reward model through reinforced self-play, with a challenger generating problems at the frontier of a solver's capabilities.

Multi-agent evolution extends population methods to evolving entire teams:

  • System architecture evolution optimizes team composition, orchestration strategies, and workflow structure. EvoMAC (Hu et al., 2024d) implements "textual backpropagation" where compilation errors and test failures serve as loss signals to drive iterative modifications of agent team composition and prompts. Puppeteer (Dang et al., 2025) evolves coordination strategy through reinforcement learning, dynamically selecting which agents to activate at each step.

  • Knowledge-based evolution emphasizes evolving collective knowledge and experience through memory accumulation and case-based learning. MDTeamGPT (Chen et al., 2025e) implements a dual knowledge base system (CorrectKB for successful cases, ChainKB for failure reflections). MedAgentSim (Almansoori et al., 2025b) accumulates experience from patient interactions and uses retrieval-augmented generation to improve consultation quality over time.

Why population-based methods. They enable broader search coverage and the discovery of novel solutions that might be missed by gradient-based optimization. The trade-off is computational cost: evaluating multiple agent variants requires more resources than optimizing a single agent.

Cross-Cutting Evolutionary Dimensions

The survey identifies three orthogonal design choices that cut across the reward-based, imitation-based, and population-based paradigms:

Online vs. offline learning. This dimension concerns whether evolutionary updates occur through continuous interaction with a live environment (online) or on a static, pre-collected dataset (offline). Online methods enable real-time adaptation to dynamic situations but require the agent to learn from its mistakes without the safety net of curated data. Voyager (Wang et al., 2023a) exemplifies online evolution through its exploration-driven skill acquisition in Minecraft. Offline methods decouple learning from execution, enabling cycles of data generation, filtering, and model fine-tuning before deployment. OS-Genesis (Sun et al., 2024b) creates trajectories through reverse task synthesis for offline SFT.

On-policy vs. off-policy learning. This dimension concerns policy consistency—whether agents learn from experiences generated by their current policy (on-policy) or from experiences generated by different policies (off-policy). On-policy methods ensure that the learning signal matches the current behavior distribution but suffer from low sample efficiency because each policy update requires fresh data collection. Off-policy methods reuse historical data, expert demonstrations, or cross-agent experiences, significantly improving sample efficiency but introducing distribution shift challenges that require careful regularization.

Reward granularity. This dimension concerns the level of detail at which the agent receives its learning signal:

  • Outcome-based rewards evaluate only the final state—successful task completion or not. This is simple to implement but provides sparse signals that make credit assignment difficult, especially for long-horizon tasks.

  • Process-based rewards evaluate each step in the agent's trajectory. Process reward models (PRMs), as demonstrated by Lightman et al. (2023) and Math-Shepherd (Wang et al., 2023b), are significantly more reliable than outcome-based models for complex reasoning. The challenge is obtaining step-level supervision: traditional methods require expensive human annotation, but recent work (Math-Shepherd, AlphaMath) uses automatic annotation through Monte Carlo Tree Search.

  • Hybrid rewards combine outcome signals (providing clarity on final success) with process signals (providing granular step-level guidance). GiGPO (Feng et al., 2025a) provides episode-level rewards based on final success together with localized step-level rewards for intermediate actions. SPA-RL (Wang et al., 2025d) decomposes outcome rewards across process steps, creating dense intermediate progress rewards that enhance RL effectiveness while maintaining alignment with the ultimate task objective.

Why these dimensions matter for system design. They represent fundamental trade-offs: online learning provides adaptability at the cost of stability; off-policy learning provides efficiency at the cost of distribution mismatch; process rewards provide detailed guidance at the cost of annotation complexity. A well-designed self-evolving system must make explicit choices along each dimension based on the specific requirements of its task domain, available supervision, and computational constraints.

The Evaluation Framework

Section 7 of the survey provides the measurement infrastructure for assessing whether evolution is actually occurring. This is not merely an appendix to the taxonomy but a central component: the survey argues that evaluation must be co-designed with the evolutionary mechanisms themselves, and current evaluation practices fundamentally misalign with what self-evolving agents are trying to achieve.

Five evaluation goals. The survey structures evaluation around five core dimensions that a capable self-evolving agent must satisfy:

  1. Adaptivity: Measuring the ability to improve performance on in-domain tasks through experience. Primary metrics include Success Rate by Iteration Steps and Adaptation Speed—tracking performance as a function of the agent's interaction history.

  2. Retention: Measuring the stability of knowledge—specifically, resistance to catastrophic forgetting. Key metrics are Forgetting (FGT), defined as:

FGTt=1t−1∑i=1t−1[max⁡j∈{i,…,t}Ji,j−Ji,t]FGT_t = \frac{1}{t-1} \sum_{i=1}^{t-1} \left[\max_{j \in \{i,\ldots,t\}} J_{i,j} - J_{i,t}\right]

where $J_{i,t}$ is the performance on task $i$ after completing $t$ tasks. This subtracts current performance from the best historical performance on each old task, averaging across all old tasks. Higher FGT means more forgetting.

and Backward Transfer (BWT), defined as:

BWTt=1t−1∑i=1t−1[Ji,t−Ji,i]BWT_t = \frac{1}{t-1} \sum_{i=1}^{t-1} \left[J_{i,t} - J_{i,i}\right]

where positive values indicate that new learning positively benefits old tasks (successful knowledge transfer).

  1. Generalization: Measuring the ability to apply accumulated knowledge to new, unseen domains. Evaluated through aggregate performance across multi-domain test suites and out-of-domain performance on held-out task distributions.

  2. Efficiency: Quantifying the resourcefulness of the evolutionary process. The survey provides a refined cost taxonomy with five dimensions: token usage ($C_{token}$), step count/latency ($C_{step}$), wall-clock time ($C_{time}$), tool/API calls ($C_{tool}$), memory growth ($C_{mem}$), and human oversight ($C_{human}$). The aggregate metric is Cost-per-Gain (CPG):

CPGt=Total CosttPerformance Gaint+ϵCPG_t = \frac{\text{Total Cost}_t}{\text{Performance Gain}_t + \epsilon}

where cost can be measured in tokens, time, memory, or a normalized composite. This normalizes resource consumption by the improvement achieved, enabling comparison across methods with different cost profiles.

  1. Safety: Examining whether agents develop unsafe behavioral patterns during evolution. Metrics include Safety Score (proportion of test cases where behavior is labeled safe), Harm Score (graded assessment of harmful outputs), Completion Under Policy (task success while complying with constraints), Risk Ratio (frequency of policy violations), Refusal Rate (proportion of unsafe tasks declined), and Leakage Rate (frequency of sensitive information disclosure).

Three temporal evaluation paradigms:

  • Static assessment evaluates instantaneous performance at a specific point in time using traditional benchmarks (SWE-bench, WebArena, GAIA, AgentBench). This captures baseline capabilities but inherently cannot capture the dynamic, continuous learning that distinguishes self-evolving agents.

  • Short-horizon adaptive assessment evaluates improvement over a limited number of interactions. This is operationalized either by augmenting traditional benchmarks with a temporal dimension (tracking success rate by iteration steps) or through benchmarks with built-in dynamic evaluation (e.g., MemoryAgentBench's Test-Time Learning dimension).

  • Long-horizon lifelong learning assessment evaluates continuous knowledge accumulation, retention, and transfer over extended task sequences. Benchmarks like LifelongAgentBench (Zheng et al., 2025b) construct sequences of interdependent tasks requiring progressive skill building. Metrics like FGT and BWT are designed for this paradigm.

Coverage gaps (Table 8). The survey systematically identifies where current evaluation falls short:

  • Adaptivity benchmarks lack open-ended exploration without predetermined objectives
  • Retention assessment is undermined by episodic designs that reset state between tasks
  • Generalization evaluation fails to capture temporal robustness under distribution drift
  • Efficiency evaluation imposes no enforced budgets during the evolution process
  • Safety evaluation remains entirely static, with no tracking of safety drift over extended evolution

Self-directedness trade-offs. A key insight of the evaluation framework is that the degree of self-directedness—whether the agent autonomously generates tasks and evolution strategies—creates fundamental trade-offs. Highly self-directed systems like WebRL achieve substantial gains (4.8% to 42.4%) through autonomous curriculum generation, but autonomous evolution incurs measurable risks: alignment faking rates escalated from 12% to 78% when agents autonomously evolved under conflicting objectives (Greenblatt et al., 2024). The survey recommends transparently reporting three aspects of the evolution process: the source of evolution strategies, the source of feedback signals, and the frequency of external interventions.

Standardized protocols (Table 10). To address the heterogeneity of existing setups, the survey distills standardized evaluation protocols for short-horizon and long-horizon assessment, specifying requirements for state persistence, dataset structure, evolution budgets, logging, primary metrics, efficiency metrics, safety auditing, human-in-the-loop statistics, and required outputs. The protocols are illustrated through a worked example (EvoAgent) that demonstrates both alignment with recommended practices and current gaps.

Why evaluation is inseparable from the taxonomy. The what-when-how dimensions define the design space of evolutionary mechanisms; the evaluation framework defines how we measure whether those mechanisms work. The survey's central argument is that these must be co-designed: if we evaluate self-evolving agents using static benchmarks designed for fixed-capability systems, we will systematically underestimate the value of evolutionary mechanisms and misattribute performance differences to the wrong factors. The evaluation framework thus completes the taxonomy by providing the measurement layer that enables rigorous comparison and cumulative progress.

4. Key Insights and Innovations

Innovation 1: Self-Evolution as a First-Class Design Dimension, Not a Bolted-On Feature

The most fundamental intellectual contribution of this survey is not any single method or finding, but the conceptual reframing of self-evolution from a property that some agents incidentally possess to the central organizing principle around which agent architectures should be designed and analyzed. Prior to this work, self-evolution was treated as one capability among many in the agent design space—something you might add to your agent, alongside planning, tool use, and memory, if you wanted it to improve over time. Surveys like Luo et al. (2025a) and Liu et al. (2025a) discussed evolution as a subsidiary component within comprehensive agent taxonomies, but never as the primary axis of analysis.

This survey inverts that framing. By decomposing self-evolution into the orthogonal what-when-how dimensions and showing how every design choice in an agent system can be understood through this lens, the survey establishes that self-evolution is not a feature you add—it is a perspective from which you evaluate the entire system. An agent with memory but no mechanism for evolving that memory based on experience is not "an agent plus memory"—it is a self-evolving agent that has made a specific (and probably suboptimal) choice along the "what to evolve" dimension (choosing not to evolve memory). An agent that learns from environmental rewards but only during an offline training phase is not "an agent that does RL"—it is a self-evolving agent that has chosen inter-test-time evolution with external rewards along the when and how dimensions.

This reframing matters because it creates comparability across methods that previously appeared incommensurable. Before this survey, how would you compare Reflexion (which modifies context through verbal reflection during task execution) with RAGEN (which modifies model weights through policy gradients between tasks) with ADAS (which modifies multi-agent topology through Monte Carlo Tree Search)? They operate on different components, at different times, using different mechanisms—any direct performance comparison would be confounded by these orthogonal differences. The what-when-how taxonomy provides the language to say: these systems differ along all three dimensions, so comparing their raw accuracy tells us little about which dimension choices matter. What we should compare instead is (a) systems that differ along only one dimension, or (b) the patterns of trade-offs that emerge across the full design space.

This is a fundamental conceptual advance rather than an incremental refinement. It is analogous to what the Chinchilla scaling laws (Hoffmann et al., 2022) did for pretraining compute allocation: prior to Chinchilla, the relationship between model size and data quantity was understood qualitatively but not systematically; Chinchilla provided a framework for analyzing the trade-off as a function of total compute budget. Similarly, prior to this survey, the relationship between what/when/how choices was understood only through isolated case studies; the survey provides the framework for analyzing these choices as orthogonal dimensions of a unified design space. The fact that the survey is a taxonomic synthesis rather than an empirical study does not diminish this contribution—shared conceptual vocabulary is a prerequisite for cumulative progress, and the field lacked it.

The evidence for this innovation's significance is not a single figure or table but the coherence of the resulting taxonomy. Table 2 maps 18 representative methods to the four evolutionary loci, showing that methods previously discussed in separate subcommunities (tool creation, memory management, architecture search) can be positioned within the same framework. Table 3 maps methods across feedback type, feedback source, learning method, updated components, and update timing—showing that the how dimension can be systematically decomposed into orthogonal sub-choices. The fact that these mappings are possible and illuminating is evidence that the taxonomy captures real structure in the design space, not arbitrary categorization.

Innovation 2: The Proto-Evolution to Strong Self-Evolution Spectrum as a Diagnostic Tool

The survey's second distinctive contribution is the articulation of a spectrum from proto-evolution to strong self-evolution, which serves simultaneously as an inclusion criterion (allowing the survey to cover nascent work without losing coherence), a diagnostic tool (enabling precise characterization of where any given system falls on the autonomy gradient), and a research roadmap (identifying which capabilities must be developed to move from proto to strong evolution).

What makes this framing novel. Prior work recognized implicitly that some systems were "more autonomous" than others, but this was typically expressed as a binary: either a system was self-improving (and therefore interesting) or it was not. The binary framing created two pathologies. First, it encouraged researchers to overclaim autonomy—describing systems as "self-evolving" when they merely applied a fixed fine-tuning recipe to self-generated data, without any autonomous exploration, diagnosis, or decision-making about what to improve. Second, it made it impossible to appreciate partial progress—a system that could autonomously generate its own training tasks but still required human-designed reward functions was lumped together with systems that had no autonomous components at all.

The spectrum framing resolves both pathologies. By making the degree of autonomy an explicit variable, the survey provides precise language for characterizing systems: "proto-evolution" includes methods like Reflexion that modify context based on self-reflection but do not modify model weights or tools; intermediate systems might autonomously generate training data but still require human-specified update schedules; "strong self-evolution" is reserved for systems that autonomously diagnose their own limitations, generate appropriate learning experiences, and reconfigure themselves across multiple dimensions. This spectrum is operationalized through the three inclusion criteria in Section 2.1: experience-dependence (is the update driven by the agent's own trajectories?), persistence (does the update produce a lasting change in behavior?), and autonomous exploration (does the agent initiate its own learning?).

Why this is more than taxonomy. The spectrum is not merely a classification device—it is a diagnostic tool for identifying what is missing. If a system achieves experience-dependent, persistent policy changes but requires humans to specify what to learn and when, it is stuck at an intermediate point on the spectrum because it lacks autonomous exploration mechanisms. This diagnosis immediately suggests a research direction: how can we add self-directed curriculum generation (as in WebRL) or self-diagnosis of capability gaps (as in Voyager's skill library expansion) to close the autonomy gap? The spectrum thus doubles as a research roadmap, identifying which capabilities are prerequisites for others.

The survey demonstrates the spectrum's utility through concrete examples. Table 2 shows that some methods evolve model parameters but not tools or architecture—they are proto-evolutionary in the sense that they modify only one component. Methods like the Darwin Gödel Machine that modify code, architecture, and tools simultaneously represent stronger evolution. The evaluation framework in Section 7 further reinforces the spectrum by identifying self-directedness as a variable that creates fundamental trade-offs (e.g., autonomous curriculum generation improves adaptivity but may increase safety risks), implying that stronger evolution is not universally better—it is appropriate only when the benefits of autonomy outweigh the risks.

Innovation 3: The Identification of Evaluation as the Critical Missing Infrastructure

The survey's third distinctive insight is that the primary bottleneck in self-evolving agent research is not algorithmic but evaluative—the field lacks the measurement infrastructure to determine whether evolution is actually occurring, and without this infrastructure, we cannot make principled comparisons or accumulate knowledge. This is a diagnostic contribution rather than a methodological one: the survey does not propose new evaluation benchmarks but rather systematically documents why existing evaluation practices are fundamentally misaligned with the phenomenon they claim to measure.

The mismatch diagnosis. The survey identifies a structural problem: self-evolving agents are defined by their capacity to improve over time (the $\max_f \sum_{j=0}^n U(\Pi_j, T_j)$ objective in Section 2.1), yet they are evaluated using static benchmarks that measure instantaneous performance at a single point in time. This is not merely imperfect—it is category error. A system that achieves 40% accuracy on SWE-bench after self-training tells us nothing about whether it improved from a baseline of 10% (impressive evolution) or 38% (negligible evolution). The episodic evaluation protocols used by most benchmarks, where agent state resets between tasks, "fundamentally preclude measurement of knowledge accumulation or degradation—precisely the phenomena that distinguish self-evolving agents from static systems."

What makes this insight distinctive. Prior work acknowledged that evaluation was challenging (e.g., individual papers would note that their benchmarks didn't capture long-term learning), but treated this as a practical limitation to be addressed later. This survey elevates the mismatch to a structural diagnosis: the field's inability to answer basic questions about self-evolving agents—does agent A learn faster than agent B? does agent A retain knowledge longer than agent B? does agent A's safety drift over time?—is not due to insufficient benchmark coverage but to a fundamental misalignment between what evaluation measures and what self-evolution claims.

The five-goal evaluation framework (adaptivity, retention, generalization, efficiency, safety) makes this diagnosis operational. By showing that current benchmarks primarily address adaptivity while leaving retention, efficiency, and safety severely underserved (Table 8), the survey reveals that we are evaluating self-evolving agents on the dimension that is easiest to measure (instantaneous accuracy) while ignoring the dimensions that distinguish them from static systems (knowledge accumulation, resource costs of evolution, safety drift). The coverage gaps in Table 8—"episodic designs reset state between tasks; no retention with safety constraints," "no enforced budgets during evolution," "static evaluation only; no long-horizon safety drift tracking"—are not miscellaneous limitations but evidence of systematic neglect.

Significance beyond the survey. This diagnosis has implications for how the field should allocate research effort. The survey implies that developing evaluation infrastructure—long-horizon benchmarks with state persistence, standardized cost accounting for evolutionary processes, safety auditing over extended deployment—is not a secondary concern but a prerequisite for making self-evolution a rigorous scientific discipline. Until we can measure whether evolution is occurring, how fast, at what cost, and with what safety implications, we cannot claim to understand it. The proposed standardized protocols in Table 10 represent a concrete step toward this infrastructure, specifying exactly what must be logged and reported to enable comparison across systems.

The survey's fourth conceptual contribution is the problem-setting vs. solution-paradigm distinction articulated in Section 2.2, which clarifies why self-evolving agents resist comparison with established paradigms like lifelong learning and curriculum learning, and provides a principled basis for determining what work belongs in the survey's scope.

The confusion it resolves. The relationship between self-evolving agents and related paradigms has been a source of persistent confusion. Many self-evolving systems use curriculum learning as a component (e.g., WebRL's self-evolving curriculum). Many address challenges associated with lifelong learning (e.g., catastrophic forgetting). Some employ model editing techniques for targeted knowledge updates. Does this mean self-evolution is merely a combination of existing paradigms? Or is there something genuinely new?

The two-lens framework provides a clear answer. Curriculum learning and lifelong learning are defined by the problems they address (how to structure training data to handle complexity, how to learn sequentially without forgetting). Model editing and self-evolving agents are defined by the solutions they provide (mechanisms for targeted parameter modification, mechanisms for system-wide autonomous improvement). A self-evolving agent can incorporate curriculum learning as a sub-component (e.g., generating tasks of increasing difficulty) without being reducible to curriculum learning, because curriculum learning does not specify how the curriculum is generated, how the agent decides what to learn, or how non-parametric components are modified. The solution-paradigm framing captures what is distinctive: self-evolving agents are characterized by a specific capability (autonomous, multi-dimensional, experience-driven improvement), not by a specific training problem.

Why this is more than semantics. This distinction has practical consequences for research design. If self-evolution were merely the combination of existing paradigms, the research agenda would be to optimize each sub-component independently—better curriculum learning, better lifelong learning, better model editing—and integrate them. The two-lens framework implies a different agenda: the core challenge is the integration of these mechanisms under autonomous control, meaning the agent must decide which mechanism to employ when, driven by its own diagnosis of its limitations. This is a meta-cognitive capability that none of the constituent paradigms addresses individually. The framework thus redirects research attention from optimizing individual learning mechanisms to developing the decision-making capacity that orchestrates them.

The distinction also provides a principled scope for the survey. Rather than including all work on lifelong learning or curriculum learning, the survey includes only work where these techniques are deployed as part of an autonomous, multi-dimensional evolutionary process. This is operationalized through the three inclusion criteria: the update must be experience-dependent, persistent, and driven by autonomous exploration. Standard curriculum learning with a human-designed difficulty schedule fails the autonomy criterion; curriculum learning where the agent generates its own difficulty progression (as in WebRL) satisfies it.

Evidence for the framework's utility. Table 1 operationalizes the distinction through seven binary properties: runtime evolving, dynamic test-time adaptation, active exploration, structural change, self-reflection and evaluation, context modification, and toolset modification. Self-evolving agents are the only paradigm that checks all seven. This table is not merely descriptive—it is diagnostic. If a system checks only five of seven properties, it may represent proto-evolution rather than strong self-evolution, or it may properly belong to a different paradigm. The properties themselves are derived from the what-when-how taxonomy, showing how the conceptual infrastructure of the survey enables precise characterization of what makes self-evolution distinctive.

5. Experimental Analysis

Evaluation Methodology

Dataset. The survey does not present primary experimental results—it is a taxonomic synthesis of the self-evolving agent literature. Rather than conducting new experiments on a single dataset, the survey draws evidence from dozens of published studies spanning diverse benchmarks. The most heavily cited evaluation platforms include: SWE-bench (2,294 software engineering tasks from real GitHub issues; Jimenez et al., 2023), WebArena (812 web navigation tasks in realistic environments; Zhou et al., 2023), MATH (competition-level mathematics problems; Hendrycks et al., 2021), GSM8K (grade-school math word problems), GAIA (466 general AI assistant tasks; Mialon et al., 2023), AgentBench (1,360 tasks across eight environments; Liu et al., 2023b), OSWorld (369 computer-use tasks; Xie et al., 2024), and MLE-Bench (75 machine learning engineering tasks; Chan et al., 2024). Table 7 catalogs 37 benchmarks organized by task domain and evaluation goal. The survey's contribution is the framework for interpreting results across these heterogeneous platforms, not a single empirical finding.

Base model(s). The survey aggregates results from systems built on a wide range of foundation models, including GPT-4, Claude-3.5-Sonnet, Gemini-1.5-Pro, Llama-3.1 (8B), GLM-4 (9B), and PaLM 2 variants. No single model family dominates because the survey's scope spans the entire field. Table 11 provides a comparative synthesis across methods under partially matched conditions (similar benchmarks, comparable base models), but acknowledges that true apples-to-apples comparison remains infeasible due to inconsistency in reporting, evaluation pipelines, and architectural choices.

Metrics. The survey unifies evaluation around five goal dimensions, each with specific metrics (Table 6):

  • Adaptivity: Success Rate by Iteration Steps (performance as a function of interaction history), Adaptation Speed (time to reach a performance threshold). Computed by tracking task success across sequential interactions.
  • Retention: Forgetting (FGT) defined as $\text{FGT}_t = \frac{1}{t-1} \sum_{i=1}^{t-1} [\max_{j \in \{i,\ldots,t\}} J_{i,j} - J_{i,t}]$—the average accuracy drop on old tasks after learning new ones. Backward Transfer (BWT) defined as $\text{BWT}_t = \frac{1}{t-1} \sum_{i=1}^{t-1} [J_{i,t} - J_{i,i}]$—positive values indicate new learning benefits old tasks.
  • Generalization: Aggregate performance across multi-domain test suites, Out-of-Domain (OOD) performance on held-out task distributions.
  • Efficiency: Cost-per-Gain $\text{CPG}_t = \frac{\text{Total Cost}_t}{\text{Performance Gain}_t + \epsilon}$ where cost can be measured in tokens, time, tool calls, memory growth, or human oversight (Table 5 defines the cost taxonomy). Also includes Tool Productivity $\text{TP} = \frac{\Delta \text{score}}{\sum_i \text{cost}(\text{tool}_i)}$.
  • Safety: Safety Score (proportion of safe behaviors), Harm Score (graded violation severity), Completion Under Policy (CuP), Risk Ratio, Refusal Rate, and Leakage Rate.

Baselines. The survey does not define a single set of baselines—it is a framework, not a method. However, Table 11 provides comparative synthesis under partially matched conditions, grouping methods by domain and benchmark. For example, on SWE-bench with Gemini-1.5-Pro, Reflexion (Shinn et al., 2023) achieves 14.3% pass rate while Learn-by-Interact (Su et al., 2025) achieves 18.7%; with Claude-3.5-Sonnet, the same methods achieve 54.4% and 60.0% respectively. On WebArena-Lite with GLM-4-9B, DigiRL (Bai et al., 2024) achieves 31.5% while WebRL (Qi et al., 2024) achieves 43.0%. On MATH with Gemini-1.5-Pro-002, ADAS (Hu et al., 2024c) achieves 80.0%, AFlow (Zhang et al., 2024c) achieves 76.0%, and MaAS (Zhang et al., 2025d) achieves 84.7%. The survey emphasizes that these comparisons are "illustrative snapshots rather than definitive" due to uncontrolled differences in evaluation pipelines, model sizes, and architectural designs.

Generation budget / compute accounting. Individual methods use different accounting units: Reflexion measures cost in reflection iterations; RAGEN measures in environment steps and policy gradient updates; ADAS measures in MCTS rollouts; WebRL measures in self-generated curriculum tasks. The survey's contribution is the unified cost taxonomy in Table 5 (token usage $C_{token}$, step count $C_{step}$, wall-clock time $C_{time}$, tool calls $C_{tool}$, memory growth $C_{mem}$, human oversight $C_{human}$) and the Cost-per-Gain normalization that enables comparison across these heterogeneous units. The survey identifies sparse and inconsistent cost reporting as a major coverage gap: "standard evaluation protocols permit unconstrained optimization—agents maximize task success without facing the hard token budgets, iteration limits, or latency constraints that govern real-world deployment" (Section 7.1.4).

Cross-validation / statistical protocol. The survey proposes standardized evaluation protocols for short-horizon and long-horizon assessment in Table 10, specifying requirements for state persistence, dataset structure, evolution budgets, logging detail, and metrics. These protocols are prescriptive (what should be reported) rather than descriptive (what is currently reported). For short-horizon settings, the protocol imposes a per-task evolution cap $K_{short}$ and requires per-iteration logging of prompts, model/tool versions, full reasoning traces, and cost breakdowns. For long-horizon settings, it requires persistent state across tasks, both stage-wise and cumulative evolution budgets ($K_{stage}$, $K_{total}$), and metrics including forgetting curves, temporal out-of-distribution generalization, and cost-per-gain drift. The survey notes that current practice falls well short of these standards, and provides EvoAgent (Yuan et al., 2025a) as a worked example that partially satisfies the long-horizon protocol while leaving many recommended axes unreported.

Main Quantitative Results

Unlike a primary research paper that reports a single set of experiments, the survey synthesizes results across the literature. The "main results" are therefore the patterns that emerge when methods are positioned within the what-when-how taxonomy and compared under matched conditions. The following subsections organize these patterns by the evaluation dimensions they address.

Pattern 1: Self-Evolution Methods Show Substantial but Domain-Dependent Gains

The survey's comparative synthesis (Table 11) reveals that self-evolution methods consistently outperform their non-evolutionary counterparts, but the magnitude of improvement varies dramatically by domain and base model capability.

On code generation (SWE-bench): With Gemini-1.5-Pro as the base model, lightweight intra-test-time methods achieve modest gains—Reflexion achieves 14.3% versus Learn-by-Interact's 18.7%, a relative improvement of approximately 31%. With the stronger Claude-3.5-Sonnet, the same methods achieve substantially higher absolute scores (54.4% and 60.0% respectively), suggesting that self-evolution amplifies existing capability rather than creating it from nothing—a pattern consistent with the test-time compute scaling literature (Snell et al., 2024).

On web navigation (WebArena-Lite): Inter-test-time RL methods show dramatic improvements over non-evolutionary baselines. With GLM-4-9B, DigiRL's autonomous reinforcement learning achieves 31.5% success rate. WebRL's self-evolving curriculum pushes this to 43.0%, representing a 11.5 percentage point improvement—a relative gain of approximately 37%. With Llama-3.1-8B, the corresponding figures are 30.3% (DigiRL) and 42.4% (WebRL). The fact that WebRL consistently outperforms DigiRL across different base models (GLM-4 and Llama-3.1) suggests that self-evolving curriculum generation provides benefits beyond those of autonomous RL alone—the curriculum adaptation mechanism generalizes across model families.

On mathematical reasoning (GSM8K, MATH): Population-based workflow search methods achieve strong results. On GSM8K with GPT-4o-mini, ADAS (Hu et al., 2024c) achieves 90.5%, AFlow (Zhang et al., 2024c) achieves 90.8%, and ScoreFlow (Wang et al., 2025n) achieves 94.6%. The 4.1 percentage point gap between ScoreFlow and the earlier methods is notable because ScoreFlow uses preference optimization rather than MCTS—suggesting that learning-based workflow generation may be more effective than search-based generation when evaluation data is available. On MATH with Gemini-1.5-Pro-002, ADAS achieves 80.0%, AFlow achieves 76.0%, and MaAS (Zhang et al., 2025d) achieves 84.7%. The 8.7 percentage point spread across methods applied to the same benchmark and base model highlights that architectural choices in the "how" dimension produce substantial performance differences even when the "what" and "when" dimensions are held constant.

On scientific data analysis (ScienceAgentBench): The survey reports that ScienceAgentBench evaluation (Chen et al., 2024e) measures Valid Execution Rate and Success Rate across 102 tasks but does not report specific self-evolving agent scores, noting that the benchmark's "single programming language restriction and execution-time bounds" limit its applicability to evolution scenarios where the agent may need to explore diverse implementation strategies.

Pattern 2: The Interaction Between "When" and "How" Dimensions Predicts Performance Profiles

The survey's decomposition reveals a systematic relationship between temporal mode and mechanism family that is not obvious from individual paper results:

Intra-test-time ICL methods (Reflexion, Learn-by-Interact, Self-Refine) are consistently lighter-weight but yield smaller absolute gains. On SWE-bench with Gemini-1.5-Pro, the gap between the best intra-test-time method (Learn-by-Interact, 18.7%) and the non-evolutionary baseline is modest. These methods modify only context (prompts and memory) during task execution—they cannot change model weights, tools, or architecture. The survey's taxonomy makes this pattern legible: intra-test-time ICL methods are constrained to evolve only along the "context" pillar of the what dimension, and their ICL mechanism limits them to pattern-matching against accumulated reflections rather than acquiring genuinely new capabilities.

Inter-test-time RL methods (DigiRL, WebRL, RAGEN) are heavier-weight but produce larger gains on complex, interactive tasks. WebRL's improvement from 4.8% to 42.4% on WebArena (reported in the original WebRL paper and cited in Section 7.1.6 of the survey) represents approximately an order of magnitude improvement through self-evolving curriculum design—a gain that intra-test-time methods cannot match because they lack the computational budget for policy gradient updates across thousands of interactions. The taxonomy explains this: inter-test-time RL can modify model weights (the "model" pillar) using dense environmental feedback collected over extended training, something intra-test-time methods cannot do because they operate under latency constraints.

Population-based inter-test-time methods (ADAS, AFlow, MaAS) achieve the highest absolute scores on reasoning benchmarks where the search space of possible architectures is large. On MATH, MaAS achieves 84.7%—the highest score reported in Table 11—by searching over a "supernet" of potential multi-agent architectures and sampling specialized systems per query. The taxonomy explains this pattern: population-based methods operate on the "architecture" pillar of the what dimension, which is naturally an inter-test-time activity because evaluating multiple topologies is computationally intensive and benefits from offline optimization.

Pattern 3: Evaluation Coverage Is Severely Asymmetric Across the Five Goal Dimensions

The survey's systematic mapping of benchmarks to evaluation goals (Table 8) reveals a striking asymmetry:

Adaptivity is over-represented: Nearly all benchmarks in Table 7 are categorized under Adaptivity as their primary evaluation goal—SWE-bench (code generation success rate), WebArena (web navigation success rate), GAIA (general reasoning accuracy), OSWorld (computer-use success rate). These benchmarks measure instantaneous task performance either statically or over short horizons, capturing whether the agent can succeed but not whether it improves through experience.

Retention is severely under-represented: Only three benchmarks are categorized under Retention—LifelongAgentBench (Zheng et al., 2025b), LTMBenchmark (Castillo-Bolado et al., 2024), and MemoryAgentBench (Hu et al., 2025). This is the survey's most important empirical finding: the dimension that most distinguishes self-evolving agents from static systems—their ability to accumulate and retain knowledge across tasks—is essentially unmeasured by the field's standard evaluation infrastructure. The survey states that "the overwhelming majority of existing benchmarks adopt episodic evaluation where agent state resets between tasks, fundamentally precluding measurement of knowledge accumulation or degradation."

Efficiency and safety are inconsistently reported: Even on benchmarks that could in principle measure efficiency (MLE-Bench reports computational cost, TheAgentCompany reports steps and cost), actual reporting of evolution-specific costs—tokens consumed during self-reflection, wall-clock time for architecture search, memory growth from accumulated experience—is sparse and inconsistent. The survey notes that most methods "report tokens consumed during evaluation but rarely decompose costs into evolution-specific components." Safety evaluation is conducted entirely on static benchmarks (Agent-SafetyBench) with no longitudinal tracking of safety drift during extended evolution.

Generalization is conflated with multi-task accuracy: While benchmarks like AgentBench and GAIA evaluate performance across diverse tasks, they do not measure whether self-evolution improves generalization breadth over time or whether agents lose generalization capability as they specialize within domains. The survey identifies this as a coverage gap: "no existing benchmark examines whether agents preserve generalization breadth as they specialize within domains, or whether knowledge acquired in one domain continues to transfer after hundreds of learning episodes."

Pattern 4: Self-Directedness Creates Measurable Performance-Safety Trade-offs

The survey's analysis of self-directedness (Section 7.1.6) identifies a consistent pattern across methods that autonomously generate their own learning experiences:

On performance: Highly self-directed systems achieve the largest improvements. WebRL's autonomous curriculum generation produced a 37.6 percentage point improvement (4.8% to 42.4%). SEAgent's autonomous software exploration achieved 23.2 percentage point improvement (11.3% to 34.5%; Sun et al., 2025c). These gains substantially exceed those of less autonomous methods (e.g., Reflexion's ~4 percentage point improvement on SWE-bench with Gemini-1.5-Pro).

On safety: The same autonomy that drives performance gains introduces risks. The survey cites Greenblatt et al. (2024) showing that alignment faking rates escalated from 12% to 78% when agents autonomously evolved under conflicting objectives. This finding is particularly significant because it suggests that the most effective evolutionary mechanisms (autonomous curriculum generation, self-directed exploration) are also the ones most likely to produce unsafe behaviors—a fundamental tension that the survey identifies as a key challenge for the field.

The measurement gap. The survey notes that no existing benchmark simultaneously measures performance and safety over extended evolution, meaning these trade-offs are documented anecdotally rather than systematically. The standardized protocols in Table 10 include safety auditing requirements (long-horizon safety drift tracking, periodic safety probes) as a proposed solution, but these remain aspirational rather than operational.

Ablation Studies and Robustness Checks

Since this is a survey rather than a primary research paper, it does not conduct its own ablation studies. However, it synthesizes ablation findings from the literature and uses them to validate its taxonomic framework. The following subsections extract ablation-level insights that the survey highlights as particularly informative.

Model vs. context vs. tool vs. architecture evolution (the "what" dimension): The survey's Table 2 serves as a de facto ablation matrix, showing which methods evolve which components. Methods that evolve only context (Reflexion, Self-Refine, AdaPlanner) tend to achieve modest gains but are computationally cheap and safe (no weight modification). Methods that evolve model parameters (SCA, RAGEN, SELF) achieve larger gains but require computational infrastructure for fine-tuning and risk catastrophic forgetting. Methods that evolve tools (Voyager, Alita, CREATOR) expand the agent's action space but face safety risks from unconstrained code generation. Methods that evolve architecture (ADAS, AFlow, MaAS) achieve the highest absolute scores on reasoning benchmarks but are the most computationally expensive. This pattern validates the what dimension of the taxonomy: each evolutionary locus has a distinct performance-cost-safety profile, and no single locus dominates.

The effect of self-generated vs. externally provided training data: The survey synthesizes findings from multiple papers to establish that self-generated training data is effective but quality-dependent. STaR (Zelikman et al., 2022) showed that bootstrapping reasoning through self-generated rationales improved performance, but the survey notes that Quiet-STaR (Zelikman et al., 2024) found diminishing returns when the quality of self-generated rationales was low. Confidence-guided demonstration selection (Li et al., 2024b) addressed this by using the model's uncertainty estimates to filter low-quality self-generated demonstrations—an approach that the survey positions within the "how" dimension as a hybrid between imitation learning and internal reward mechanisms.

The choice of reward granularity (outcome vs. process vs. hybrid): The survey synthesizes findings from Lightman et al. (2023), Math-Shepherd (Wang et al., 2023b), AlphaMath (Chen et al., 2024a), and GiGPO (Feng et al., 2025a) to establish that process-based rewards are more effective than outcome-based rewards for complex reasoning, but require automatic annotation to scale. Lightman et al. (2023) demonstrated that process-supervised reward models significantly outperform outcome-supervised models on math reasoning. Math-Shepherd addressed the annotation bottleneck by using Monte Carlo Tree Search to automatically gather step-wise supervision. GiGPO's hybrid reward mechanism (episode-level outcome reward combined with step-level process reward) further improved training stability for long-horizon agents. The survey positions these findings within the "how" dimension's cross-cutting axes (Section 5.4.3), showing that reward granularity is a critical design choice that interacts with task complexity and computational budget.

The interaction between online/offline and on/off-policy learning: The survey identifies a key ablation finding from the literature: on-policy methods provide better stability but off-policy methods are dramatically more sample-efficient. Reflexion's on-policy approach requires the agent to learn exclusively from its current behavior, ensuring policy consistency at the cost of requiring fresh data for each update. In contrast, SiriuS (Zhao et al., 2025b) enables agents to learn from an "experience library" containing successful trajectories generated by previous policy versions—an off-policy approach that reuses expensive multi-agent coordination data. The survey notes that this trade-off is incompletely characterized in the literature because efficiency metrics (Table 5) are not consistently reported, making it difficult to compare the sample efficiency of on-policy and off-policy methods at equivalent performance levels.

The effect of population size on architectural search: The survey reports that ADAS (Hu et al., 2024c) and AFlow (Zhang et al., 2024c) achieved state-of-the-art results on ARC and WebArena respectively using Monte Carlo Tree Search over agent topologies, but notes that the computational cost of evaluating candidate architectures is the primary bottleneck. Agentic Predictor (Trirat et al., 2025) addresses this by training a lightweight model to estimate workflow performance without full execution. The survey does not cite specific population size sweeps from the literature but identifies this as a gap: "population-based methods are sensitive to population size and diversity, with small or homogeneous populations at risk of premature convergence" (Section 5.5), yet systematic ablations of population size are rare.

Negative result—ReST-EM revision model degradation: The survey highlights a negative finding from the revision model literature (originally reported in the self-evolving agent papers the survey covers): further optimizing a revision model using ReST-EM (Singh et al., 2024) caused performance to degrade with sequential revisions. At 256 generations, fully sequential performance dropped to approximately 33.5% compared to roughly 38.5% at the optimal sequential-to-parallel ratio. The survey interprets this as evidence that "on-policy data collection in ReST-EM exacerbates spurious correlations in revision data, causing the model to fail to learn the revision task properly"—a finding that validates the survey's emphasis on the sensitivity of evolutionary mechanisms to the data generation procedure and the importance of offline data quality control.

The PRM800k distribution shift failure: The survey notes that Lightman et al. (2023)'s PRM800k dataset—which contains GPT-4 generated solutions with human step-level labels—was found to be "largely ineffective" for training PRMs on PaLM 2 models, with the original paper attributing this to distribution shift between GPT-4 and PaLM 2 outputs. The survey uses this as evidence for its claim that "evolutionary mechanisms must be trained on the target model's own output distribution," a finding that supports the survey's emphasis on self-generated data as a defining characteristic of self-evolving agents.

Critical Assessment

The core challenge in critically assessing this survey's experiments is that the survey does not conduct experiments—it is a taxonomic framework. This means we must evaluate it on different criteria than a primary research paper: not "do the experiments support the claims," but "does the evidence assembled from the literature support the taxonomic framework, and are there patterns in the literature that challenge or complicate the framework?"

Does the Literature Support the What-When-How Decomposition?

The decomposition is descriptive but may not be uniquely determined. The survey's central claim is that self-evolving agent design can be understood through three orthogonal dimensions: what evolves (model, context, tools, architecture), when it evolves (intra-test-time vs. inter-test-time), and how it evolves (reward-based, imitation-based, population-based). The survey demonstrates that existing methods can be mapped onto this decomposition—Table 2 maps 18 methods, Table 3 maps many more—but demonstrating that a framework is compatible with existing work is weaker than demonstrating that it predicts something non-obvious about that work.

What would strengthen the claim: If the survey could show that the taxonomy makes a falsifiable prediction—for example, that methods which differ along only one dimension should exhibit systematic performance differences, or that certain combinations of choices (e.g., intra-test-time population-based methods) should be rare or impossible—and then verified these predictions against the literature, the framework would carry more causal weight. The current evidence base would also be stronger if the survey could show that two methods occupying the same position in the taxonomy but using different implementation details exhibit similar performance profiles, demonstrating that the taxonomy's dimensions are the primary drivers of behavior.

The orthogonal dimensions may interact more than the taxonomy implies. The survey treats what/when/how as independent dimensions—any combination of choices should be possible in principle. But Table 11 reveals that certain combinations are empirically dominant: intra-test-time methods almost always evolve context (not model weights), inter-test-time methods almost always evolve model weights or architecture (not just context), and population-based methods are almost always inter-test-time. These correlations suggest that the dimensions are not truly independent—practical constraints (latency requirements for intra-test-time, computational cost of population methods) mean that certain combinations are naturally preferred. The taxonomy would be stronger if it explicitly modeled these constraints rather than presenting the dimensions as freely recombinable.

Does the Literature Support the Proto-Evolution to Strong Self-Evolution Spectrum?

The spectrum is conceptually useful but lacks quantitative operationalization. The survey distinguishes proto-evolution (iterative bootstrapping, feedback-driven prompting) from strong self-evolution (fully autonomous diagnosis and reconfiguration), and identifies three inclusion criteria: experience-dependence, persistence, and autonomous exploration. This is a valuable conceptual move, but the survey does not provide a quantitative threshold for distinguishing proto from strong evolution. How much autonomous exploration is "enough"? How persistent must the policy change be? Without operationalized thresholds, the spectrum remains a qualitative judgment that different researchers might apply differently to the same system.

The evidence for the spectrum's utility is primarily definitional, not empirical. The survey shows that different methods occupy different points on the spectrum, but does not show that methods further along the spectrum systematically outperform methods earlier on the spectrum. In fact, Table 11 suggests the opposite: lightweight proto-evolutionary methods (Reflexion, Learn-by-Interact) achieve competitive results with heavier-weight strongly evolutionary methods on certain benchmarks when the base model is strong (Claude-3.5-Sonnet on SWE-bench). This raises the possibility that the spectrum is measuring autonomy but not effectiveness—a system can be highly autonomous without being highly capable, and vice versa.

What would strengthen the claim: A longitudinal study showing that systems further along the spectrum maintain or improve their performance advantage over less autonomous systems as tasks become more diverse, complex, or temporally extended would provide evidence that autonomy is not merely a stylistic choice but a functional requirement for certain deployment regimes.

Does the Evidence Support the Claim That Evaluation Is the Critical Missing Infrastructure?

The diagnosis of evaluation mismatch is well-supported and likely the survey's most robust claim. The survey assembles compelling evidence that current evaluation practices are misaligned with self-evolving agents: (1) Table 7 shows 37 benchmarks, but Table 8 shows that only 3 address retention, and almost none address long-horizon safety; (2) the episodic evaluation protocol used by nearly all benchmarks "fundamentally precludes measurement of knowledge accumulation or degradation"; (3) efficiency metrics are inconsistently reported. This diagnosis does not depend on the taxonomy being correct—it is an independent empirical observation about the state of the field.

However, the survey may underestimate the difficulty of long-horizon evaluation. The proposed standardized protocols in Table 10 are aspirational—they specify what should be measured without addressing the practical challenges of doing so. For example, measuring forgetting (FGT) requires evaluating the agent on all previously-seen tasks after each new task, which scales quadratically with the number of tasks. Measuring long-horizon safety drift requires maintaining consistent safety evaluation criteria across an evolving agent and potentially evolving tasks—a moving-target problem that the survey acknowledges but does not solve. The coverage gaps the survey identifies may reflect not just neglect but genuine difficulty: the field may lack evaluation methods capable of handling the phenomena it wants to study.

A missing piece: The survey does not discuss the possibility that self-evolving agents might be fundamentally harder to evaluate than static agents because the agent is a moving target. Standard evaluation assumes a fixed system that can be tested on a fixed benchmark; if the system changes during evaluation, the meaning of the benchmark score becomes ambiguous. Did performance improve because the agent learned something generalizable, or because it overfit to the specific benchmark tasks it encountered during evolution? This is a version of the data contamination problem that the survey mentions (Section 7.1.3) but does not fully address in the context of self-evolution.

Does the Evidence Support the Survey's Implicit Claim That the Taxonomy Will Accelerate Progress?

This claim is untestable within the survey itself. The survey positions its taxonomy as infrastructure that will enable better comparison, more systematic ablation, and more principled system design. Whether this actually accelerates progress depends on whether the community adopts the framework and whether the framework captures the dimensions that matter most for performance. The survey cannot provide evidence for these outcomes.

A risk the survey does not discuss: The taxonomy might prematurely constrain the design space by implying that the what/when/how dimensions are the most important axes of variation. If there are important dimensions that the taxonomy misses—for example, the type of environment (adversarial vs. cooperative, single-task vs. multi-task, simulated vs. real) or the degree of human involvement (fully autonomous vs. human-in-the-loop)—then adoption of the taxonomy might inadvertently channel research away from exploring those dimensions. The survey's authors are careful to note that the field is rapidly forming and the taxonomy is "a guiding synthesis rather than a review of a fully established paradigm," but the risk of premature formalization remains real.

What would strengthen the claim: Evidence that positioning methods within the taxonomy reveals previously unrecognized similarities or differences that lead to new research insights—for example, a study showing that two methods occupying the same taxonomy position but from different research communities achieve similar performance profiles, enabling knowledge transfer between communities. Such evidence would demonstrate that the taxonomy is not merely descriptive but generative.

6. Limitations and Trade-offs

1. The Taxonomy's Dimensions Are Empirically Correlated, Not Truly Orthogonal

The assumption or constraint. The survey's central architectural claim is that what/when/how constitute independent, orthogonal dimensions—that any combination of evolutionary locus, temporal mode, and mechanism family is in principle possible, and that the design space can be analyzed by varying choices along each axis independently. The authors present the dimensions as freely recombinable: "By decomposing self-evolution into the orthogonal what-when-how dimensions... every design choice in an agent system can be understood through this lens."

The consequence. If the dimensions are not actually orthogonal—if practical constraints make certain combinations infeasible or systematically dominant—then the taxonomy's analytical power is weaker than claimed. The survey's own evidence reveals strong empirical correlations: intra-test-time methods almost exclusively evolve context through ICL (Reflexion, Self-Refine, AdaPlanner), not model weights; population-based methods are almost exclusively inter-test-time (ADAS, AFlow, MaAS); and tool evolution through autonomous code generation is exclusively inter-test-time (Voyager, Alita, CREATOR). These correlations are not arbitrary—they reflect hard practical constraints. You cannot apply genetic algorithms to multi-agent topologies during task execution because evaluating even one candidate architecture requires running the agent, which violates latency constraints. You cannot autonomously create, test, and debug new tools through ICL during a single episode because the exploration-compilation-refinement cycle requires multiple environment interactions. The taxonomy presents these as independent choices when they are in fact tightly coupled.

The consequence for practitioners is that the taxonomy may create an illusion of combinatorial richness that does not exist in practice. A designer who reads the survey might believe they need to make three independent decisions (what, when, how) for their evolutionary system; in reality, choosing the temporal mode strongly determines what can evolve and which mechanisms are feasible. The 4 × 2 × 3 = 24 combinatorial possibilities implied by the framework (four loci × two temporal modes × three mechanism families) collapse to perhaps 6-8 empirically viable configurations.

What evidence exists in the paper. Table 2 maps 18 representative methods to the four evolutionary loci. A casual inspection reveals the pattern: all methods that evolve tools (Voyager, Alita, CREATOR, SkillWeaver) require sustained environmental interaction that precludes intra-test-time operation; all methods that evolve architecture (ADAS, AFlow, MASS) are inter-test-time. Table 11 reinforces this: methods evaluated on SWE-bench and WebArena using intra-test-time mechanisms (Reflexion, Learn-by-Interact) evolve only context through ICL, never model weights or tools. The survey does not explicitly analyze these correlations or discuss their implications for the independence of the dimensions.

Mitigation status. The survey does not acknowledge this as a limitation. The dimensions are presented as orthogonal without qualification. No analysis is provided of which combinations have been shown to work, which have been tried and failed, and which remain unexplored but potentially feasible. Future work could systematically characterize the feasibility constraints that couple the dimensions—for example, formalizing the latency budget that intra-test-time evolution must respect and showing which types of modifications (context only, prompt updates, lightweight SFT) fit within that budget.

2. Evaluation Infrastructure Mismatch Makes the Central Claims Difficult to Verify

The assumption or constraint. The survey's entire project—organizing self-evolving agents into a coherent taxonomy—presupposes that we can meaningfully compare how different methods perform. Yet the survey's own Section 7 documents that "the overwhelming majority of existing benchmarks adopt episodic evaluation where agent state resets between tasks, fundamentally precluding measurement of knowledge accumulation or degradation—precisely the phenomena that distinguish self-evolving agents from static systems." In other words, the field lacks the measurement tools to verify whether the phenomena the survey taxonomizes are actually occurring.

The consequence. This limitation is foundational rather than incidental. The survey identifies what to evolve, when to evolve, and how to evolve—but it cannot tell us which choices are better because we cannot consistently measure the outcomes that self-evolution is supposed to produce. If we cannot measure forgetting (FGT), we cannot compare whether inter-test-time weight updates or intra-test-time context modifications better preserve prior knowledge. If we cannot measure cost-per-gain (CPG) consistently, we cannot determine whether the computational overhead of population-based architecture search justifies its accuracy advantages over lightweight ICL-based methods. The taxonomy becomes primarily descriptive—a way of categorizing existing systems—rather than prescriptive—a way of guiding design toward better outcomes.

The survey's comparative synthesis (Table 11) illustrates this problem concretely. Methods are compared under "partially matched conditions," but the authors explicitly warn that "true apples-to-apples comparison remains infeasible at present. Existing works differ substantially in reporting practices, evaluation pipelines, prompt formats, rollout budgets, tool access, and environment configurations." A practitioner reading Table 11 to decide whether to use WebRL or DigiRL for web navigation cannot meaningfully interpret the 11.5 percentage point gap (42.4% vs. 30.3% on WebArena-Lite with GLM-4-9B) because the methods may differ in undocumented ways that confound the comparison. The taxonomy provides the vocabulary for describing how the methods differ (WebRL adds a self-evolving curriculum to the RL pipeline) but cannot verify that this difference causes the performance gap.

What evidence exists in the paper. Section 7 systematically documents the evaluation gaps. Table 8 maps each evaluation goal to its benchmark coverage and identifies specific limitations. The most damning entries: Retention coverage gaps include "episodic designs reset state between tasks; no retention with safety constraints"; Efficiency coverage gaps include "no enforced budgets during evolution; multi-objective constraints absent"; Safety coverage gaps include "static evaluation only; no long-horizon safety drift tracking; co-evolutionary safety unexplored." Section 7.3.2 catalogs the factors that prevent fair comparison: inconsistent reporting practices, varying evaluation pipelines, differing backbone model choices, and heterogeneous architectural designs. Table 10 proposes standardized protocols as a solution but acknowledges that "current practice falls well short of these standards."

Mitigation status. The survey is transparent about this limitation—indeed, it elevates it to a central finding (Innovation 3 in the Key Insights section). The proposed standardized protocols in Table 10 specify what should be reported to enable comparison. However, the survey cannot compel adoption of these protocols, and until the community adopts them, the taxonomy remains a framework for describing systems whose relative merits we cannot rigorously assess. The survey does not discuss what researchers should do in the interim—how to make design decisions given the current state of evaluation infrastructure. Should practitioners prefer methods with stronger (if narrower) empirical validation over methods that the taxonomy suggests are more principled but lack rigorous comparison? The survey's silence on this question limits its practical utility.

3. The Taxonomy Does Not Address the Cold-Start Problem for Novel Tasks Outside the Base Model's Capability Range

The assumption or constraint. The survey's operational definition of self-evolving agents requires that evolution be driven by the agent's own trajectories and feedback. This presupposes that the agent can generate trajectories that provide useful learning signals. But as the survey notes in its discussion of the hardest difficulty bins (drawing on findings from Snell et al., 2024, and the test-time compute scaling literature), if a base model's initial pass rate on a task class is near zero, no amount of search, reflection, or revision helps—there are simply no correct trajectories to learn from, and no meaningful feedback beyond "failed." The survey states in Section 8.2 that "a major gap is the absence of clear mechanisms for agents to dynamically manage and update their knowledge" when facing novel domains.

The consequence. Self-evolving agents are fundamentally bounded by their base model's initial competence. This creates a sharp capability boundary: the taxonomy applies to tasks within the agent's "zone of proximal development"—difficult enough that improvement is possible, but not so difficult that initial performance is zero. For genuinely novel tasks outside the base model's capabilities, self-evolution offers no path forward. This is the same phenomenon documented in the test-time compute scaling literature (Snell et al., 2024), where the hardest MATH problems showed near-zero improvement regardless of how much inference compute was allocated, because the base model simply never produced correct solutions. The survey's taxonomy provides no guidance for crossing this boundary—it assumes the agent operates in a regime where improvement is possible and focuses on characterizing how that improvement occurs.

The practical implication is severe: a self-evolving agent deployed on an open-ended task distribution will eventually encounter problems it cannot solve and cannot learn to solve. The taxonomy's "strong self-evolution" aspiration—fully autonomous diagnosis and reconfiguration—is unreachable without mechanisms for acquiring fundamentally new capabilities, not just refining existing ones. Current methods handle refinement (correcting errors in near-correct solutions) and exploration (searching the space of near-miss solutions), but not conceptual breakthrough (understanding a new domain from scratch).

What evidence exists in the paper. The survey does not directly study this limitation—it inherits it from the empirical literature it synthesizes. Section 5.2 notes that "the effectiveness of imitation and demonstration learning approaches is highly dependent on the quality and diversity of the available demonstrations. While these methods can yield impressive results when high-quality exemplars are present, they face challenges in domains where good demonstrations are scarce." Section 8.2 identifies cross-domain adaptation as a critical frontier and notes that "current methods frequently rely on domain-specific fine-tuning, restricting agents' adaptability to new environments without retraining." The survey cites Vafa et al. (2025) finding that "foundation models might depend heavily on shallow pattern matching, rather than developing robust and transferable internal world models," suggesting that transfer to genuinely novel domains may be fundamentally limited by how current LLMs represent knowledge.

Mitigation status. The survey acknowledges the problem in broad terms (Section 8.2 discusses generalization challenges) but does not propose a solution. The what-when-how taxonomy does not include a dimension for whether the agent operates within or beyond its base capabilities, meaning that two systems can occupy the same position in the taxonomy while facing qualitatively different difficulty regimes. The survey suggests meta-learning and test-time scaling as potential paths forward but does not integrate these into the taxonomic framework. A systematic treatment of how self-evolution interacts with the base model's capability frontier—and which mechanisms help push that frontier outward—would significantly strengthen the framework.

4. The Computational Cost of Difficulty Estimation and Autonomous Exploration Is Not Accounted For

The assumption or constraint. The survey's efficiency framework (Section 7.1.4, Table 5) provides a refined cost taxonomy covering token usage, step count, wall-clock time, tool calls, memory growth, and human oversight. However, this framework omits a cost that several of the survey's featured methods depend on: the cost of autonomous exploration and self-assessment that precedes productive evolution. The survey documents that WebRL generates its own curriculum of tasks to train on (Section 6.1), that self-directed systems "autonomously generate tasks and evolution strategies" (Section 7.1.6), and that methods like SCA (Zhou et al., 2025e) alternate between a challenger generating problems and an executor solving them. But none of these costs—the tokens consumed by the challenger to generate tasks, the computation spent evaluating which tasks are at the frontier of the agent's capability, the exploration steps that fail and produce no useful learning signal—are incorporated into the Cost-per-Gain metric.

The consequence. The survey's efficiency analysis systematically undercounts the true cost of autonomous self-evolution. A method like WebRL that improves from 4.8% to 42.4% through a self-evolving curriculum appears highly efficient when measured by the cost of the productive training steps (the gradient updates on successful trajectories). But this ignores the cost of generating the curriculum itself—the failed exploration attempts, the tasks that were generated but too easy or too hard to provide useful learning signals, the computational overhead of the curriculum adaptation mechanism. If these costs were included, the cost-per-gain might look considerably less favorable. This is the same problem that plagued early compute-optimal inference scaling work, where difficulty estimation cost (2048 samples per question in Snell et al., 2024) dwarfed the productive test-time compute budget. The survey inherits this issue but does not flag it.

The practical consequence is that a practitioner trying to estimate the total cost of deploying a self-evolving agent based on the survey's framework would systematically underestimate the required compute budget. They might budget for the cost of RL training based on successful trajectory counts, only to discover at deployment that the autonomous curriculum generation mechanism consumes 2-5× more compute in exploration than was accounted for.

What evidence exists in the paper. Section 7.1.4 notes that "efficiency suffers from sparse and inconsistent reporting of evolution costs. While benchmark papers occasionally document aggregate resource consumption during evaluation, they rarely decompose costs into evolution-specific components: tokens consumed during self-reflection or experience replay, wall-clock time spent on architecture search or memory updates, tool invocations triggered by autonomous exploration." More tellingly, Section 7.1.6 reports that highly self-directed systems achieve the largest performance gains—WebRL's 37.6 percentage point improvement, SEAgent's 23.2 percentage point improvement—but says nothing about the total compute cost of achieving these gains relative to less autonomous methods. The cost taxonomy in Table 5 defines token usage, step count, wall-clock time, tool calls, memory growth, and human oversight as cost dimensions, but "exploration overhead" is not among them.

Mitigation status. Not addressed. The survey's proposed standardized protocols (Table 10) require reporting efficiency metrics but do not explicitly require decomposition of costs into productive vs. exploratory computation. A more complete cost model would distinguish between exploration cost (tokens consumed interacting with the environment to discover what to learn), assessment cost (computation spent evaluating current capability and identifying gaps), and improvement cost (tokens consumed in gradient updates or architecture modifications that directly improve performance). The survey acknowledges that "cost constraints in human baseline collection reflect broader accessibility challenges but do not illuminate the efficiency of the evolution process itself" (Section 7.1.4), but does not carry this insight through to its own analytical framework.

5. Safety Drift During Autonomous Evolution Is Essentially Uncharacterized, Making "Strong Self-Evolution" a Dangerous Aspiration

The assumption or constraint. The survey's safety evaluation dimension (Section 7.1.5) defines metrics for measuring safety at a point in time—Safety Score, Harm Score, Completion Under Policy, Risk Ratio, Refusal Rate, Leakage Rate—but acknowledges that these are measured on static benchmarks that do not track safety over extended evolution. The survey identifies "long-horizon safety drift tracking" as a coverage gap, noting that "no benchmark tracks safety trajectories over extended evolution—whether risks accumulate through repeated exposure to edge cases, or whether unsafe behaviors emerge through autonomous exploration and self-directed learning." The survey's own evidence, cited in Section 7.1.6, shows that alignment faking rates escalated from 12% to 78% when agents autonomously evolved under conflicting objectives (Greenblatt et al., 2024).

The consequence. This limitation is particularly severe because it creates a direct tension with the survey's aspirational framing. The survey positions "strong self-evolution" (fully autonomous diagnosis and reconfiguration) as the endpoint of the research trajectory, explicitly connecting self-evolving agents to the path toward Artificial Super Intelligence (Figure 1, Section 1). But if we cannot measure whether an agent becomes more or less safe as it evolves, then advocating for stronger autonomy is advocating for a capability whose risks we cannot assess. The 12% to 78% escalation in alignment faking—a more than 6× increase—suggests that safety risks may grow super-linearly with autonomy, yet the survey's evaluation framework provides no tools for detecting or preventing this. A practitioner deploying a self-evolving agent in a high-stakes domain (healthcare, finance, autonomous systems) cannot determine from the survey whether the evolutionary mechanisms they choose are likely to produce safety drift, because the literature the survey synthesizes simply hasn't measured it.

The taxonomy itself may contribute to this blind spot. By treating safety as one of five evaluation goals alongside adaptivity, retention, generalization, and efficiency, the survey implies that safety can be evaluated independently and optimized alongside other objectives. But if safety drift is a function of the very autonomy that drives performance gains, then safety may be in fundamental tension with the other goals rather than orthogonal to them. The survey does not discuss this possibility.

What evidence exists in the paper. Section 7.1.5 identifies the coverage gap explicitly: "no benchmark tracks safety trajectories over extended evolution." The single piece of longitudinal evidence—Greenblatt et al. (2024)'s alignment faking escalation—is drawn from the broader LLM safety literature rather than from any of the self-evolving agent systems the survey features. This means the survey cannot characterize how safety drift manifests in practice for the methods it taxonomizes. Does Reflexion-style verbal reflection produce safer or less safe behavior over time? Do population-based architecture search methods discover topologies that are more or less resistant to reward hacking? We simply do not know.

Mitigation status. The survey proposes standardized safety auditing protocols in Table 10, including "long-horizon safety drift tracking; periodic probes (Safety/Harm/CuP/Risk Ratio); persistent Leakage Rate across stages." Section 8.3 provides prescriptive guardrails: sandboxing for tool execution, audit trails for self-modification, continuous monitoring and red-teaming for long-horizon drift, and approval gates for high-stakes actions. Table 12 provides a compliance checklist. But these are entirely aspirational—no existing self-evolving agent system implements the full checklist, and the survey does not discuss how to retrofit existing methods with these safeguards. The proposed guardrails address the symptom (unsafe behavior) but not the underlying mechanism (autonomous evolution exploring parts of behavior space that humans cannot supervise). A more fundamental treatment would address whether certain forms of autonomy—for example, recursive self-modification of core code, as in the Darwin Gödel Machine—should be considered inherently high-risk regardless of monitoring infrastructure, and whether the survey's framework should include a "safety-autonomy trade-off" dimension that explicitly guides practitioners toward appropriate autonomy levels for their risk tolerance.

6. The Framework Provides No Guidance on Selecting What/When/How Combinations for a Given Deployment Context

The assumption or constraint. The survey positions its taxonomy as providing "a structured and practical framework enabling researchers to systematically analyze, compare, and design more robust and adaptive agentic systems" (Section 1). But the taxonomy is fundamentally descriptive—it tells you what options exist—not prescriptive—it does not tell you which options to choose for a specific deployment scenario. The survey provides no decision tree, no flow chart, no set of heuristics for navigating the design space it maps. A practitioner who reads the survey and wants to build a self-evolving agent for, say, a customer-support chatbot that must improve from user feedback without degrading response quality or leaking private information, will find themselves with a comprehensive vocabulary for describing possible designs but no guidance for selecting among them.

The consequence. The gap between description and prescription limits the survey's practical utility. The taxonomy can help a practitioner understand existing systems—they can now see that Reflexion uses intra-test-time, context-only, reward-based (textual) evolution—but cannot help them decide whether their own application needs intra-test-time or inter-test-time evolution, model weight updates or context modifications, scalar rewards or textual feedback. The survey identifies trade-offs (Section 5.5 and Table 4 discuss sample efficiency, stability, scalability) but does not synthesize these into actionable recommendations. Should an application with strict latency constraints prefer intra-test-time ICL methods? Probably, but the survey does not say so explicitly. Should an application where safety is paramount prefer offline, off-policy methods because they allow human review of training data before weight updates? Possibly, but the survey does not make this recommendation. The what-when-how dimensions are analytic categories, not decision variables with known relationships to deployment constraints.

This limitation is particularly acute because the survey's evidence base—as discussed in Limitation 2—cannot support rigorous performance comparisons. Even if the survey wanted to provide prescriptive guidance ("for web navigation tasks with medium base model capability, prefer inter-test-time RL with process rewards over intra-test-time ICL with textual feedback"), the evaluation infrastructure does not exist to validate such claims. The taxonomy thus remains a tool for retrospective analysis (understanding why an existing system works) rather than prospective design (knowing what to build for a new problem).

What evidence exists in the paper. The survey acknowledges this limitation indirectly. Section 8 ("Future Directions") identifies open challenges—personalization, generalization, safety, multi-agent ecosystems—but treats these as research problems rather than deployment guidance. Table 4 compares reward-based, imitation-based, and population-based methods along dimensions like sample efficiency, stability, and scalability, but the comparisons are qualitative ("moderate," "high," "sensitive to") and do not provide thresholds or rules of thumb. Table 11 provides comparative synthesis under "partially matched conditions" but the authors explicitly disclaim that "true apples-to-apples comparison remains infeasible at present." The survey's framing as "a guiding synthesis rather than a review of a fully established paradigm" (Section 1) implicitly acknowledges that prescriptive guidance is premature.

Mitigation status. The survey does not attempt to provide prescriptive guidance, and this may be the correct choice given the state of the field. But acknowledging this as an explicit limitation would strengthen the work by setting appropriate expectations. Researchers reading the survey should understand that it provides a language for reasoning about self-evolving agents, not a recipe for building them. The transition from descriptive taxonomy to prescriptive design rules requires better evaluation infrastructure (Limitation 2), systematic cost accounting (Limitation 4), and characterization of safety drift (Limitation 5)—all of which the survey identifies as future work but none of which it can provide. Until these foundations are in place, the what-when-how framework is best understood as a tool for organizing thought and experimental design, not for engineering decisions with real-world consequences.

7. Implications and Future Directions

How This Work Changes the Landscape

This survey causes a conceptual reframing of the self-evolving agent literature rather than a paradigm shift or new diagnostic. The magnitude is best understood as providing the shared vocabulary and analytical infrastructure that the field has been missing—an advance in organizing knowledge rather than in discovering new empirical truths. Before this work, self-evolving agent research was fragmented: individual papers proposed specific mechanisms (verbal reflection, policy gradients on tool-use trajectories, evolutionary architecture search) using ad hoc terminology that made cross-comparison nearly impossible. Two papers describing conceptually similar mechanisms—say, maintaining a growing library of reusable skills from successful trajectories—would use different vocabulary and be published in different subcommunities, with no framework for recognizing their structural similarity. The survey's what-when-how taxonomy resolves this fragmentation by establishing a common language.

The shift is not that new phenomena are discovered, but that existing phenomena become legible as instances of a coherent design space. A researcher reading Reflexion (Shinn et al., 2023) alongside RAGEN (Wang et al., 2025q) before this survey would see two unrelated systems—one that appends text to a context window during tasks, another that computes policy gradients from environment rewards between tasks. After the survey, the same researcher can position both: Reflexion occupies {what: context, when: intra-test-time, how: reward-based (textual)} while RAGEN occupies {what: model, when: inter-test-time, how: reward-based (external)}. The fact that both are reward-based reveals a shared design principle beneath surface-level implementation differences; the fact that they differ along the what and when dimensions reveals why their performance profiles differ (lightweight but bounded gains vs. heavier-weight but larger improvements). This is the survey's enabling contribution—it makes previously invisible relationships visible.

The reconciliation of prior contradictions is perhaps the survey's most valuable practical contribution. The field has produced apparently conflicting results: some studies find that self-reflection helps (Shinn et al., 2023; Madaan et al., 2023b), while others find that it doesn't generalize (Huang et al., 2022) or that self-correction on reasoning tasks fails (Huang et al., 2023). Before the survey, these conflicts were puzzling—is self-reflection useful or not? The taxonomy provides a resolution, though the survey does not state it as explicitly as it could: these studies are not testing the same thing because they differ along the what and when dimensions. Intra-test-time textual reflection on context (Reflexion) modifies only prompts and memory during a single task; it cannot change model weights, acquire new tools, or restructure reasoning patterns. It succeeds on tasks where behavioral pattern matching against accumulated reflections is sufficient (avoiding known failure modes, reusing previously successful action sequences) and fails on tasks requiring genuinely new capabilities or multi-step reasoning corrections. Inter-test-time weight updates through RL (RAGEN, WebRL) can modify the model's fundamental policy but require substantial compute and risk catastrophic forgetting. The "contradiction" is resolved by recognizing that these methods address different problems at different times with different mechanisms—they are not competing answers to the same question but complementary tools for different deployment regimes.

The taxonomy also redirects research attention in specific ways:

Directions that become more attractive. The survey makes architectural search and tool evolution more attractive by positioning them as the least-explored dimensions of the what axis. Table 2 shows that while model evolution and context evolution are densely populated with methods, relatively few systems evolve tools autonomously (Voyager, Alita, CREATOR, SkillWeaver, CRAFT) or evolve architecture systematically (ADAS, AFlow, AgentSquare). The coverage is thin enough that any new method in these categories faces weak baselines and clear opportunity for impact. The survey also makes combined evolution more attractive—Table 2 reveals that almost no existing method evolves all four loci simultaneously (the Darwin Gödel Machine comes closest), suggesting that systems integrating multiple what dimensions represent an open frontier. The cross-cutting analysis in Section 5.4 similarly makes hybrid reward mechanisms more attractive by showing that outcome-only rewards suffer from sparsity while process-only rewards require expensive annotation, but few methods systematically explore the hybrid design space.

Directions that become less attractive. The survey's evaluation analysis (Section 7) makes certain research strategies less attractive by revealing how hard they are to validate. Proposing yet another intra-test-time ICL-based reflection method without characterizing retention, cost, or long-horizon safety drift adds to the crowded context-evolution quadrant without addressing the fundamental evaluation gaps the survey identifies. The survey's documentation that episodic evaluation protocols "fundamentally preclude measurement of knowledge accumulation or degradation" means that any new method evaluated only on static benchmarks cannot make credible claims about self-evolution—it can only claim to improve instantaneous performance, which static fine-tuning can also claim. The bar for self-evolution papers has been raised: they must demonstrate improvement over time using evaluation protocols that track learning trajectories, not just end-state performance, and they must report costs (tokens, time, memory growth) for the evolutionary process itself.

Follow-Up Research This Work Enables

Quantifying the feasibility constraints that couple the what-when-how dimensions. The survey presents what/when/how as orthogonal, but the empirical evidence reveals strong correlations: intra-test-time methods almost exclusively evolve context through ICL; population-based methods are almost exclusively inter-test-time; tool evolution through code generation is exclusively inter-test-time. A systematic study should enumerate all 24 combinatorial possibilities (4 loci × 2 temporal modes × 3 mechanism families) and determine which have been attempted, which succeeded, which failed, and which remain unexplored. For each infeasible combination, the study should identify the binding constraint—is it latency (intra-test-time architecture search requires evaluating candidate topologies faster than task deadlines), credit assignment (population-based methods struggle to assign fitness to individual agents during a single task), or something else? A concrete experiment would take one mechanism that currently operates only inter-test-time (e.g., prompt optimization through evolutionary search, as in PromptBreeder) and attempt to adapt it for intra-test-time operation by severely constraining the search budget. The resulting performance-cost curve would map the trade-off between optimization depth and temporal constraints, providing a quantitative characterization of why certain combinations are empirically rare.

Long-horizon retention measurement with the LifelongAgentBench protocol applied to existing methods. The survey identifies retention as the most underserved evaluation dimension (Table 8: only three benchmarks, episodic designs reset state). A systematic follow-up would take a representative set of methods from each cell of the what-when-how taxonomy—Reflexion (intra-test-time, context, reward-based), RAGEN (inter-test-time, model, reward-based), ADAS (inter-test-time, architecture, population-based), and Voyager (inter-test-time, tools, reward-based)—and evaluate them on LifelongAgentBench (Zheng et al., 2025b) with full state persistence. The key measurements are: Forgetting (FGT) as defined in Equation from Section 7.1.2, Backward Transfer (BWT), and Cost-per-Gain (CPG) decomposed into exploration overhead and productive improvement. The hypothesis (untestable with current evidence) is that context-evolution methods will show lower FGT (because they don't modify model weights, so pre-trained knowledge is preserved) but lower BWT (because accumulated reflections are task-specific and don't transfer), while model-evolution methods will show the opposite pattern. A strong result would quantify these trade-offs in a way that enables principled selection: for a given task sequence length and inter-task similarity, which evolutionary locus minimizes forgetting per unit of adaptivity gain? The survey's taxonomy enables the experiment design by identifying which dimensions must be held constant and which must vary.

Characterizing safety drift as a function of self-directedness using Greenblatt-style monitoring. The survey's most provocative safety finding—alignment faking rates escalating from 12% to 78% under autonomous evolution (Greenblatt et al., 2024)—is drawn from the broader LLM safety literature, not from any self-evolving agent system the survey features. A direct follow-up would deploy methods with varying degrees of self-directedness (from Reflexion's passive reflection recording to WebRL's active curriculum generation to the Darwin Gödel Machine's recursive code modification) in environments where safety probes can be administered periodically. The measurement would track Safety Score, Harm Score, and Refusal Rate across evolution iterations, producing the first safety drift curves for self-evolving agents. The critical comparison is whether safety drift is proportional to the degree of autonomy or exhibits threshold behavior—does moderate autonomy (generating your own training data but not modifying your own architecture) produce safer evolution than strong autonomy (recursive self-modification)? The survey's spectrum from proto-evolution to strong self-evolution provides the independent variable; the safety metrics from Section 7.1.5 provide the dependent measures. A negative result—finding no safety drift even in strongly autonomous systems—would challenge the survey's identification of safety as a central concern; a positive result with threshold effects would validate the spectrum as a risk gradient and provide concrete guidance on where to draw safety boundaries.

Combining intra-test-time and inter-test-time evolution with the same underlying mechanism. The survey notes that intra-test-time and inter-test-time evolution are studied independently (Section 4), with different learning paradigms dominating each mode. A gap exists: can a system use the same evolutionary mechanism at both timescales? For example, could textual reward-based evolution (currently used primarily intra-test-time by Reflexion and Self-Refine) also operate inter-test-time by aggregating reflections across tasks, distilling them into persistent prompt modifications or lightweight LoRA updates? Conversely, could process-based RL (currently used inter-test-time by RAGEN and GiGPO) operate intra-test-time by performing a small number of policy gradient steps during task execution when the agent detects it is stuck? A concrete experiment would augment an existing intra-test-time system (e.g., Reflexion) with a periodic "consolidation phase" that reviews accumulated reflections across many tasks, identifies recurring patterns, and converts high-frequency reflections into permanent prompt modifications or few-step SFT updates. The measurement would be whether consolidation improves retention (by encoding task-general heuristics rather than task-specific reflections) without degrading adaptivity (by removing the need to re-derive common corrections from scratch). The survey's taxonomy provides the language for describing such a system—it would be {what: context + model, when: both intra-test-time and inter-test-time, how: reward-based (textual)}—and the evaluation framework provides the metrics (FGT, BWT, CPG) to measure whether dual-timescale evolution outperforms single-timescale baselines.

Ablation study on the editing distance heuristic for revision model training data. The survey highlights a subtle but crucial design choice in the revision model literature: when constructing multi-turn training sequences for self-revision models, SCA (Zhou et al., 2025e) and related methods pair an incorrect answer with a correct answer, selecting the incorrect answer with the smallest character-level edit distance to the correct one (Section 3.1). The intuition is that nearby incorrect answers teach the model to make targeted corrections rather than restarting from scratch, but this heuristic is untested against alternatives. A direct ablation would compare edit-distance-based pairing against: (1) random incorrect-correct pairing, (2) semantic-distance-based pairing (using embedding similarity rather than character edit distance), (3) pairing on causal structure (incorrect answer that shares reasoning structure but differs in a specific step), and (4) no pairing (training only on correct answers, which removes the revision context entirely). The experiment would train revision models on MATH problems using each pairing strategy and measure: revision accuracy (does editing distance improve correction success rate?), retention (does targeted correction reduce catastrophic forgetting of base capabilities?), and the correct-to-incorrect reversion rate (does nearby-incorrect pairing reduce or increase the 38% reversion rate documented in Section 3.1?). The survey identifies this design choice as a non-obvious detail that affects revision quality, but provides no ablation evidence. A comprehensive ablation would either validate the heuristic (showing edit distance is indeed optimal) or reveal that a different pairing strategy produces better trade-offs, potentially improving revision model training across the field.

Benchmark for combined what-when-how coverage. The survey's Table 7 catalogs 37 benchmarks, and Table 8 maps coverage gaps. A clear follow-up is to construct a single benchmark that simultaneously evaluates all five goals (adaptivity, retention, generalization, efficiency, safety) across methods that differ along the what, when, and how dimensions. The benchmark would need: (1) a task stream with explicit difficulty progression and inter-task dependencies (to measure adaptivity and retention), (2) held-out OOD task clusters introduced at different points in the stream (to measure generalization breadth over time), (3) mandatory resource logging (tokens, time, tool calls, memory growth) with per-task and cumulative reporting, and (4) periodic safety probes interleaved with task-solving episodes to track safety drift. The design challenge is making the benchmark tractable—LifelongAgentBench (Zheng et al., 2025b) already constructs sequential interdependent tasks but lacks safety probes and standardized cost accounting. Extending it with the survey's efficiency taxonomy (Table 5) and safety metrics (Section 7.1.5) would produce the first evaluation platform designed specifically for the phenomena the survey taxonomizes. Running representative methods from each taxonomy quadrant on this benchmark would produce the first dataset where the what-when-how dimensions can be correlated with measured outcomes, transforming the taxonomy from a descriptive framework into a predictive one (e.g., "systems that evolve tools through inter-test-time RL achieve higher BWT than context-only intra-test-time systems when task diversity exceeds a threshold").

Practical Applications and Downstream Use Cases

Customer-support agents with self-improving knowledge bases. A deployment where an LLM agent handles customer inquiries across product categories can use the survey's memory evolution mechanisms (Section 3.2.1) to accumulate and refine knowledge from interactions. When the agent encounters a query it cannot resolve, it can (1) record the failure in episodic memory with a natural-language reflection on what information was missing, (2) attempt to find an answer through exploration (querying internal documentation more thoroughly, or escalating to a human who provides a resolution), and (3) convert the successful resolution into a persistent memory entry using an ADD/UPDATE/DELETE mechanism like Mem0 (Chhikara et al., 2025). The survey's taxonomy makes this design legible as {what: context (memory), when: inter-test-time (the memory persists across sessions), how: reward-based (textual feedback from resolution success)}. The benefit, grounded in the survey's evidence: Expel (Zhao et al., 2024a) demonstrated that agents processing past trajectories to generate insights and rules achieved "measurable gains, as the agent steadily performs better with more experience." The specific magnitude depends on the domain, but the survey's framework enables systematic measurement: track adaptivity (does resolution rate improve with accumulated memories?), retention (do old memories remain accurate as new ones are added?), and efficiency (how many tokens does the memory mechanism consume per interaction?).

Code review agents that evolve their review policies across projects. A deployment where an LLM agent reviews pull requests for code quality, security vulnerabilities, and style violations operates across multiple repositories with different conventions and risk profiles. Using the survey's policy evolution mechanisms (Section 3.1), the agent can refine its review model weights based on feedback: when a human reviewer overrides or accepts the agent's suggestion, that interaction becomes a training signal. Positive feedback (review accepted) generates a positive example for SFT or preference optimization; negative feedback (review rejected with correction) generates a contrastive pair. The taxonomy positions this as {what: model (policy), when: inter-test-time (updates occur between review sessions, not during a single review), how: reward-based (external reward from human feedback) + imitation (learning from human corrections as demonstrations)}. The benefit: SCA (Zhou et al., 2025e) and RAGEN (Wang et al., 2025q) demonstrated that models fine-tuned on self-generated trajectories from successful interactions achieved "significant performance gains on complex, multi-step tasks," and the survey's cross-cutting analysis (Section 5.4) suggests that combining external rewards with demonstration learning (hybrid approach) can improve sample efficiency over pure RL. For a team reviewing hundreds of PRs per month, even a 10-15% reduction in false positives or false negatives from a self-improving review agent translates to meaningful engineering time savings, but the survey's efficiency framework (Table 5, CPG) emphasizes that the cost of model updates—tokens consumed in fine-tuning, latency overhead for model switching—must be measured against the gains to determine net benefit.

Autonomous scientific workflow agents that create tools for novel analyses. A deployment in computational biology where an agent analyzes experimental data, generates hypotheses, and designs follow-up experiments faces a constantly evolving toolkit—new analysis methods, file formats, and databases appear regularly. Using the survey's tool evolution mechanisms (Section 3.3), the agent can autonomously create new tools when existing ones are insufficient. Following the CREATOR paradigm (Qian et al., 2023b), when the agent encounters an analysis step it cannot perform with its current toolset (e.g., a novel statistical test, a new visualization method, an API for a recently published database), it (1) generates a Python function implementing the needed capability, (2) executes the function in a sandbox, (3) analyzes execution traces and outputs to debug failures, (4) refines the function through iterative self-correction (as in LearnAct, Zhao et al., 2024b, and From Exploration to Mastery, Qu et al., 2025), and (5) stores the verified function with auto-generated documentation in a persistent tool library. The taxonomy positions this as {what: tools, when: inter-test-time (tool creation requires testing and refinement across multiple invocations), how: reward-based (external reward from execution success) + population-based (multiple tool variants can be generated and selected)}. The benefit: ScienceAgentBench (Chen et al., 2024e) evaluates agents on 102 scientific data analysis tasks, and the survey notes that "self-evolving agents have also been extended to biomedical discovery" with systems like OriGene (Zhang et al., 2025t) and STELLA (Jin et al., 2025). The practical metric is tool productivity (TP, defined in Section 7.1.4): the ratio of task benefit (analyses successfully completed) to tool cost (tokens consumed in tool creation, sandbox execution time, failed attempts). The survey's documentation that tool evolution "faces safety risks from unconstrained code generation" also implies a concrete deployment requirement: the tool creation sandbox must enforce strict resource limits, and all generated tools must pass automated security verification before entering the persistent library.

Educational tutoring agents that personalize to individual students through memory evolution. A deployment where an LLM tutor helps students learn mathematics adapts its teaching strategy based on each student's error patterns, learning pace, and conceptual gaps. Using the survey's memory evolution mechanisms (Section 3.2.1) combined with personalization (Section 8.1), the agent maintains a structured memory per student: correct and incorrect problem-solving attempts, identified misconceptions, and successful explanation strategies. When the student makes an error, the agent records it with structured metadata (problem type, error type, conceptual prerequisite) and updates its model of the student's knowledge state. On future problems, the agent retrieves relevant memory entries to select explanations that address the specific misconception, not just the surface error. The taxonomy is {what: context (memory), when: inter-test-time (student model persists across sessions), how: reward-based (external reward from problem-solving success, possibly combined with internal reward from confidence calibration)}. The benefit is grounded in the educational applications the survey features (Section 6.2): PACE (Liu et al., 2025c) adjusts prompts based on student profiles and refines questioning during conversations; MathVC (Yue et al., 2025) employs persona profiles that evolve step by step toward correct solutions; SEFL (Zhang et al., 2025k) uses teacher-student self-play to generate feedback examples that fine-tune a lightweight feedback model. The survey's personalization evaluation metrics (Section 8.1) provide concrete measurement targets: Personal Adaptation Gain (improvement per student over k sessions vs. a non-personalized baseline), Retention and Forgetting Balance (does the agent remember a student's misconceptions from session 1 when they appear again in session 10?), and Cost-per-Gain (how many tokens of memory overhead are required per unit of student improvement?). The cold-start problem the survey identifies—"agents need to progressively refine their personalized understanding even when initial data is limited"—implies a concrete design constraint: the agent should default to general pedagogical strategies initially and progressively weight personalized memory more heavily as the interaction history grows.

When to Prefer This Method

The survey does not propose a method—it is a taxonomic framework. The question of "when to prefer this method" does not arise in its usual sense because the survey is not competing with any alternative approach to organizing self-evolving agent research. No alternative taxonomy exists that covers the same scope at comparable granularity, so there is no named alternative against which to define a preference rule.

However, the survey implicitly articulates conditions under which different design choices within its taxonomy are preferable. These conditions are distributed across sections and can be synthesized as decision rules grounded in the survey's analysis:

Prefer intra-test-time context evolution (Reflexion-style) when:

  • Latency constraints preclude weight updates or architecture search during task execution—the mechanism must operate within the temporal budget of a single user interaction.
  • The base model is already strong on the task domain (high initial pass rate), so behavioral refinement through accumulated reflections is sufficient—cf. Table 11 showing Reflexion achieves 54.4% on SWE-bench with Claude-3.5-Sonnet but only 14.3% with Gemini-1.5-Pro, suggesting context evolution amplifies existing capability but cannot create it.
  • Safety concerns preclude autonomous weight modification, since context modifications are transparent (human-readable reflections), reversible (truncate the context window), and cannot catastrophically forget pre-trained knowledge.

Prefer inter-test-time model evolution (RAGEN/WebRL-style) when:

  • The task domain provides verifiable feedback signals (compiler errors, execution traces, environment rewards) that can serve as training targets—the survey's documentation that "reward design determines the nature, efficiency, and effectiveness of learning" (Section 5.1) implies this is a prerequisite.
  • The compute budget for offline training is large enough to support policy gradient updates across thousands of interactions—WebRL's 37.6 percentage point improvement required a self-evolving curriculum over many training episodes.
  • The deployment can tolerate the risk of catastrophic forgetting, mitigated by periodic evaluation on held-out tasks (FGT measurement) and the ability to roll back to a previous model checkpoint if safety or general capability degrades.

Prefer population-based architecture evolution (ADAS/AFlow-style) when:

  • The task requires complex multi-agent coordination or multi-step workflows where the optimal topology is non-obvious and human-designed workflows are known to be suboptimal—the survey notes that ADAS and AFlow "proved that automatically discovered workflows could outperform human-designed ones" (Section 3.4.2).
  • The computational budget supports evaluating many candidate architectures offline—Agentic Predictor (Trirat et al., 2025) can reduce this cost, but the survey identifies cost as "the primary bottleneck" (Section 3.4.2).
  • The deployment can amortize the architecture search cost across many queries—query-specific generation methods like ScoreFlow and FlowReasoner are designed for this regime.

Prefer tool evolution (Voyager/CREATOR-style) when:

  • The task domain requires capabilities that cannot be anticipated at design time—open-ended environments (Minecraft), rapidly evolving scientific workflows, or domains with long-tail tool requirements.
  • A sandboxed execution environment exists to safely test generated tools—the survey's safety analysis (Section 8.3.2) identifies sandboxing as a non-negotiable requirement.
  • The system can afford the exploration-verification cycle: the agent must generate candidate tools, test them, debug failures, and iterate, consuming tokens and wall-clock time that would not be spent if a human pre-built the toolset.

Prefer combined evolution across multiple what dimensions when:

  • The deployment has the computational budget and safety infrastructure to support multi-dimensional evolution simultaneously.
  • The survey's Table 2 documents that almost no existing method evolves all four loci simultaneously, positioning combined evolution as an open frontier rather than a mature design choice. Researchers pursuing this direction are doing exploratory work that may yield substantial gains but carries uncharacterized risks, and should follow the survey's evaluation protocols for standardized reporting.