Requirement Adherence: Boosting Data Labeling Quality Using LLMs

🎯 Pitch

LLMs can dramatically improve their own output—by up to 49 percentage points—simply by critiquing and revising their work, with no external supervision or model updates. This SELF-REFINE loop unlocks this capability instantly in GPT-4 and GPT-3.5 across seven tasks ranging from math proofs to dialogue generation.

1. Executive Summary

This paper introduces SELF-REFINE, an approach for improving LLM outputs through iterative self-feedback and refinement without any supervised training data, additional training, or reinforcement learning—using a single LLM as both the generator and the feedback provider. The work evaluates SELF-REFINE across 7 diverse tasks—including dialogue response generation, code optimization, math reasoning, and sentiment reversal—using GPT-3.5, ChatGPT, and GPT-4, demonstrating that the same model can both critique its own outputs and incorporate that critique into improved generations through alternating FEEDBACK and REFINE steps (e.g., pinpointing inefficient loop structures in code and rewriting to use a closed-form formula, or identifying that a dialogue response lacks specificity and making it more engaging). Across all evaluated tasks, SELF-REFINE improves absolute task performance by approximately 20% on average, with particularly large gains on preference-based tasks like Dialogue Response Generation (GPT-4 preference score improving from 25.4% to 74.6%, a 49.2% absolute increase), establishing that the approach yields substantial improvements primarily when the base model possesses sufficient few-shot or instruction-following capabilities to generate actionable, specific feedback—a boundary condition evidenced by Vicuna-13B's inability to consistently produce feedback in the required format, causing the refinement loop to fail.

2. Context and Motivation

The Core Problem: LLMs Don't Always Nail It on the First Try

The fundamental problem SELF-REFINE addresses is both intuitive and pervasive: even state-of-the-art LLMs frequently produce outputs that are suboptimal on their first attempt, particularly on tasks involving multiple interacting constraints or hard-to-specify quality criteria. The paper opens with a direct observation (Section 1):

"Although large language models (LLMs) can generate coherent outputs, they often fall short in addressing intricate requirements. This mostly includes tasks with multifaceted objectives, such as dialogue response generation, or tasks with hard-to-define goals, such as enhancing program readability."

This is not about LLMs generating wrong answers—it is about them generating adequate but improvable ones. An LLM may produce a functionally correct piece of code that nonetheless brute-forces through six nested loops when dynamic programming would be far more efficient. It may generate a dialogue response that stays on-topic but reads as generic and unengaging. It may write a sentiment-reversed review that technically flips the polarity but uses bland, unconvincing language.

The gap the paper identifies is therefore not about base capability, but about the mismatch between single-pass generation and iterative quality improvement. Humans do not typically finalize complex work in one draft—they draft, review, identify weaknesses, and revise. This paper asks: can LLMs do the same, using only their own internal feedback, without any external supervision or training?

Why This Matters: Practical and Conceptual Significance

Practical significance: removing barriers to output improvement. The paper emphasizes that existing methods for improving LLM outputs impose substantial costs that limit their applicability (Section 1):

"Other approaches that rely on external supervision or reward models require large training sets or expensive human annotations (Madaan et al., 2021; Ouyang et al., 2022), which may not always be feasible to obtain."

Training a task-specific refinement model requires curated data of (input, flawed-output, feedback, improved-output) tuples—expensive to collect at scale, and requiring repetition for every new domain. Reinforcement learning from human feedback (RLHF) demands ongoing human annotation and model retraining. Both paths create a barrier between wanting to improve a model's output and actually being able to do so.

SELF-REFINE's promise is that it removes these barriers entirely: no training data, no supervised learning, no RL, no reward model. Just the same LLM, prompted appropriately, providing feedback on its own work and incorporating that feedback into a revision. If this works reliably, it means that any deployment of a capable LLM can get substantially better outputs at test time with minimal engineering overhead—a direct practical benefit for anyone building LLM-powered applications.

Conceptual significance: self-improvement without parameter updates. Beyond practical convenience, SELF-REFINE probes a deeper question about LLM capabilities: can a model recognize and articulate the weaknesses in its own output, and then act on that diagnosis to produce something better? This touches on metacognitive abilities—the capacity for self-evaluation and self-correction—that are central to human intelligence. Prior work had shown conflicting evidence on this front, as we will discuss below. SELF-REFINE's positive results suggest that these abilities do exist in modern LLMs when prompted appropriately, but that they require specific mechanisms (actionable, specific feedback; iterative refinement; multi-aspect evaluation) that prior approaches did not consistently provide.

Where Prior Approaches Fall Short

The paper identifies several categories of existing work and articulates specific limitations for each (Section 5, Table 3, Appendix B).

Trained refinement models require per-task supervision. Several prior works learn a separate refiner model from pairs of feedback and refinement (Schick et al., 2022b with PEER; Du et al., 2022; Yasunaga and Liang, 2020 with DrRepair; Madaan et al., 2021). For example, PEER (Schick et al., 2022b) trains a model on Wikipedia edits to learn document revision; Self-Correction (Welleck et al., 2022) trains a task-specific "corrector" model that refines initial outputs. These approaches work, but the paper identifies a key limitation:

"gathering supervised data is costly. [...] However, the refiners are trained for each new domain."

This means that deploying such a system on a new task or dataset requires a fresh round of data collection and model training—a significant barrier for practitioners working across diverse applications.

Reinforcement learning alternatives require parameter updates and lack intermediate feedback. An alternative paradigm to explicit refinement is optimizing a scalar reward function via RL (Stiennon et al., 2020 with RLHF; Lu et al., 2022 with QUARK; Le et al., 2022a with CodeRL). The paper notes two disadvantages compared to SELF-REFINE (Appendix B):

"the model cannot access feedback on an intermediate generation. Second, these reinforcement learning methods require updating the model's parameters, unlike SELF-REFINE."

The first point is subtle but important: in RL-based approaches, the model only sees a final reward signal (a scalar score), never a detailed diagnosis of what specifically went wrong and why. This limits the model's ability to learn targeted improvements. The second point is practical: RLHF requires significant infrastructure and compute for parameter updates, whereas SELF-REFINE works as a zero-shot prompting strategy at inference time.

Prompted feedback methods either use external sources or are domain-restricted. Recent work has explored using LLMs to generate feedback in a prompted, training-free manner. But the paper draws sharp distinctions:

External feedback sources: Augmenter (Peng et al., 2023) uses external knowledge bases to provide factuality feedback—the LLM is not reflecting on its own output but receiving signals from outside. Re³ (Yang et al., 2022) uses trained critics rather than self-generated feedback. The paper positions these as less general than pure self-feedback.
Single-domain self-feedback: Re³ (Yang et al., 2022) demonstrates prompted feedback and refinement for story generation specifically but does not demonstrate cross-task generality.
Free-form vs. structured feedback: Reflexion (Shinn et al., 2023), a concurrent work, refines plans using free-form reflection, but the paper argues this is less granular:

"our approach is more granular and structured, with multi-dimensional feedback and scores. This distinction allows our method to offer more precise and actionable feedback, making it suitable for a wider range of natural language generation tasks."

The closest prior work: Self-Correction (Welleck et al., 2022). The paper explicitly acknowledges Self-Correction as the closest predecessor and enumerates three specific shortcomings that SELF-REFINE addresses (Appendix B):

"1. Self-Correction does not train their model to generate explicit feedback; instead, Welleck et al. (2022) trained their models to refine only. As we show in Section 4 and Table 2, having the model generate explicit feedback results in significantly better refined outputs. 2. Self-Correction trains a separate refiner (or 'corrector') for each task. In contrast, SELF-REFINE uses instructions and few-shot prompting, and thus does not require training a separate refiner for each task. 3. Empirically, we evaluated SELF-REFINE using the same base model of GPT-3 as Self-Correction, and with the same settings on the GSM8K benchmark. Self-Correction achieved 45.9% accuracy while SELF-REFINE (this work) achieved 55.7% (↑9.8)."

This is a critical comparative claim: the feedback generation step is not incidental—it is what enables the model to produce better refinements, as evidenced by the ablation showing a 9.8-point gap even when the base model and task are held constant.

How SELF-REFINE Positions Itself

The paper frames SELF-REFINE as filling a specific, previously unoccupied cell in the design space of refinement approaches (Table 3, Section 5). The key differentiators are:

1. Supervision-free in both dimensions. SELF-REFINE requires neither supervised training of a refiner (unlike PEER, Self-Correction, DrRepair) nor supervised training of a feedback provider (unlike methods that use human annotators, scalar reward models, or external knowledge sources). Everything is done through few-shot prompting of a single LLM. This makes it uniquely low-cost to deploy across domains.

2. Multi-aspect, natural language feedback. Rather than a scalar reward or a binary correct/incorrect flag, SELF-REFINE generates structured, actionable feedback in natural language that can address multiple quality dimensions simultaneously (relevance, informativeness, engagement, specificity, etc. for dialogue; pronunciation ease, spelling ease, relation to title, positive connotation for acronyms). The paper explicitly contrasts this with approaches that use "single" or "scalar" feedback signals (Table 5).

3. Iterative, with history retention. SELF-REFINE does not just do one round of feedback-and-refine; it iterates (up to a default of 4 iterations), maintaining the full history of previous outputs and their feedback in context. Equation 4 in Section 2 makes this explicit:

$y_{t+1} = M(p_{\text{refine}} \| x \| y_0 \| fb_0 \| ... \| y_t \| fb_t)$

This allows the model to learn from past mistakes across iterations, avoiding repeated errors and progressively converging toward higher quality. The paper emphasizes that this iterative structure with memory is absent in single-pass refinement methods.

4. Same-model architecture. A subtle but important design choice highlighted in Figure 1 and Section 2: the same model $M$ serves as generator, feedback provider, and refiner. This is not just an implementation convenience—it means the model critiques its own outputs using the same knowledge and capabilities it used to generate them, which the paper frames as a form of self-consistency that avoids distribution mismatch between separately trained components.

5. Task-agnostic, benchmark-spanning evaluation. While many prior refinement approaches are demonstrated on one or two tasks, SELF-REFINE is evaluated across 7 diverse tasks spanning natural language and code generation, with three different base LLM families. This breadth is positioned as evidence for generality rather than domain-specific tuning.

In essence, the paper's thesis is that the combination of (a) explicit, multi-aspect natural language feedback, (b) iterative refinement with memory, (c) the same model performing all roles, and (d) zero training cost enables consistent improvements across diverse tasks where prior approaches either required per-task supervision, used impoverished feedback signals, or failed to demonstrate cross-domain generality. The results in Table 1 showing improvements across 6 of 7 tasks for GPT-4 (the trivial gains on Math Reasoning being the exception) are presented as confirmation of this thesis.

3. Technical Approach

3.1 Reader Orientation (Approachable Technical Breakdown)

SELF-REFINE is not a new model, a training procedure, or a specialized architecture—it is a prompting algorithm that orchestrates a single LLM through alternating cycles of self-critique and self-improvement, using few-shot examples to teach the model how to generate feedback and how to incorporate it. The system solves the problem of producing higher-quality outputs from an already-capable LLM without any external supervision, by giving the model itself the role of critic and reviser, iterating until a stopping condition is met.

3.2 Big-Picture Architecture (Diagram in Words)

The system has three functional roles, all played by the same model M:

Initial Generator (INIT) — Given an input x and a task-specific prompt p_gen, the model produces an initial output y_0. This is the standard single-pass LLM generation—the baseline against which improvements are measured.
Feedback Provider (FEEDBACK) — The same model M receives its own output y_t (along with the original input x) and a feedback prompt p_fb, and produces structured, multi-aspect natural language feedback fb_t that diagnoses specific weaknesses and suggests concrete improvements.
Refiner (REFINE) — The model receives the full history (input, previous outputs, all previous feedback) plus a refinement prompt p_refine, and generates a revised output y_{t+1} that addresses the identified issues.

These three roles alternate in a loop: INIT → FEEDBACK → REFINE → FEEDBACK → REFINE → ... until a stopping condition is met (either a maximum iteration count, typically 4, or a model-generated stop signal in the feedback). The full trajectory of past outputs and feedback is retained in the prompt context at each refinement step, giving the model access to its entire revision history.

3.3 Roadmap for the Deep Dive

First, the formal generation and refinement equations (Equations 1–4), which define the algorithmic loop and specify exactly what context each step receives—understanding these is prerequisite to understanding every other design choice.
Second, the prompt structure (p_gen, p_fb, p_refine) and few-shot example design, since the entire approach depends on in-context learning to teach the model how to provide actionable, specific, multi-aspect feedback and how to refine based on that feedback.
Third, the feedback quality requirements (actionable, specific, multi-aspect), which are empirically shown to be the critical determinant of SELF-REFINE's effectiveness via the ablation in Table 2.
Fourth, the iteration and stopping mechanism—how the system decides when to stop and which output to select—including multi-aspect scoring for tasks where quality is non-monotonic across iterations.
Fifth, the task-specific instantiations, to illustrate how the same abstract algorithm maps onto diverse domains (dialogue, code, math, sentiment, acronyms, constrained generation) through different prompt designs and feedback rubrics.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a method paper whose core idea is that a single LLM, prompted with carefully designed few-shot examples, can serve as its own critic and revisor across diverse tasks, producing better outputs through iterative self-feedback without any training or external supervision.

Formal Algorithm: Equations 1–4

SELF-REFINE is defined by three generation steps that alternate in a loop. Each step uses the same underlying model M but a different prompt, and each step serves a distinct computational role in the refinement process.

Initial Generation (Equation 1)

$y_0 = M(p_{\text{gen}} \| x)$

where $M$ is the language model, $p_{\text{gen}}$ is the task-specific few-shot prompt for initial generation, $x$ is the input (e.g., a dialogue context, a piece of code, a review to reverse), and $\|$ denotes string concatenation (the prompt and input are concatenated into a single text sequence fed to the model).

What it computes: the model generates a first-draft output conditioned on the task prompt and input. The prompt $p_{\text{gen}}$ contains few-shot examples of input-output pairs $\langle x^{(k)}, y^{(k)} \rangle$ that demonstrate what a successful output for this task looks like. The model samples from its conditional distribution to produce $y_0$ , which serves as the starting point for all subsequent refinement.

Why this form: this is exactly the standard few-shot prompting baseline—it is the direct generation that SELF-REFINE aims to improve upon. The paper deliberately initializes with standard generation (rather than, say, a random or empty output) to ensure that any observed gains are attributable to the feedback-and-refine loop, not to a better starting point. The initial output is generated once and then passed into the refinement cycle; it is not regenerated.

FEEDBACK Step (Equation 2)

$fb_t = M(p_{\text{fb}} \| x \| y_t)$

where $fb_t$ is the feedback generated at iteration $t$ , $p_{\text{fb}}$ is the task-specific feedback prompt, $x$ is the original input, and $y_t$ is the current output (the initial output at $t=0$ , or the refined output from the previous iteration for $t > 0$ ).

What it computes: the model examines its own output $y_t$ in the context of the original input $x$ , and produces a natural language diagnosis of what is wrong or suboptimal, along with suggestions for improvement. The feedback is structured and multi-aspect—for dialogue, it scores along 10 dimensions (relevance, informativeness, interestingness, consistency, helpfulness, engagement, specificity, safety, user understanding, fluency) on a 3-point scale each. For acronym generation, it scores along 5 dimensions (ease of pronunciation, ease of spelling, relation to title, positive connotation, well-knownness) on a 5-point scale. For code optimization, it identifies specific inefficiencies (e.g., "This code is slow because it is using a brute force approach") and suggests concrete algorithmic improvements (e.g., "use the formula (n*(n+1))/2").

Why this form: the model sees its own output $y_t$ rather than being asked to generate feedback "blind"—this is what makes the feedback self-feedback (the model critiques its own work, not an external reference). The prompt $p_{\text{fb}}$ contains few-shot examples of input-output-feedback triples $\langle x^{(k)}, y^{(k)}, fb^{(k)} \rangle$ . These examples are carefully designed to demonstrate actionable and specific feedback: they identify concrete phrases or structural issues, explain why they are problematic, and suggest what to do instead. The paper argues that this specificity is what distinguishes useful feedback from generic statements like "improve the code"—Table 2 shows that replacing actionable feedback with generic feedback substantially degrades performance across tasks (e.g., Code Optimization drops from 27.5 to 26.0, Sentiment Reversal drops from 43.2 to 31.2).

REFINE Step (Equations 3–4)

The refinement step has two formulations: a simplified version (Equation 3) and the actual implementation with full history retention (Equation 4).

$y_{t+1} = M(p_{\text{refine}} \| x \| y_t \| fb_t) \quad \text{(Equation 3)}$

$y_{t+1} = M(p_{\text{refine}} \| x \| y_0 \| fb_0 \| ... \| y_t \| fb_t) \quad \text{(Equation 4)}$

where $p_{\text{refine}}$ is the task-specific refinement prompt, and the key difference between the two equations is that Equation 4 appends the entire sequence of past outputs and feedback ( $y_0, fb_0, y_1, fb_1, ..., y_t, fb_t$ ) rather than only the most recent output and feedback.

What it computes: the model generates an improved version of the output, conditioned on the original input, the most recent output, the most recent feedback (Equation 3), and—crucially—all previous outputs and feedback (Equation 4). The refinement prompt $p_{\text{refine}}$ contains few-shot examples of input-output-feedback-refined quadruples $\langle x^{(k)}, y^{(k)}_t, fb^{(k)}_t, y^{(k)}_{t+1} \rangle$ that demonstrate how to incorporate feedback into an improved draft.

Why the full-history form (Equation 4): The paper argues that retaining all past outputs and their feedback in context serves a specific purpose:

"Intuitively, this allows the model to learn from past mistakes and avoid repeating them."

Without history retention, the model at iteration $t+1$ only knows about the immediately preceding output $y_t$ and feedback $fb_t$ . It might "fix" a problem in a way that reintroduces an earlier problem that had already been corrected. With full history, the model sees the trajectory of changes and can ensure monotonic improvement across multiple quality dimensions. The paper does not ablate this choice directly, but the design mirrors how humans revise—we look at previous drafts to track what we have already tried and what worked or did not.

Relationship between the three prompts: The prompts $p_{\text{gen}}$ , $p_{\text{fb}}$ , and $p_{\text{refine}}$ are independent few-shot prompts—they are not concatenated or merged. The model is called separately with each prompt at each step of the loop: once with $p_{\text{gen}}$ to initialize, then alternating between $p_{\text{fb}}$ and $p_{\text{refine}}$ for feedback and refinement. Each call is a standard LLM generation (temperature 0.7, greedy decoding as noted in Section 3.1). The prompts share underlying task understanding through their few-shot examples but serve different functions: $p_{\text{gen}}$ teaches "what a good output looks like," $p_{\text{fb}}$ teaches "how to diagnose problems," and $p_{\text{refine}}$ teaches "how to fix problems based on a diagnosis."

Prompt Design and Few-Shot Example Construction

The prompts are the core intellectual contribution of SELF-REFINE—they encode the knowledge of how to give feedback and how to refine. The paper does not use a generic "give feedback" instruction; instead, for each task, the authors manually construct few-shot examples that instantiate the following principles.

Few-shot example structure by role:

$p_{\text{gen}}$ : Contains $\langle x^{(k)}, y^{(k)} \rangle$ pairs—input and a good output. For example, for acronym generation (Figure 16, Appendix S), the prompt shows 11 title-acronym pairs like "Bidirectional Encoder Representations from Transformers → BERT" and "Sequence to Sequence Learning with Neural Networks → Seq2Seq." The model learns the output format and quality standard from these demonstrations. The paper uses 3-11 examples depending on the task (e.g., 3 for dialogue, 11 for acronym generation).
$p_{\text{fb}}$ : Contains $\langle x^{(k)}, y^{(k)}, fb^{(k)} \rangle$ triples. Critically, the feedback examples are designed to be multi-aspect (scoring along several dimensions with separate numerical scores), actionable (containing concrete suggestions for what to change), and specific (identifying particular words, phrases, or structural elements that are problematic). Figure 28 (Appendix S) shows a dialogue feedback example that scores the response along 10 dimensions with individual 3-point scales and a total score, then provides specific critiques like "The response does not directly address the user's question about why kebabs are special. 1/3" and "The response is not specific and does not provide any details or examples. 1/3." The paper constructs 2-6 such triples per task.
$p_{\text{refine}}$ : Contains $\langle x^{(k)}, y^{(k)}_t, fb^{(k)}_t, y^{(k)}_{t+1} \rangle$ quadruples that demonstrate improvement trajectories. These show the model a "before" output (with identified weaknesses), the feedback that diagnoses those weaknesses, and an "after" output that fixes them. Figure 29 (Appendix S) shows a dialogue refinement example where a response scoring 17/30 is revised to 30/30 by addressing each feedback point: the original response "That's just the way it is" becomes an engaging, specific, informative response about kebabs and robot machinery.

The rubric-based multi-aspect scoring design: A particularly important design choice is the use of explicit numerical rubrics in feedback prompts. For dialogue (Section M.1, Appendix M), each response is scored on a 3-point scale across 10 fine-grained dimensions (Relevant, Informative, Interesting, Consistent, Helpful, Engaging, Specific, Safe, User understanding, Fluent), producing a "/30" total. For acronyms (Appendix Q), each acronym is scored on a 5-point scale across 5 dimensions (Ease of pronunciation, Ease of spelling, Relation to title, Positive connotation, Well-known), producing a "/25" total. For code optimization (Appendix N), the feedback is free-text rather than rubric-based but still identifies specific inefficiencies and suggests algorithmic alternatives.

Why rubrics rather than free-text alone: The paper implicitly argues that multi-aspect scoring provides two benefits. First, it forces the model to evaluate each quality dimension separately rather than producing a holistic "looks good" or "needs work" judgment—this prevents the model from missing specific weaknesses that are averaged away in an overall assessment. Second, as noted in Section 4, for tasks where quality improvements can be non-monotonic across dimensions (e.g., Acronym Generation where a revision might improve pronunciation but worsen relation to title), the numerical scores serve as a selection mechanism: SELF-REFINE selects the output with the highest aggregate score across all iterations, not necessarily the final iteration's output. Table 10 in Appendix H.1 illustrates this: iteration 2 produces acronym "TACC-SIM" scoring 17/25, iteration 3 produces "TACCSF" scoring 12/25 (worse), and iteration 4 produces "TACC-SIMF" scoring 17/25—the algorithm correctly selects the best-scoring iteration rather than blindly taking the last.

Prompt construction effort: The paper is transparent about the human effort involved. The authors manually wrote the few-shot examples for each task, including writing flawed outputs, multi-aspect feedback with scores, and improved outputs. For tasks using existing datasets, they adapted existing data: for Code Optimization, they used slow/fast program pairs and explanations from Madaan et al. (2023); for Math Reasoning, they selected problems where CODEX fails with PaL-style prompts and manually wrote corrections; for Constrained Generation, they created variants with deliberately missing concepts or incoherent outputs. This upfront human effort per task is a one-time cost—once the prompts are constructed, SELF-REFINE can be applied to any new input in that domain without further human involvement.

Feedback Quality Requirements: Actionable and Specific

The paper does not merely assert that feedback should be "good"—it defines two precise properties and provides empirical evidence for their importance through a controlled ablation.

Actionable feedback is defined in Section 2:

"By 'actionable', we mean the feedback should contain a concrete action that would likely improve the output."

The example given is the code optimization feedback: "This code is slow as it uses a for loop which is brute force. A better approach is to use the formula ... (n*(n+1))/2." This is actionable because it tells the model what to do (use the closed-form formula) rather than just that something is wrong (the code is slow). Contrast with generic feedback like "Improve the efficiency of the code," which provides no guidance on how to improve.

Specific feedback is defined in Section 2:

"By 'specific', we mean the feedback should identify concrete phrases in the output to change."

The feedback identifies the "for loop" as the problematic element. For dialogue, specific feedback would point to particular phrases (e.g., "the response does not provide any details or examples") rather than issuing a vague quality judgment.

Ablation evidence (Table 2): The importance of these properties is established through a three-way comparison:

Task	SELF-REFINE feedback	Generic feedback	No feedback
Code Optimization	27.5	26.0	24.8
Sentiment Reversal	43.2	31.2	0
Acronym Generation	56.4	54.0	48.0

All experiments in this table used ChatGPT (for Code Optimization and Sentiment Reversal) or GPT-3.5 (for Acronym Generation). The metrics are defined in Section 3.2: Code Optimization uses % of programs optimized; Sentiment Reversal and Acronym Generation use GPT-4-based preference evaluation (win rate vs. baseline).

What this establishes: The generic feedback condition replaces the specific, action-oriented feedback from SELF-REFINE with vague feedback that still acknowledges something needs improvement but does not specify what or how. In Code Optimization, the drop is modest (27.5 → 26.0), suggesting that for technical tasks, the model can partially self-diagnose even with vague cues. In Sentiment Reversal, the drop is dramatic (43.2 → 31.2), indicating that for subjective quality tasks, the model relies heavily on specific guidance about which phrases to change and how aggressively to shift sentiment. The "no feedback" condition removes the FEEDBACK step entirely—the model still generates multiple candidates iteratively but without explicit criticism to guide refinement. Performance degrades in all tasks, and Sentiment Reversal collapses completely (score of 0), suggesting that iterative regeneration without guided feedback does not converge toward the target.

Why this matters: Table 2 is the paper's primary evidence that the FEEDBACK step is not incidental—it is the mechanism that makes refinement directed rather than random. This finding distinguishes SELF-REFINE from approaches like Self-Correction (Welleck et al., 2022), which train refiners without explicit feedback generation. The paper argues this is why SELF-REFINE achieves 55.7% vs. Self-Correction's 45.9% on GSM8K with GPT-3 (Appendix B).

Qualitative analysis of feedback quality (Section 4): The paper manually analyzed 70 samples (35 successes, 35 failures) from Code Optimization and Math Reasoning. Key findings:

In successful cases, feedback was predominantly actionable: 61% of successes involved the refiner acting on accurate, useful feedback to make precise fixes. In 33% of successes, the refiner corrected issues despite partially incorrect feedback, suggesting some robustness to imperfect diagnosis.
In failure cases, the problem was almost always in the feedback, not the refinement: 33% of failures involved feedback that accurately identified that something was wrong but located the error incorrectly; 61% involved feedback that suggested an inappropriate fix. Only 6% of failures were due to the refiner incorrectly implementing good feedback.

This imbalance (feedback errors cause 94% of failures, refinement errors cause only 6%) is a critical insight: it means the bottleneck in SELF-REFINE is the model's ability to diagnose problems, not its ability to fix them once diagnosed. This has direct implications for when SELF-REFINE works and when it does not—as we see in Math Reasoning, where the base model cannot reliably identify errors in its own reasoning chains.

Iteration, Stopping Conditions, and Output Selection

Default iteration limit: The FEEDBACK-REFINE loop runs for a maximum of 4 iterations (Section 3.1):

"The FEEDBACK-REFINE iterations continue until the desired output quality or task-specific criterion is reached, up to a maximum of 4 iterations."

The choice of 4 iterations is not theoretically motivated; it is a practical default that balances improvement against compute cost (each iteration requires two LLM calls: one for feedback, one for refinement). The paper does not ablate this maximum, though Figure 4 (discussed below) provides indirect evidence that gains diminish after 2-3 iterations.

Model-generated stopping: The stopping condition stop(fb_t, t) can be based on either "a specified timestep $t$ " or by extracting "a stopping indicator (e.g. a scalar stop score) from the feedback." In practice, when the feedback includes multi-aspect scores, the model can be prompted to indicate when no further improvement is possible—for example, feedback stating "The review is already Very negative" followed by a maximum score signals that refinement is complete. The algorithm stops if the stopping condition is triggered or the maximum iteration count is reached.

Output selection strategy: SELF-REFINE does not simply return the last refinement $y_t$ . For tasks with multi-aspect numerical feedback, the system selects the output with the highest aggregate score across all iterations. This is described implicitly in Section 4:

"To counter this, SELF-REFINE generates numerical scores for different quality aspects, leading to a balanced evaluation and appropriate output selection."

And explicitly in Appendix H.1, Table 10, which demonstrates non-monotonic quality in Acronym Generation: the best acronym (scoring 17/25) appears at iterations 2 and 4, while iteration 3 produces a worse acronym (12/25). By selecting based on maximum score rather than taking the final output, SELF-REFINE avoids the degradation that can occur when one quality dimension is improved at the expense of others.

For tasks without multi-aspect scores: In Code Optimization and Math Reasoning, where feedback is free-text rather than rubric-based, the system either takes the last output or, in the Math Reasoning case where the paper uses Oracle feedback (Section H.1), is guided by external correctness information. The paper notes in Section 4 that "we observe that output quality increases monotonically with iterations" for Math Reasoning and Sentiment Reversal, implying the selection mechanism is less critical for these tasks.

Diminishing returns across iterations (Figure 4): The paper reports iteration-wise scores for three tasks (averaged over ChatGPT, GPT-3.5, and GPT-4):

Iteration	Code Opt.	Sentiment Rev.	Constrained Gen.
$y_0$	22.0	33.9	29.0
$y_1$	27.0	34.9	40.3
$y_2$	27.9	36.1	46.7
$y_3$	28.8	36.8	49.7

The right panel of Figure 4 breaks down the deltas: most gains occur in the first iteration ( $y_0 \rightarrow y_1$ : +5.0 for Code Opt., +1.0 for Sentiment Rev., +11.3 for Constrained Gen.). Subsequent iterations yield progressively smaller improvements ( $y_1 \rightarrow y_2$ : +0.9, +1.2, +6.4; $y_2 \rightarrow y_3$ : +0.9, +0.7, +3.0). This pattern empirically justifies the 4-iteration maximum: additional iterations would likely produce negligible improvement.

Why iteration helps: The paper argues that multiple iterations allow the model to address progressively finer-grained issues. The first iteration typically fixes the most glaring problems (missing concepts, incorrect sentiment, major inefficiencies). Subsequent iterations refine subtler aspects (specific word choices, nuanced engagement quality, minor algorithmic improvements). The full-history context (Equation 4) ensures that later iterations do not undo fixes applied earlier.

Task-Specific Instantiations

The paper demonstrates SELF-REFINE across 7 tasks, each requiring a different prompt design and feedback rubric. Here we detail the key instantiations to illustrate the method's flexibility.

Dialogue Response Generation (Appendix M): The FEEDBACK prompt uses a 10-dimension rubric scored 1-3 each, totaling /30. The dimensions are: Relevant, Informative, Interesting, Consistent, Helpful, Engaging, Specific, Safe, User understanding, Fluent. The prompt provides 6 in-context examples showing responses, their dimension-by-dimension scores, and explanations for each score. The REFINE prompt shows the same context-response-feedback followed by an improved response. The evaluation uses both GPT-4-based preference (automatic, reported in Table 1) and blind human A/B testing (150 examples, reported in Table 6 of Appendix C). Human judges preferred SELF-REFINE over direct generation 47.58% vs. 19.66% (32.76% ties). The paper notes that SELF-REFINE outputs were "more engaging and interesting and generally more elaborate."

Code Optimization (Appendix N): The task is to transform a slow, functionally correct program into a faster version while maintaining correctness. The INIT prompt provides pairs of slow/fast programs from the PIE dataset (Madaan et al., 2023). FEEDBACK generates free-text natural language explanations identifying specific inefficiencies (e.g., Figure 5 in the main paper shows feedback diagnosing six nested loops and suggesting dynamic programming). REFINE takes the slow code, the feedback, and the original input, and produces a more efficient implementation. The metric is "% optimized"—the percentage of programs where SELF-REFINE successfully produces a faster version. GPT-4+SELF-REFINE achieves 36.0% vs. base GPT-4's 27.3%. The paper also reports relative speedup: SELF-REFINE achieves 3.74× speedup on average vs. 3.09× for direct generation (Table 17, Appendix N), though this is on the subset where optimization was successful.

Math Reasoning (Appendix O): The task uses the GSM-8k dataset. INIT generates Python code solutions following the PaL format (Gao et al., 2022). FEEDBACK examines the code for logical errors, walking through the solution "step-by-step" and checking "if everything looks good." Figure 31 (Appendix S) shows a detailed example where FEEDBACK identifies that cup_cost = plate_cost is wrong because "The cost of a cup is $1200 less than the total cost of half a dozen plates." REFINE rewrites the code incorporating the correction. This task shows the smallest gains in the default setup: GPT-4+SELF-REFINE improves only 0.2% (92.9% → 93.1%). The paper diagnoses this in Section 3.3:

"The modest performance gains in Math Reasoning can be traced back to the inability to accurately identify whether there is any error. In math, errors can be nuanced and sometimes limited to a single line or incorrect operation. Besides, a consistent-looking reasoning chain can deceive LLMs to think that 'everything looks good' (e.g., ChatGPT feedback for 94% instances is 'everything looks good')."

This is a critical finding about the boundary conditions of self-feedback: when the model cannot reliably detect its own errors, the feedback cycle provides no benefit. The paper confirms this by introducing Oracle Feedback (Appendix H.1): if an external signal correctly identifies when the answer is wrong, SELF-REFINE improves substantially—GPT-3.5+SELF-REFINE with Oracle Feedback achieves 68.9% vs. 64.1% base.

Sentiment Reversal (Appendix P): The task is to rewrite a multi-sentence review to reverse its sentiment (positive→negative or vice versa) while maintaining fluency and making the reversed sentiment convincing. The FEEDBACK prompt explicitly demonstrates how to adjust sentiment intensity: it teaches the model to recognize that certain words are "extremely positive" or "very negative" and to adjust word choice accordingly. Figure 34 (Appendix S) shows feedback explaining: "This review is 'Very negative' because of extremely toxic phrases like 'crawled into a hole to rot' and 'terrible.' To make it 'Negative', we will tone down the extremely negative phrases." The multi-aspect feedback includes a "dramatic" or "intensity" dimension that guides how aggressively to shift sentiment. GPT-4+SELF-REFINE achieves 36.2% preference rate vs. 3.8% for base GPT-4 (↑32.4%), one of the largest absolute improvements across all tasks.

Constrained Generation (Appendix R): The paper introduces a harder variant of CommonGen (Lin et al., 2020) called "CommonGen-Hard," where the model must generate a coherent sentence incorporating 20-30 given concepts (compared to 3-5 in the original). The FEEDBACK prompt has two components: (1) "Concept Feedback"—which concepts from the required list are missing from the generated sentence; (2) "Commonsense Feedback"—whether the sentence makes logical sense. The paper argues this task benefits particularly from SELF-REFINE because "there are more opportunities to miss some of the concepts on the first attempt, and thus SELF-REFINE allows the model to fix these mistakes subsequently" (Section 3.3). GPT-4+SELF-REFINE achieves 45.0% coverage vs. 15.0% for base GPT-4 (↑30.0%).

Acronym Generation (Appendix Q): The task is to generate a good acronym for a given title (e.g., "Radio Detecting and Ranging" → "RADAR"). The FEEDBACK prompt uses a 5-dimension rubric scored 1-5 each, totaling /25: Ease of pronunciation, Ease of spelling, Relation to title, Positive connotation, Well-known. The prompt (Figures 16-18, Appendix S) demonstrates how acronyms evolve: an initial somewhat-flawed acronym receives dimension-by-dimension scores with explanations, and a refined acronym addresses the identified weaknesses. The non-monotonic nature of acronym quality across iterations (Table 10, Appendix H.1) motivates the score-based output selection mechanism.

Code Readability Improvement (Appendix L): This task is distinctive because there is no INIT step—the starting point is an existing piece of code that needs readability improvement. The evaluation uses three automatic metrics: Meaningful Variable Ratio (fraction of variable names that are semantically meaningful), Comment Per Line (average number of comment pieces per code line), and Function Units (number of modularized functional units, with higher being better for readability). Table 14 (Appendix L) shows SELF-REFINE at temperature 0.7 producing more meaningful variables, more comments, and more function units than human annotator rewrites on average, though the human comparison is on a 60-example subset and the metrics are imperfect proxies for readability.

Model Configuration and Inference Details

The paper uses consistent settings across experiments (Section 3.1):

Temperature: 0.7 for all setups, with greedy decoding. Temperature 0.7 introduces some randomness that may help explore diverse refinements in the FEEDBACK step, while the REFINE step benefits from deterministic or near-deterministic decoding to reliably follow instructions.
Prompt format: All three components (INIT, FEEDBACK, REFINE) are implemented as few-shot prompts, even for instruction-tuned models like ChatGPT and GPT-4:

"To make our evaluation consistent across different models, we implemented both FEEDBACK and REFINE as few-shot prompts even with models that respond well to instructions, such as ChatGPT and GPT-4."

This is a deliberate choice—using few-shot examples rather than zero-shot instructions ensures that the feedback format (multi-aspect, rubric-based, with numerical scores) is demonstrated explicitly, reducing format errors.

Base LLMs: GPT-3.5 (text-davinci-003), ChatGPT (gpt-3.5-turbo), GPT-4, and Codex (code-davinci-002) for code tasks. The paper uses these as black-box APIs without any fine-tuning or parameter access.
Prompts reuse: Where prior work provided prompts (Code Optimization from Madaan et al., 2023; Math Reasoning from Gao et al., 2022), the paper reused them directly for INIT. For other tasks, the authors created prompts from scratch, releasing them in full in Appendix S.

Why This Design: Summary of Architectural Choices

Same model for all roles reduces distribution mismatch. If a separate model provided feedback, its critiques might address issues the generating model cannot fix, or miss problems the generating model could have addressed. Using the same model ensures the feedback is aligned with the model's own capabilities and limitations.
Multi-aspect, numerically-scored feedback prevents the "everything looks good" failure mode observed in Math Reasoning and enables robust output selection when quality is non-monotonic.
Full history retention in context prevents the model from cycling between flawed states and provides a learning signal across iterations.
Few-shot prompting rather than instruction-following provides format enforcement and task-specific demonstrations that standardized instructions alone cannot match—especially for enforcing the structured rubric format.
No training, no parameters means SELF-REFINE is purely an inference-time strategy, deployable with any sufficiently capable LLM without infrastructure for fine-tuning or RL.

4. Key Insights and Innovations

Innovation 1: Self-Critique as a Prompting Discipline, Not a Learned Behavior

The most fundamental conceptual move in this paper is the reframing of self-critique from a capability to be trained into a model to a behavior to be elicited through prompt design. Prior work treated the ability to evaluate and improve one's own outputs as something that required either explicit training data (Schick et al., 2022b; Welleck et al., 2022) or reinforcement learning from external reward signals (Stiennon et al., 2020; Le et al., 2022a). The underlying assumption was that models do not naturally possess the metacognitive capacity to diagnose their own errors and that this capacity must be installed through parameter updates.

SELF-REFINE's innovation is demonstrating that this assumption is wrong—at least for sufficiently capable LLMs. The same model, without any fine-tuning, can serve as generator, critic, and revisor, provided it is given demonstrations of how to structure feedback and how to incorporate it. This is not a mechanism claim (the prompting algorithm is described in Section 3) but a capability claim: the metacognitive abilities that prior work sought to train into models through supervised learning or RL already exist latently in pretrained LLMs and can be activated through few-shot prompting alone.

The evidence for this reframing comes from two complementary results. First, the consistent gains across 6 of 7 tasks (Table 1), including on subjective, preference-based tasks where no ground-truth correctness signal exists—the model cannot be verifying against an answer key, so its feedback genuinely reflects its own evaluation capabilities. Second, and perhaps more telling, the negative result on Math Reasoning (GPT-4+SELF-REFINE: 92.9% → 93.1%, a negligible 0.2% gain). The paper's diagnosis is precisely the metacognitive boundary: the model cannot reliably detect errors in its mathematical reasoning chains, generating unhelpful feedback like "everything looks good" for 94% of instances. When an external Oracle signal corrects this (Appendix H.1), SELF-REFINE's refinement mechanism works (GPT-3.5 improves from 64.1% to 68.9%), confirming that the bottleneck is diagnostic, not remedial. This is a clean demonstration that the approach works when—and only when—the model's self-evaluation abilities are reliable, establishing a boundary condition that prior work had not empirically mapped.

This reframing has practical implications that extend beyond SELF-REFINE itself. It suggests that heavy investment in training feedback models or reward functions may be unnecessary for many applications where the base model already possesses sufficient self-evaluation capability. It also shifts the research challenge: rather than asking "how do we train a model to self-improve?" (the framing of Welleck et al., 2022 and related work), the question becomes "how do we design prompts that elicit the self-evaluation and self-improvement behaviors already present in the model?" This is a fundamentally different research agenda—one centered on prompt engineering and format design rather than on data collection and training.

A persistent debate in the LLM self-improvement literature concerns where the improvement bottleneck lies. Do models fail to produce better second drafts because they cannot recognize what is wrong with their first draft, or because they cannot effectively fix problems even when they are identified? The question matters because it determines where research effort should concentrate: on better diagnostic mechanisms or on better revision mechanisms.

SELF-REFINE provides the strongest empirical answer to this question available at its time of publication. The manual error analysis in Section 4 (35 successes, 35 failures across Code Optimization and Math Reasoning) reveals a stark asymmetry: 94% of failures are attributable to feedback errors (33% incorrect error localization + 61% inappropriate fix suggestions), while only 6% stem from the refiner incorrectly implementing correct feedback. In successful cases, the refiner demonstrated robustness to imperfect feedback, correctly fixing issues even when the diagnosis was partially wrong in 33% of cases. This imbalance—refinement works more reliably than diagnosis—is not an incidental finding; it is the central diagnostic insight of the paper.

The ablation in Table 2 provides convergent evidence through a completely different methodology. By degrading feedback quality systematically (SELF-REFINE feedback → generic feedback → no feedback), the paper shows that output quality tracks feedback quality closely, with the largest drops occurring on the tasks most dependent on precise guidance (Sentiment Reversal: 43.2 → 31.2 → 0). The fact that the model can still produce some improvement with generic feedback (Code Optimization: 27.5 → 26.0 → 24.8) but collapses without any feedback at all confirms that the feedback signal is the primary driver of improvement.

This finding is significant because it contradicts the implicit assumption behind approaches that train refiners without explicit feedback generation (notably Welleck et al., 2022). If refinement capability were the bottleneck, training a corrector model without feedback would be the right strategy—the model would learn to map flawed outputs to improved ones directly. But SELF-REFINE shows the opposite: when given good feedback, the same base model refines effectively; when feedback is poor, refinement fails regardless of the model's correction ability. This implies that the value of explicit, structured feedback is not just pedagogical for the model—it is the primary mechanism enabling improvement, and omitting it (as Self-Correction does) leaves performance on the table (empirically confirmed by SELF-REFINE's 55.7% vs. Self-Correction's 45.9% on GSM8K with the same base model).

The practical corollary is that efforts to improve LLM output quality at inference time should prioritize feedback quality over refinement sophistication. Better prompting for feedback generation, more structured feedback formats (the multi-aspect rubrics used in SELF-REFINE), and potentially external verification signals for domains where self-diagnosis is unreliable (as the Math Reasoning Oracle experiment demonstrates) are likely to yield larger gains than developing more sophisticated refinement mechanisms. This reorients the research direction in a specific, empirically-grounded way.

Innovation 3: Multi-Aspect Structured Feedback as a General-Purpose Scaffold for Quality Improvement

A subtler but equally important contribution is SELF-REFINE's demonstration that multi-aspect, rubric-based feedback with explicit numerical scores serves as a domain-general scaffold for output improvement. This is not an architectural innovation—it is a format innovation in prompt design that proves effective across radically different tasks: dialogue response (10 dimensions on a 3-point scale), acronym generation (5 dimensions on a 5-point scale), code optimization (free-text but still structured around specific inefficiencies), and constrained generation (split into concept coverage and commonsense coherence).

Before SELF-REFINE, the dominant paradigm for test-time output improvement relied on either scalar reward signals (RL-based methods: a single number summarizing quality) or holistic natural language critiques (Reflexion, Re³: paragraph-level feedback without dimensional decomposition). The problem with scalar rewards is that they provide no information about which aspects of an output are deficient—a low score tells the model something is wrong but not what or where. The problem with holistic critiques is that they can average over strengths and weaknesses, missing specific, fixable issues (the "everything looks good" problem that plagues Math Reasoning).

SELF-REFINE's multi-aspect rubric design solves both problems simultaneously. By requiring the model to evaluate each dimension separately and assign a numerical score, the feedback format forces disaggregation—a dimension scoring 1/3 stands out even if the overall score is acceptable. This makes the feedback more likely to identify concrete, fixable issues. The numerical scores then serve a secondary function: they enable principled output selection when quality fluctuates across iterations (as demonstrated for Acronym Generation in Table 10, Appendix H.1, where the algorithm selects the highest-scoring iteration rather than blindly taking the final output).

The generality of this approach is demonstrated by its successful application to tasks as different as dialogue (subjective quality, no ground truth), code optimization (objective efficiency, verifiable), and sentiment reversal (subjective intensity, multiply-realizable). In each case, the rubric design captures the relevant quality dimensions for that domain, but the underlying mechanism—multi-aspect scoring → targeted refinement → score-based selection—remains invariant. This is a domain-general framework for structuring LLM self-improvement that can be instantiated for any task where quality can be decomposed into evaluable dimensions, and it requires no task-specific training—only the one-time human effort of designing the rubric and writing few-shot examples.

The practical significance is that this framework lowers the barrier to deploying self-improvement in new domains. Rather than collecting training data and fine-tuning a critic model, a practitioner identifies relevant quality dimensions, writes a handful of demonstration examples showing dimension-by-dimension scoring, and immediately gets a working self-improvement loop. The paper's demonstration across 7 diverse tasks provides the existence proof that this approach is not domain-specific—it is a general pattern that transfers across task types, modalities (text and code), and model families (GPT-3.5, ChatGPT, GPT-4, Codex).

Innovation 4: Difficulty-Dependence of Self-Improvement as a Diagnostic Framework

Although the paper does not use the terminology of "difficulty," SELF-REFINE's results collectively establish a capability-gated model of self-improvement that reconciles conflicting prior findings in the literature. This is not presented as a formal framework—it is an emergent insight from the pattern of results across tasks—but it constitutes one of the paper's most intellectually significant contributions because it explains when self-refinement works and when it does not in a principled way.

The pattern is clearest in the contrast between three task types. High-capability tasks (where the base model already performs well and just needs refinement): Dialogue Response Generation shows massive gains (GPT-4: 25.4% → 74.6%, +49.2%); Sentiment Reversal shows similar magnitude (GPT-4: 3.8% → 36.2%, +32.4%). In both cases, the base model can produce reasonable outputs and can articulate what makes one output better than another—the metacognitive ability is intact. Boundary-capability tasks (where the base model can sometimes produce good outputs but cannot reliably evaluate them): Math Reasoning shows negligible gains with self-feedback (GPT-4: +0.2%) but substantial gains with Oracle feedback (GPT-3.5: +4.8%), indicating that the model can refine when told that something is wrong but cannot reliably determine whether something is wrong on its own. Capability-dependent improvement: the Vicuna-13B experiment (Appendix G) shows that a weaker base model fails at SELF-REFINE entirely—it cannot follow the feedback format, cannot generate useful critiques, and cannot incorporate feedback when externally provided.

This pattern provides a unified explanation for why prior work reached contradictory conclusions about LLM self-improvement. Studies finding that LLMs "cannot self-correct reasoning" (Huang et al., 2023) were testing on tasks near the metacognitive boundary, where the model's diagnostic ability is unreliable. Studies finding that self-refinement helps (Madaan et al., 2023; Yang et al., 2022) were testing on tasks where the model's evaluative capabilities are stronger. SELF-REFINE's contribution is not just anecdotally observing this variation but providing systematic evidence across a range of tasks and models, and crucially, experimentally disentangling the diagnostic and remedial components through the Oracle feedback experiment on Math Reasoning.

The practical implication is a decision rule for practitioners: SELF-REFINE-like approaches will work when the base model can reliably generate useful feedback—which roughly corresponds to tasks where it can articulate quality criteria and distinguish good outputs from bad ones. When the model cannot do this (as in Math Reasoning, or for any task with a weaker model like Vicuna-13B), external verification signals or stronger base models are necessary. This transforms the question from "does self-refinement work?" (which has no single answer) to "under what conditions, and for which models, does self-refinement work?"—a more productive framing that the paper's results partially answer.

5. Experimental Analysis

Evaluation Methodology

Dataset. SELF-REFINE is evaluated on 7 diverse tasks spanning natural language and code generation: Dialogue Response Generation (FED dataset; Mehri and Eskenazi, 2020; 372 conversations), Code Optimization (PIE dataset; Madaan et al., 2023; 1,000 programs), Code Readability Improvement (CodeNet; Puri et al., 2021; 300 programs, with a 60-example human-annotated subset), Math Reasoning (GSM-8k; Cobbe et al., 2021; 1,319 questions), Sentiment Reversal (1000 review passages from Zhang et al., 2015), and two newly introduced tasks—Acronym Generation (250 acronyms sourced from Wikipedia and manually pruned) and Constrained Generation ("CommonGen-Hard," 200 samples extending Lin et al., 2020 with 20–30 keyword constraints instead of 3–5). For Math Reasoning and Code Optimization, the paper uses the standard train/test splits from prior work (the MATH benchmark's test split of 500 for search experiments is not used here—SELF-REFINE uses GSM-8k's 1,319 examples); for tasks requiring human evaluation, subsets of 100–150 examples are randomly sampled. Dataset statistics and examples are provided in Table 4 (Appendix A).
Base model(s). The paper uses three main model families across all tasks: GPT-3.5 (text-davinci-003), ChatGPT (gpt-3.5-turbo), and GPT-4, with Codex (code-davinci-002) additionally evaluated on code-based tasks. These models were chosen because they represent state-of-the-art across the evaluated tasks—"in all tasks, either GPT-3.5 or GPT-4 is the previous state-of-the-art" (Section 3.1)—and because their instruction-following and few-shot learning capabilities are prerequisites for SELF-REFINE's feedback-and-refinement loop. A weaker model, Vicuna-13B (Chiang et al., 2023), is tested separately to probe the lower bound of required capability (Appendix G). All models are used as black-box APIs without fine-tuning or parameter access. The paper uses greedy decoding with temperature 0.7 for all setups (Section 3.1).
Metrics. The paper reports three types of metrics (Section 3.2): (1) Task-specific automated metrics: Math Reasoning uses % solve rate (exact match of final answer); Code Optimization uses % of programs successfully optimized (producing a faster, correct version) and relative speedup for optimized programs; Constrained Generation uses coverage % (fraction of required keywords appearing in the generated sentence); Code Readability uses Meaningful Variable Ratio (fraction of distinct variables with semantically meaningful names, computed via few-shot prompted LLM), Comments Per Line, and Function Units (number of modularized code units). (2) Human-pref: For Dialogue Response Generation, Sentiment Reversal, Code Readability, and Acronym Generation, blind human A/B evaluation is conducted on subsets of outputs, with annotators selecting which of two outputs (SELF-REFINE vs. baseline) better aligns with the task instruction. Human evaluation is performed by the authors with 150 examples per task; judges are blind to which method generated each output (Appendix C). Preference rates reported as the percentage of times SELF-REFINE is selected over the baseline, with a "both equal" option. (3) GPT-4-pref: GPT-4 is used as an automated proxy for human preference, with high reported correlation to human judgments—82% for Sentiment Reversal, 68% for Acronym Generation, and 71% for Dialogue Response Generation (Section 3.2). This serves as the primary automatic metric for tasks without objective correctness signals, reducing the cost and latency of human evaluation for the main results in Table 1.
Baselines. The primary baseline in all experiments is Direct generation (called "Base" or "INIT"): the same LLM prompted with p_gen to produce a single output without any feedback or refinement. This is the standard few-shot prompting approach against which SELF-REFINE's iterative improvements are measured. For the multiple-sample comparison (Section 4, Appendix H, Figure 6), an additional MULTI baseline generates k = 4 independent samples from the base model without feedback or refinement; SELF-REFINE's single best output (selected via score maximization) is compared against all k initial outputs in a 1-vs-k evaluation. For Math Reasoning, the paper compares against several prior methods: Chain-of-Thought with Codex (Wei et al., 2022; 65.6%), PaL with various models (Gao et al., 2022; GPT-4: 93.3%), and Self-Correction with GPT-3 (Welleck et al., 2022; 45.9%), with SELF-REFINE's few-shot GPT-3 version achieving 55.7% (Table 7, Appendix F). For Code Optimization, baselines include the PIE fine-tuned models (PIE-2B, PIE-16B, best@32: 26.6%), SCALENE (Berger et al., 2022; best@32: 19.6%), and human references (38.2% optimized; Table 8, Appendix F). For Code Readability, human annotator rewrites on a 60-example subset serve as an upper-bound comparison (Table 14, Appendix L).
Generation budget / compute accounting. SELF-REFINE's compute cost is measured in terms of LLM API calls: each iteration requires two calls (one for FEEDBACK, one for REFINE), plus one initial call for INIT. The paper runs a maximum of 4 iterations, corresponding to at most 1 + 4×2 = 9 total LLM calls per input. This cost is compared against the baselines' single call (Direct) or k calls (MULTI, where k = 4). The paper does not report wall-clock time or FLOPs, and does not account for the cost of generating the few-shot prompts themselves (which are pre-written and shared across all inputs). For the multiple-sample comparison (Section 4, Appendix H), SELF-REFINE's cost (up to 9 calls) is compared against MULTI's cost (4 calls for generation), making SELF-REFINE approximately 2.25× more expensive in API calls, though the paper does not explicitly compute this ratio. The paper does not perform a FLOPs-matched comparison of the sort found in the prior example paper on test-time compute scaling—the compute analysis is limited to counting LLM calls and acknowledging that SELF-REFINE uses "at most 4 samples" versus baselines using "16 and 32 samples" (Table 8 caption, Appendix F).
Cross-validation / statistical protocol. Wilson confidence intervals at 95% confidence are reported for all main results in Table 13 (Appendix J), with statistically significant gains over the base model marked with an asterisk. The paper reports that "nearly all of GPT-4 gains are statistically significant, ChatGPT gains are significant for 4 out of 7 datasets, and GPT-3.5 gains are significant for 3 out of 7 datasets" (Appendix J). Human A/B evaluations are conducted on 150 examples per task (Table 6, Appendix C); GPT-4-preference evaluations are run on the full test sets. The paper does not use cross-validation for strategy selection (unlike the compute-optimal test-time scaling paper), since the SELF-REFINE algorithm has no learned parameters or hyperparameter selection at test time—the prompts and maximum iterations are fixed across all examples within a task. The paper does not report multiple random seeds or quantify variance due to the temperature 0.7 sampling.

Main Quantitative Results

Aggregate Performance Across All Tasks and Models

Table 1 presents the headline results: SELF-REFINE improves over direct generation across all three base models (GPT-3.5, ChatGPT, GPT-4) on 6 of 7 tasks, with the sole exception of Math Reasoning, which shows near-zero improvement (GPT-4: 92.9% → 93.1%, +0.2%; GPT-3.5: 64.1% → 64.1%, 0%). The magnitude of improvement varies substantially by task and model. For GPT-4, the largest absolute gains appear on preference-based tasks: Dialogue Response Generation improves from 25.4% to 74.6% (+49.2%), Sentiment Reversal from 3.8% to 36.2% (+32.4%), Acronym Generation from 30.4% to 56.0% (+25.6%), and Constrained Generation from 15.0% to 45.0% (+30.0%). Smaller but still substantial gains appear on code tasks: Code Optimization from 27.3% to 36.0% (+8.7%), Code Readability from 27.4% to 56.2% (+28.8%).

The relative ordering across base models is consistent: GPT-4+SELF-REFINE generally outperforms ChatGPT+SELF-REFINE, which generally outperforms GPT-3.5+SELF-REFINE, even on tasks where the base GPT-4 underperforms base GPT-3.5 (e.g., Dialogue Response base: GPT-4 25.4% vs. GPT-3.5 36.4%, but GPT-4+SELF-REFINE reaches 74.6% vs. GPT-3.5+SELF-REFINE's 63.6%). This pattern leads the authors to conclude that "SELF-REFINE allows stronger models (such as GPT-4) to unlock their full potential, even in cases where this potential is not expressed in the standard, single-pass, output generation" (Section 3.3).

Difficulty-Dependent Gains: Constrained Generation and Preference Tasks Lead

Section 3.3 provides per-task analysis explaining the variation in improvement magnitude. Constrained Generation (20–30 keywords) shows the largest relative gains because "there are more opportunities to miss some of the concepts on the first attempt, and thus SELF-REFINE allows the model to fix these mistakes subsequently." The FEEDBACK step explicitly enumerates missing concepts, providing a clear, verifiable checklist for refinement. Dialogue Response Generation and Sentiment Reversal show large gains because they are open-ended tasks where quality is multi-dimensional and base outputs are often generic or fail to meet nuanced criteria; the multi-aspect feedback rubric systematically identifies specific weaknesses (e.g., "not engaging," "not specific," "lacks user understanding") that the refiner can address.

The Math Reasoning near-failure (0–0.2% improvement) is attributed to feedback quality breakdown: "ChatGPT feedback for 94% instances is 'everything looks good'" (Section 3.3). The base model cannot reliably detect errors in its own reasoning chains, so the FEEDBACK step provides no useful guidance and the REFINE step makes no meaningful changes. This is the paper's clearest evidence that feedback quality, not refinement capability, is the bottleneck.

Iteration-by-Iteration Improvement Trajectories (Figure 4)

Figure 4 reports averaged scores (over ChatGPT, GPT-3.5, and GPT-4) at each iteration for three tasks, demonstrating diminishing returns:

Code Optimization: 22.0 (y₀) → 27.0 (y₁) → 27.9 (y₂) → 28.8 (y₃). Delta: y₀→y₁: +5.0; y₁→y₂: +0.9; y₂→y₃: +0.9.
Sentiment Reversal: 33.9 (y₀) → 34.9 (y₁) → 36.1 (y₂) → 36.8 (y₃). Delta: y₀→y₁: +1.0; y₁→y₂: +1.2; y₂→y₃: +0.7.
Constrained Generation: 29.0 (y₀) → 40.3 (y₁) → 46.7 (y₂) → 49.7 (y₃). Delta: y₀→y₁: +11.3; y₁→y₂: +6.4; y₂→y₃: +3.0.

The right panel of Figure 4 visualizes these deltas, showing that for Code Optimization and Constrained Generation, the majority of improvement occurs in the first iteration. The paper notes this as evidence for diminishing returns but also for the value of multiple iterations: "having multiple FEEDBACK-REFINE iterations significantly enhances the quality of the output, although the marginal improvement naturally decreases with more iterations" (Section 4). The paper does not report whether gains beyond iteration 3 would continue or asymptote to zero, as there is no experiment extending beyond the maximum budget.

To distinguish whether SELF-REFINE's gains come from iterative refinement or simply from generating more outputs (any one of which might be better), the paper compares SELF-REFINE against a MULTI baseline that generates k = 4 independent samples from the same base model without feedback. In a blind 1-vs-k human evaluation on Sentiment Reversal and Acronym Generation (Figure 6, Appendix H):

Sentiment Reversal (ChatGPT): SELF-REFINE wins in 51.1% of comparisons, MULTI wins in 27.2%, and 35.6% are ties. This means SELF-REFINE's single best output is preferred over all 4 independent samples more than half the time, despite having a maximum of 9 LLM calls vs. MULTI's 4.
Acronym Generation (ChatGPT): SELF-REFINE wins 53.82%, MULTI wins 11.4%, ties 45.4%.

The paper argues this "shows the importance of refinement according to feedback over the alternative of just generating multiple initial outputs" (Section 4). However, it is worth noting that SELF-REFINE here uses up to 9× the compute of a single generation, while MULTI uses exactly 4×—so SELF-REFINE is roughly 2.25× more expensive than MULTI in API calls. The paper does not normalize for compute or report a cost-adjusted comparison.

Comparison to Prior State-of-the-Art (Tables 7–8, Appendix F)

Math Reasoning (Table 7): SELF-REFINE with GPT-4 achieves 94.5% solve rate on GSM-8k, outperforming all prior methods including PaL with GPT-4 (93.3%; Gao et al., 2022) and Self-Correction with GPT-3 (45.9%; Welleck et al., 2022). SELF-REFINE with GPT-3 achieves 55.7%, a 9.8-point improvement over Self-Correction's 45.9% using the same base model. However, the GPT-4 base without SELF-REFINE already scores 92.9% (Table 1), so the +1.6-point gain from SELF-REFINE is small in absolute terms—consistent with the feedback quality bottleneck on math.

Code Optimization (Table 8): SELF-REFINE with GPT-4 achieves 36.0% programs optimized, approaching the human reference level of 38.2% (Puri et al., 2021) and surpassing the best fine-tuned model PIE-Few-shot at best@32 (35.2% with 32 samples, vs. SELF-REFINE's "4 samples at most"). SELF-REFINE with GPT-3.5 achieves 23.0%, outperforming SCALENE's best@32 (19.6%) and the fine-tuned PIE-16B at best@32 (26.6%). The caption notes that SELF-REFINE "achieves superior performance while using only 4 samples at most, significantly fewer than the 16 and 32 samples employed by other models." However, this comparison conflates different types of "samples": PIE's best@k refers to generating k candidate programs and selecting the best, while SELF-REFINE's "4 samples" refers to 4 refinement iterations (8 LLM calls)—these are not directly comparable compute budgets.

Human Evaluation Results (Table 6, Appendix C; Table 15, Appendix M)

Blind human A/B evaluation on 150 examples per task (Table 6) shows SELF-REFINE preferred over direct generation by margins of 75.00% vs. 21.43% (Sentiment Transfer), 44.59% vs. 12.16% (Acronym Generation), and 47.58% vs. 19.66% (Dialogue Response Generation), with substantial tie rates (3.57%, 43.24%, 32.76%, respectively). For Dialogue Response Generation (Table 15), human evaluation across models shows a consistent pattern: GPT-4+SELF-REFINE wins 54.0% vs. INIT's 16.0% (30.0% ties); ChatGPT+SELF-REFINE wins 48.0% vs. INIT's 18.0% (50.0% ties); GPT-3.5+SELF-REFINE wins 36.0% vs. INIT's 23.0% (41.0% ties). The high tie rates, particularly for acronym generation (43.24%) and dialogue (32.76–50.0%), indicate that in a substantial fraction of cases, SELF-REFINE and direct generation produce outputs of comparable quality—the improvement is reliable but not universal.

Vicuna-13B Failure (Appendix G)

SELF-REFINE fails entirely when instantiated with Vicuna-13B, a weaker open-source model. The paper reports that Vicuna-13B "was not able to consistently generate the feedback in the required format" and "often failed to adhere to the prompts for refinement. Instead of refining its output, Vicuna-13B either repeated the same output or generated a hallucinated conversation, rendering the outputs less effective" (Section 4, Appendix G). An example in Appendix G shows Vicuna-13B generating empty feedback on a first attempt (causing a parsing error), then generating unhelpful feedback that copies from the prompt, and ultimately producing an output that reverts to the prompt's example text rather than the target sentiment. This establishes a clear lower bound: SELF-REFINE requires base models with sufficient few-shot or instruction-following capability to (a) format feedback correctly, (b) generate actionable critiques, and (c) incorporate feedback without hallucination. The paper hypothesizes this is because "Vicuna-13B was trained on conversations, it does not generalize as well as instruction-based models to test-time few-shot tasks" (Section 4).

Mixed-Refine: Weaker Generator + Stronger Refiner (Appendix G)

As an exploratory experiment, the paper tests a "Mixed-Refine" configuration where Vicuna-13B serves as INIT but ChatGPT serves as FEEDBACK and REFINE. On Math Reasoning, Vicuna-13B alone achieves 24.18% solve rate; with ChatGPT providing feedback and refinement, performance improves to 40.5%. This demonstrates that refinement by a stronger model can partially compensate for a weaker generator's limitations, but falls far short of the 75.0% that ChatGPT+SELF-REFINE achieves when using its own initial outputs (Table 1). The paper does not explore this configuration systematically but notes it "shows the promise of this approach."

Ablation Studies and Robustness Checks

Feedback quality: SELF-REFINE feedback vs. generic feedback vs. no feedback (Table 2, Section 4). This is the central ablation establishing that the specificity and actionability of feedback drive SELF-REFINE's gains. On Code Optimization (ChatGPT), SELF-REFINE's specific, actionable feedback achieves 27.5%; generic feedback ("Improve the efficiency of the code") drops to 26.0%; no feedback (iterative regeneration without explicit critique) drops further to 24.8%. On Sentiment Reversal, the degradation is steeper: 43.2% → 31.2% → 0% (complete failure without feedback). On Acronym Generation (GPT-3.5), the pattern holds: 56.4% → 54.0% → 48.0%. The complete collapse on Sentiment Reversal without feedback indicates that iterative refinement without guided critique does not converge to the target sentiment—the model lacks the signal to determine whether each revision is moving toward or away from the desired polarity.
Oracle feedback on Math Reasoning (Table 9, Appendix H.1). To test whether the Math Reasoning bottleneck is diagnostic or remedial, the paper introduces Oracle Feedback: an external correctness signal that tells the model whether its current answer is correct, triggering refinement only when the answer is wrong. GPT-3.5+SELF-REFINE with Oracle Feedback achieves 68.9% vs. 64.1% base (+4.8%), a substantial improvement from the +0% in the standard SELF-REFINE setup. GPT-4+SELF-REFINE with Oracle Feedback reaches 93.8% vs. 92.9% (+0.7%), still modest but larger than the +0.2% without Oracle. This ablation confirms that the refinement mechanism works for math when the model is told that an error exists—the failure in the standard setup is specifically due to the model's inability to self-diagnose mathematical errors in its own reasoning.
Feedback vs. refinement error attribution (Section 4, qualitative). Manual analysis of 70 samples (35 successes, 35 failures) from Code Optimization and Math Reasoning reveals the bottleneck distribution: in failure cases, 33% are due to feedback inaccurately locating the error, 61% are due to feedback suggesting an inappropriate fix, and only 6% are due to the refiner incorrectly implementing good feedback. In successful cases, 61% involved the refiner acting on accurate, useful feedback to make precise fixes, while 33% involved the refiner correcting issues despite partially incorrect feedback. This asymmetry establishes that "the majority of issues were due to erroneous feedback rather than faulty refinements" (Section 4), directly supporting the paper's claim that feedback quality is the critical factor.
Iteration count and diminishing returns (Figure 4, Section 4). Averaged across three models and three tasks, the first iteration captures the majority of improvement: +5.0 for Code Optimization (out of +6.8 total by y₃), +1.0 for Sentiment Reversal (out of +2.9 total), +11.3 for Constrained Generation (out of +20.7 total). The second and third iterations contribute progressively less, consistent with the paper's choice of a 4-iteration maximum. The paper does not report a direct ablation comparing different maximum iteration counts (e.g., stopping at 1 vs. 2 vs. 4), so the optimal stopping point is not empirically established—the 4-iteration limit is a parameter choice, not an optimized finding.
Non-monotonic quality and score-based selection (Table 10, Appendix H.1; Section 4). On Acronym Generation, where multiple quality dimensions can trade off against each other, output quality fluctuates non-monotonically across iterations. Table 10 shows an example where iteration 2 achieves 17/25, iteration 3 degrades to 12/25, and iteration 4 recovers to 17/25. The paper notes that "SELF-REFINE generates numerical scores for different quality aspects, leading to a balanced evaluation and appropriate output selection"—the algorithm selects the highest-scoring iteration rather than the final one. Without this mechanism, the final output after 4 iterations would sometimes be worse than an earlier iteration. This is not presented as a formal ablation (the paper does not compare score-based selection vs. taking-the-last-output), but it is described as a design choice motivated by the observed non-monotonicity.
Dialogue Response Generation error analysis (Tables 11–12, Appendix H). A separate qualitative analysis of SELF-REFINE on Dialogue Response Generation identifies three feedback error types: incorrect feedback (occurring in 25% of analyzed failures—the feedback identifies a problem that does not exist or misdiagnoses the issue), generic feedback (30%—feedback lacks specificity), and incorrect scoring (10%—numerical scores do not reflect actual quality). On the refinement side, the model is robust to bad feedback in 60% of cases (ignoring incorrect or generic feedback and making appropriate improvements anyway), but it ignores good feedback 25% of the time and introduces new problems 20% of the time. This analysis underscores that while SELF-REFINE is robust, it is not infallible—roughly one-quarter of refinement opportunities are squandered due to the model not acting on useful feedback.
Temperature sensitivity for Code Readability (Figure 11, Appendix L). The FEEDBACK prompt for Code Readability is tested at two temperatures: T = 0.0 (greedy) and T = 0.7 (sampling). At T = 0.7, SELF-REFINE produces more meaningful variable names (0.700 vs. 0.628 at T = 0.0 by iteration 5) and more comments per line (0.25 vs. 0.12), while greedy decoding leads to more function modularization (1.41 function units vs. 1.33). This suggests that higher temperature in the FEEDBACK step (which generates the critique) leads to more diverse suggestions that favor naming and commenting improvements, while greedy critique focuses on structural refactoring. The paper does not systematically optimize temperature or ablate it across other tasks.
Multiple samples vs. refinement (Figure 6, Appendix H). As described in the main results, SELF-REFINE's outputs are preferred over all 4 independent samples from the MULTI baseline in 51.1% (Sentiment Reversal) and 53.82% (Acronym Generation) of evaluations, with high tie rates (35.6% and 45.4%). This is presented as an ablation showing that "refinement according to feedback" is more effective than "just generating multiple initial outputs" (Section 4), though the compute cost is not equalized (SELF-REFINE uses up to 9 calls vs. MULTI's 4).
Mixed-Refine: weaker base model + stronger refiner (Appendix G). Vicuna-13B as INIT with ChatGPT as FEEDBACK and REFINE improves Math Reasoning from 24.18% (Vicuna-13B alone) to 40.5%, demonstrating that cross-model refinement can partially compensate for a weaker generator's limitations, though performance remains far below ChatGPT's own SELF-REFINE performance (75.0%, Table 1). This is an exploratory result, not a systematic ablation, but it suggests that SELF-REFINE's benefits could be extended to weaker models by offloading the feedback and refinement steps to a stronger model.

Critical Assessment

Does SELF-REFINE genuinely demonstrate self-improvement, or does it just generate more tokens?

The paper's central claim is that SELF-REFINE improves outputs through iterative self-feedback and refinement, not merely through additional sampling. The evidence for this is the MULTI comparison (Figure 6, Appendix H): when given a budget of 4 independent samples without feedback, the base model's outputs are less preferred than SELF-REFINE's single best output, despite SELF-REFINE using more LLM calls (up to 9 vs. 4). This is consistent with the claim that refinement is doing something beyond random sampling. However, the comparison is not compute-normalized—SELF-REFINE's higher cost is not accounted for. A stricter test would give MULTI 9 independent samples (matching SELF-REFINE's maximum API call budget) and compare. The paper does not run this experiment, leaving open the possibility that simply sampling more outputs and selecting the best via majority voting or a verifier might match or exceed SELF-REFINE's performance at equivalent cost.

This ambiguity is particularly acute on tasks with objective correctness signals (Math Reasoning, Code Optimization), where generating more independent samples and selecting the best via execution-based verification (running the code, checking the math answer) is a strong baseline that the paper does not compare against. For Math Reasoning, majority voting over multiple samples is a standard approach (e.g., Wang et al., 2023) that the paper does not benchmark.

Does the paper demonstrate that SELF-REFINE works "without any supervised training data"?

Yes, in the narrow sense that no model parameters are updated. But the prompt construction itself requires substantial human effort: for each of the 7 tasks, the authors manually wrote few-shot examples including flawed outputs, multi-aspect feedback with numerical scores and explanations, and improved outputs that incorporate the feedback. For some tasks, this leveraged existing datasets (Code Optimization used slow/fast pairs from Madaan et al., 2023; Math Reasoning reused PaL prompts), but for others (Acronym Generation, Constrained Generation, Dialogue Response, Sentiment Reversal) the authors created examples from scratch. This human effort is a form of supervision—it is upfront prompt engineering rather than training data annotation, but it is task-specific human labor nonetheless. The claim "no supervised training data" is technically correct (no gradient updates, no dataset of (input, output, feedback) pairs collected at scale), but should be understood as "no model training," not "no human supervision at all."

The Vicuna-13B result: does it demonstrate a genuine capability boundary?

The Vicuna-13B failure (Appendix G) is presented as evidence that SELF-REFINE requires "sufficient few-shot modeling or instruction-following abilities" (Section 6). This is a reasonable interpretation, but the experiment is confounded by the use of the same prompts designed for GPT-3.5/GPT-4. The paper acknowledges this: "the limited performance of Vicuna-13B suggests that this model may require more extensive prompt-engineering for optimal performance" (Appendix G). Without testing whether Vicuna-13B could succeed with prompts optimized for its specific capabilities and instruction format, the failure does not conclusively demonstrate that SELF-REFINE is intrinsically beyond weaker models—it may only show that the specific prompt format used for GPT-family models does not transfer. A more rigorous test would involve prompt engineering efforts comparable to those invested in the GPT-3.5 prompts, adapted to Vicuna-13B's training distribution and format preferences.

Are the gains on Code Readability meaningful?

The Code Readability task (Appendix L) uses automatic metrics (Meaningful Variable Ratio, Comments Per Line, Function Units) that are proxies for readability, not direct measures of it. A program with many comments and well-named variables could still be poorly structured and hard to read; conversely, concise code with fewer comments could be highly readable. The comparison to human annotators (Table 14) shows SELF-REFINE at T = 0.7 producing more comments per line (0.25 vs. 0.24) and more function units (1.33 vs. 0.70) but fewer meaningful variables (0.700 vs. 0.653) than humans—but these metrics may reflect stylistic differences rather than true readability superiority. The 60-example human comparison subset is small, and no human evaluation of the output quality is reported for this task (unlike Dialogue, Sentiment, and Acronym tasks). The claims about Code Readability should therefore be treated as preliminary.

What experiments would strengthen the paper?

Several missing experiments limit the conclusiveness of the findings:

Compute-normalized comparison to sampling-based baselines. The paper never compares SELF-REFINE to generating N independent samples (where N equals SELF-REFINE's total LLM call budget) and selecting the best via majority voting or the same multi-aspect scoring used in SELF-REFINE's feedback. This is the most direct test of whether iterative refinement beats parallel sampling with post-hoc selection.
Full cost accounting. The paper does not report wall-clock time, dollar cost per API call, or total token counts for SELF-REFINE vs. baselines. Since SELF-REFINE requires multiple sequential calls (each dependent on the previous output), it incurs higher latency than parallel sampling, even if the total number of generated tokens is similar. For latency-sensitive applications, this is a critical practical consideration.
Ablation of history retention (Equation 4 vs. Equation 3). The paper motivates full history retention as allowing "the model to learn from past mistakes and avoid repeating them" (Section 2), but never ablates this design choice—comparing refinement with full history vs. refinement with only the most recent output and feedback. Without this ablation, the contribution of history retention to the observed gains is unknown.
Sensitivity to iteration count. The 4-iteration maximum is not ablated. For tasks where improvements plateau after 1–2 iterations (Figure 4: Code Optimization, Sentiment Reversal), the additional iterations incur cost without proportional benefit. For Constrained Generation, improvements continue through iteration 3—would iteration 5 or 6 continue to yield gains, or is there a hard ceiling? Without testing different stopping points, the optimal budget allocation per task is unknown.
Robustness to prompt variation. All experiments use a single set of prompts per task. It is unclear how sensitive SELF-REFINE's performance is to the specific wording, rubric design, or few-shot example choice. A prompt sensitivity analysis would establish whether the approach is reliable or brittle.
Broader model evaluation. All main experiments use GPT-family models (GPT-3.5, ChatGPT, GPT-4, Codex). Without testing on other model families (Claude, Llama-2-Chat, Gemini), the claim that SELF-REFINE is "an effective way to obtain better outputs from a single model without any additional training" cannot be validated beyond the OpenAI ecosystem. The Vicuna-13B negative result suggests that capability thresholds matter, but the specific threshold is uncharacterized.
Statistical power for per-task subsamples. The human evaluations use 150 examples per task (Table 6). For Dialogue Response, where the test set has 372 conversations, this is a reasonable fraction. For the newly introduced Acronym Generation (250 total examples) and the sentiment reversal dataset, 150 examples represents a large fraction of the available data, and it is unclear whether the evaluated subset is representative or was randomly sampled.

6. Limitations and Trade-offs

Capability Floor: SELF-REFINE Requires Sufficient Base Model Few-Shot or Instruction-Following Ability

The assumption or constraint. SELF-REFINE assumes that the base model can reliably follow few-shot prompts for three distinct roles—initial generation, feedback provision, and refinement—with enough precision to produce correctly formatted, actionable feedback and to incorporate that feedback without hallucination or format errors. The paper explicitly identifies this boundary in Section 6:

"The main limitation of our approach is that the base models need to have sufficient few-shot modeling or instruction-following abilities, in order to learn to provide feedback and to refine in an in-context fashion."

The consequence. When this assumption is violated, SELF-REFINE fails entirely—not just with degraded performance, but with fundamental breakdowns that render the method unusable. The Vicuna-13B experiment (Appendix G) demonstrates this concretely: the model "was not able to consistently generate the feedback in the required format" and "often failed to adhere to the prompts for refinement. Instead of refining its output, Vicuna-13B either repeated the same output or generated a hallucinated conversation." The example transcript shows Vicuna-13B generating empty feedback (causing a parsing error with "list index out of range"), then generating feedback that misidentifies the target sentiment, and ultimately producing a revision that copies text from the prompt rather than addressing the input. This is not a 10% performance degradation—it is a complete collapse of the refinement loop, indicating that SELF-REFINE is not simply weaker on smaller models but non-functional below some capability threshold.

The consequence for practitioners is that SELF-REFINE is restricted to the strongest available LLMs (GPT-3.5 class and above at the time of writing), which are expensive, API-gated, and subject to usage restrictions. Organizations using open-source models like Llama-2-13B or Mistral-7B cannot assume SELF-REFINE will work for them, even if those models achieve reasonable performance on the underlying task. The paper provides no characterization of where the capability threshold lies—is it model size, instruction-tuning dataset, architecture, or some combination?—making it impossible to predict whether a given model will succeed without empirical testing.

What evidence exists in the paper. The Vicuna-13B experiment (Appendix G) is the sole exploration of this boundary. It is a single data point with a single model, and the paper acknowledges a confound: "we used the same prompts for Vicuna-13b as those used with other models in our study. However, the limited performance of Vicuna-13b suggests that this model may require more extensive prompt-engineering for optimal performance" (Appendix G). This means we cannot distinguish between "Vicuna-13B is intrinsically incapable of SELF-REFINE" and "the prompts designed for GPT-3.5/GPT-4 do not transfer to Vicuna-13B." Without testing whether prompt adaptation (e.g., adjusting the format to match Vicuna-13B's conversational training distribution) could close the gap, the capability boundary is observed but not understood.

Mitigation status. The paper does not attempt to mitigate this limitation—Vicuna-13B is tested and found to fail, and no further model sizes or families are explored. The Mixed-Refine experiment (Appendix G), where Vicuna-13B generates initial outputs and ChatGPT handles feedback and refinement, is presented as a partial workaround: Math Reasoning improves from 24.18% (Vicuna-13B alone) to 40.5% (Mixed-Refine). However, this abandons the core SELF-REFINE premise of self-feedback—the feedback is now cross-model rather than self-generated—and the resulting 40.5% still underperforms ChatGPT's own SELF-REFINE performance (75.0%, Table 1) by a wide margin. The paper suggests this as a direction for future work but does not develop it systematically.

Feedback Quality as the Single Point of Failure—With No Safeguards

The assumption or constraint. SELF-REFINE assumes that the base model can generate accurate feedback—that when it diagnoses a problem and suggests a fix, that diagnosis is more often right than wrong, and the suggested fix actually improves the output. The paper is admirably transparent about this assumption's centrality:

"When SELF-REFINE failed to improve the original generation, the majority of issues were due to erroneous feedback rather than faulty refinements. Specifically, 33% of unsuccessful cases were due to feedback inaccurately pinpointing the error's location, while 61% were a result of feedback suggesting an inappropriate fix. Only 6% of failures were due to the refiner incorrectly implementing good feedback." (Section 4)

The consequence. This asymmetry—94% of failures originate in the FEEDBACK step, only 6% in the REFINE step—means that SELF-REFINE has no mechanism for detecting or recovering from bad feedback. The refinement step will faithfully implement an incorrect suggestion, producing an output that is worse than the original, with no internal check to catch this degradation. The multi-aspect scoring mechanism provides some protection for tasks where quality is numerically scored (e.g., Acronym Generation, Dialogue Response): if a revision scores lower than the previous iteration, the algorithm can select the earlier, better output. But this protection is limited to tasks with rubric-based feedback and does not apply to tasks like Code Optimization where feedback is free-text. Even where score-based selection is used, it only prevents returning the degraded output—it cannot prevent the degradation from occurring in the first place or steer the model away from incorrect diagnoses.

The problem is most acute on Math Reasoning, where the base model's feedback is systemically useless. The paper reports that "ChatGPT feedback for 94% instances is 'everything looks good'" (Section 3.3), meaning the model cannot identify errors in its own reasoning chains. In these cases, SELF-REFINE produces no improvement at all (GPT-4: 92.9% → 93.1%, +0.2%; GPT-3.5: 64.1% → 64.1%, 0%), making the entire iterative loop wasted computation. The practitioner pays for up to 9 LLM calls per input (1 INIT + 4 FEEDBACK + 4 REFINE) but receives effectively the same output as a single call.

What evidence exists in the paper. The 70-sample manual error analysis (Section 4) quantifies the feedback-vs-refinement error distribution. The Math Reasoning result demonstrates the worst-case scenario where feedback is near-universally unhelpful. The Oracle Feedback experiment (Appendix H.1) provides convergent evidence: when an external signal correctly identifies errors, SELF-REFINE's refinement mechanism works (GPT-3.5 improves from 64.1% to 68.9%, +4.8%), confirming that the bottleneck is diagnostic, not remedial. The Dialogue Response error analysis (Tables 11–12, Appendix H) shows that even on a task where SELF-REFINE succeeds overall, feedback quality is imperfect: 25% of failures involve incorrect feedback, 30% involve generic feedback, and 10% involve incorrect numerical scoring.

Mitigation status. The paper identifies this limitation clearly but does not solve it. The Oracle Feedback experiment demonstrates that the problem can be circumvented with external signals, but provides no general mechanism for obtaining such signals. The paper suggests in Section 4 that "future research could focus on examining the refiner's robustness to various types of feedback errors and exploring ways to enhance this resilience," but does not propose concrete approaches (e.g., ensemble feedback from multiple sampling runs, consistency checks across iterations, or learned feedback verification). The score-based output selection is a partial mitigation for rubric-based tasks but does not address the root cause—bad feedback still generates bad refinements, even if the algorithm ultimately returns an earlier, better version. For tasks where quality degrades reliably with each iteration due to systematic feedback errors (rather than fluctuating randomly), score-based selection provides no protection since the first iteration—before any degradation—is also the iteration where SELF-REFINE provides no benefit over the baseline.

Prompt Engineering Cost: "No Training Data" Masks Substantial Per-Task Human Effort

The assumption or constraint. The paper frames SELF-REFINE as requiring "no supervised training data, additional training, or reinforcement learning" (Abstract), and this is technically correct—no model parameters are updated, no gradient descent is performed, no dataset of (input, output, feedback) tuples is collected at scale. However, this framing obscures a different form of supervision: the manual construction of task-specific few-shot prompts with multi-aspect rubrics and demonstration examples. For each of the 7 tasks, the authors designed p_gen, p_fb, and p_refine prompts containing carefully crafted demonstrations of initial generation, multi-aspect feedback with numerical scores and explanations, and refinement trajectories. The paper does not quantify this effort, but it is clearly non-trivial: for Dialogue Response Generation, the prompts include 3 in-context examples each with 10-dimension scored feedback (30 individual scores with explanations); for Acronym Generation, 15 title-acronym pairs for INIT plus 3 scored examples with refinements for FEEDBACK and REFINE; for Sentiment Reversal, manually written pairs of reviews at varying sentiment intensities with corresponding feedback explaining the intensity differences.

The consequence. SELF-REFINE's "training-free" nature is more accurately described as "training-free at deployment time"—the human effort is front-loaded into prompt design rather than back-loaded into data collection. This has two practical implications. First, deploying SELF-REFINE on a new task or domain requires a skilled human to design an appropriate quality rubric, write demonstration examples that instantiate the rubric, and iterate on the prompt to achieve acceptable feedback quality—the paper's reported gains are for prompts that presumably went through such iteration, and the paper provides no methodology or guidance for prompt construction beyond releasing the final prompts. A practitioner without deep familiarity with the target domain or with prompt engineering may struggle to replicate the quality of feedback demonstrations that drive SELF-REFINE's gains.

Second, the cost of prompt engineering is per-task and non-amortizable. If a practitioner wants to use SELF-REFINE for dialogue generation, sentiment reversal, acronym generation, and constrained generation, they need to design and validate four separate prompt sets. This contrasts with approaches like RLHF, where a single reward model (once trained) can guide improvement across many prompts, or with verifier-based methods, where a single trained verifier applies to all inputs in a domain. SELF-REFINE's per-task prompt cost is lower than collecting thousands of training examples per task, but it is not zero, and the paper's framing as "no supervised training data" can mislead practitioners into underestimating the upfront investment required.

What evidence exists in the paper. The paper releases all prompts in full (Appendix S, Figures 16–35), which makes the scale of prompt engineering effort visible: approximately 2-6 pages of carefully structured few-shot examples per task. The paper does not ablate the effort—there is no experiment comparing a minimal prompt (e.g., a single demonstration) against the full prompt set, no measurement of how many iterations of prompt refinement were needed to achieve the reported results, and no sensitivity analysis showing how performance varies with the number or quality of in-context examples. The Vicuna-13B failure, where the same prompts that work for GPT-3.5 fail entirely for Vicuna-13B, illustrates a related risk: the prompts are not model-agnostic, and the effort invested in prompt design for one model family may not transfer to another.

Mitigation status. The paper does not address this limitation explicitly. The release of all prompts is a positive step for reproducibility, but it does not reduce the per-task engineering burden for new domains. The paper does not propose methods for automating prompt construction, for transferring rubrics across related tasks, or for verifying that a given prompt set produces reliable feedback before deployment. A practitioner wanting to use SELF-REFINE on a new task is left to reverse-engineer the prompt design principles from the released examples and to invest their own effort in constructing and validating domain-specific demonstrations.

The assumption or constraint. SELF-REFINE's core algorithm is iterative and sequential: each FEEDBACK step depends on the current output, each REFINE step depends on the current feedback, and each iteration depends on the full history of previous outputs and feedback (Equation 4). With a maximum of 4 iterations and three distinct LLM calls per input (1 INIT + up to 4 FEEDBACK + up to 4 REFINE), each output requires up to 9 sequential API calls, each dependent on the output of the previous call. This is fundamentally different from parallel sampling strategies (best-of-N, majority voting), where all calls can be made simultaneously and latency equals a single generation time.

The consequence. For latency-sensitive applications—interactive dialogue systems, real-time code assistants, any user-facing tool where response time matters—SELF-REFINE imposes a ~9× increase in wall-clock latency compared to direct generation, assuming each LLM call takes roughly equal time. Even if accuracy improves, this latency may be unacceptable: a dialogue system that takes 9 seconds to respond (9 sequential calls at ~1 second each) versus 1 second may lose users regardless of response quality. The paper does not report wall-clock times, API latency statistics, or total token counts per input, making it impossible for practitioners to estimate the latency impact for their specific use case.

The cost overhead is similarly unquantified. While the paper compares SELF-REFINE to baselines using "16 and 32 samples" (Table 8 caption) and notes that SELF-REFINE uses "only 4 samples at most," this comparison conflates different units: SELF-REFINE's "4 samples" are up to 4 refinement iterations (each requiring 2 LLM calls: FEEDBACK + REFINE), totaling up to 9 calls. A direct generation baseline using 9 parallel samples costs the same number of API calls as SELF-REFINE's maximum budget but incurs only single-generation latency (plus a selection step). The paper's MULTI comparison (Figure 6, Appendix H) uses only 4 parallel samples for the baseline versus SELF-REFINE's up-to-9 calls, making SELF-REFINE approximately 2.25× more expensive in terms of API calls. The paper does not report whether a MULTI baseline with 9 parallel samples would match or exceed SELF-REFINE's performance—a comparison that would directly test whether iterative refinement provides value beyond parallel sampling at equal cost.

Furthermore, the long context windows required by full history retention (Equation 4) increase per-call token counts. At iteration 4, the REFINE prompt includes the original input, the initial output, and 3 previous (output, feedback) pairs, each of which may be hundreds or thousands of tokens. This means later iterations are substantially more expensive per call than earlier ones, and the paper does not break down cost by iteration or report total token consumption.

What evidence exists in the paper. The paper provides no systematic cost accounting—no wall-clock latency measurements, no token count breakdowns, no dollar-cost estimates, and no compute-normalized comparison against parallel sampling baselines. The MULTI experiment (Figure 6, Appendix H) is the only attempt at a compute-aware comparison, and it uses a baseline budget (4 samples) that does not match SELF-REFINE's cost (up to 9 calls). The paper acknowledges latency only implicitly, through the choice of a 4-iteration maximum rather than continuing until convergence—a practical concession to cost that is not framed as a tradeoff but simply as a parameter choice.

Mitigation status. The paper does not address the latency or cost overhead. The algorithm always runs all iterations (up to the stopping condition or maximum), with no mechanism for early termination when improvements plateau—Figure 4 shows that tasks like Code Optimization and Sentiment Reversal gain very little after the first iteration (+0.9 and +0.7 from y₂ to y₃, respectively), yet the algorithm continues to 4 iterations, spending API calls for diminishing returns. The paper does not propose adaptive stopping criteria that could reduce average cost by terminating refinement when feedback indicates sufficient quality or when score improvements between iterations fall below a threshold.

English-Only and Single-Paradigm Evaluation: No Evidence for Cross-Lingual or Cross-Domain Generality

The assumption or constraint. All experiments are conducted exclusively on English-language datasets, and all base models are English-centric (GPT-3.5, ChatGPT, GPT-4, Codex). The paper explicitly acknowledges this in Section 6:

"Another limitation of our work is that we exclusively experiment with datasets in English. In other languages, the current models may not provide the same benefits."

The consequence. There is no evidence that SELF-REFINE's feedback-and-refinement mechanism generalizes to non-English languages, where base model quality and instruction-following ability may be substantially weaker—even GPT-4's performance degrades on many non-English tasks. The multi-aspect rubrics used in SELF-REFINE are designed in English and demonstrated with English examples; whether the model can generate equivalently structured, actionable, specific feedback in Japanese, Arabic, or Hindi is unknown. If the feedback format or rubric design is language-specific (e.g., certain quality dimensions like "fluency" or "specificity" may manifest differently across languages), then prompt engineering effort would need to be replicated per language, compounding the per-task prompt cost discussed above.

Beyond language, the evaluation is limited to a specific paradigm of tasks: short-to-medium-form generation with discrete, evaluable outputs. All 7 tasks involve producing a single output unit—a dialogue response, a code snippet, an acronym, a sentence, a solution—that can be evaluated against a rubric or metric. SELF-REFINE has not been tested on long-form generation (essays, articles, stories), multi-turn interactive tasks (negotiation, tutoring), structured prediction (information extraction, parsing), or tasks requiring external knowledge retrieval. The website generation example in Appendix I is suggestive of broader applicability but is a single qualitative demonstration, not a systematic evaluation. For tasks where the "output" is a complex artifact (e.g., a software system with multiple files, a research paper with citations), the current instantiation of SELF-REFINE provides no mechanism for decomposing the feedback-and-refinement process across sub-components.

What evidence exists in the paper. None, for cross-lingual evaluation. The paper's domain breadth is 7 tasks, which is substantially more than most prior refinement work (compare: Self-Correction evaluates on 2–3 tasks; Re³ evaluates on story generation; PEER evaluates on Wikipedia editing), but all tasks share the structure of "generate a single text output from a single text input." The paper does not claim cross-lingual generality or evaluate it, making the limitation acknowledged but unmeasured.

Mitigation status. The paper acknowledges the English-only limitation explicitly (Section 6) but does not propose mitigation strategies or future work directions for cross-lingual extension. Given that the capability requirements for SELF-REFINE (few-shot instruction following, structured feedback generation, multi-aspect evaluation) are likely weaker in non-English languages for current models, this limitation may be fundamental to the current generation of LLMs rather than a simple matter of running the same experiments in other languages. The paper does not discuss this.

Uncontrolled Model Access: Dependence on Black-Box, Proprietary APIs Undermines Reproducibility

The assumption or constraint. All main experiments rely on closed-source, API-gated models (GPT-3.5, ChatGPT, GPT-4, Codex) whose training data, model architecture, parameter counts, and exact training procedures are not publicly documented. The paper acknowledges this in Section 6:

"The experiments in this work were performed with language models that are not open-sourced, namely GPT-3.5, ChatGPT, GPT-4, and CODEX. Existing literature (Ouyang et al., 2022) does not fully describe the details of these models, such as the pretraining corpus, model sizes, and model biases. Further, these models are not free to use, and using them for research requires some funding."

The consequence. This dependence creates three distinct problems for the research community. First, reproducibility: the specific GPT-3.5, ChatGPT, and GPT-4 endpoints used in the paper may behave differently from current or future versions. OpenAI has deprecated text-davinci-003 and continuously updates gpt-3.5-turbo and gpt-4; the paper's exact model versions are identified (e.g., text-davinci-003, gpt-3.5-turbo) but these may not remain accessible. A researcher attempting to replicate the results in 2025 may find that the same prompts produce different outputs, and there is no way to determine whether discrepancies are due to model updates or to errors in their replication.

Second, scientific understanding: without knowing what the models were trained on, how large they are, or what instruction-tuning procedure was used, we cannot determine why SELF-REFINE works. Is it a property of model scale (GPT-4 > ChatGPT > GPT-3.5 in SELF-REFINE benefit)? Of instruction-tuning data (Vicuna-13B fails despite comparable size to some instruction-tuned models)? Of RLHF training? The Vicuna-13B failure hints at a capability threshold but cannot locate it—is the threshold a specific model size, a specific training dataset, or a specific fine-tuning recipe? Without access to model internals and training details, all observed regularities are correlational, not causal.

Third, cost and access: the paper notes that "using them for research requires some funding" (Section 6). At the time of writing, running the full set of SELF-REFINE experiments—7 tasks × 3 models × up to 9 API calls per input × hundreds of test examples—would cost hundreds to thousands of dollars in API fees. This creates an equity barrier: researchers without institutional funding or API credits cannot replicate or extend these results, and practitioners in low-resource settings cannot evaluate whether SELF-REFINE would benefit their applications before committing to the cost.

What evidence exists in the paper. The Vicuna-13B experiment (Appendix G) attempts to demonstrate SELF-REFINE with an open-source model but fails, leaving the paper with no positive results on open models. The Mixed-Refine experiment shows that combining a weaker open-source generator with a proprietary refiner yields some improvement (24.18% → 40.5% on Math Reasoning), but this hybrid approach still depends on proprietary models for the feedback and refinement steps and does not achieve the performance of fully proprietary SELF-REFINE.

Mitigation status. The paper releases all code, prompts, and model outputs "to ensure the reproducibility of our work" (Section 6), which is a meaningful step—even if the exact models become unavailable, future researchers can examine the outputs, analyze the feedback content, and compare against their own model runs. However, this does not address the underlying reproducibility problem. Outputs alone do not enable re-running the method on new inputs or new tasks, which requires API access to the same model versions. The paper does not propose or evaluate SELF-REFINE on any open-source model that successfully performs the full FEEDBACK-REFINE loop, leaving the open-source path unexplored.

7. Implications and Future Directions

How This Work Changes the Landscape

SELF-REFINE shifts the conversation around LLM self-improvement from a training problem to a prompting discipline. Before this paper, the dominant assumption—implicit in work on learned refinement models (Self-Correction, PEER, DrRepair) and RL-based alignment (RLHF, QUARK, CodeRL)—was that a model's ability to evaluate and improve its own outputs must be installed through parameter updates, either via supervised data or reward signals. SELF-REFINE demonstrates that this assumption is false for sufficiently capable contemporary LLMs: the same model, without any fine-tuning, can serve as generator, critic, and revisor when prompted with carefully structured few-shot examples demonstrating how to provide multi-aspect, actionable feedback and how to incorporate it into revisions.

The magnitude of this shift is a reframing rather than a paradigm shift—SELF-REFINE does not introduce a new architecture or training algorithm, but it changes where researchers should look for the self-improvement bottleneck. The paper's most durable contribution is the empirical demonstration that feedback quality is the binding constraint, not refinement capability. The manual error analysis in Section 4 establishes an asymmetric failure distribution—94% of failures originate in inaccurate or unhelpful feedback, while only 6% stem from the refiner incorrectly implementing good feedback—and the Math Reasoning result provides a clean boundary case where the feedback mechanism collapses entirely (94% of ChatGPT feedback instances are "everything looks good") while refinement with Oracle-corrected feedback succeeds (GPT-3.5 improves from 64.1% to 68.9%, Appendix H.1). This asymmetry reframes the research challenge: the question is no longer "how do we train models to self-improve?" but rather "how do we design prompts and formats that reliably elicit accurate self-evaluation from models that already possess latent metacognitive capabilities?"

The paper also reconciles contradictory prior findings in the self-improvement literature. Huang et al. (2023) found that "large language models cannot self-correct reasoning yet"—a conclusion SELF-REFINE corroborates for math reasoning specifically, where the base model's self-evaluation is unreliable. But Madaan et al. (2023) and others found that LLMs can meaningfully improve code through iterative refinement—which SELF-REFINE confirms for domains where the model can produce actionable feedback. The resolution is that self-improvement capability is not a unitary property of a model but a function of the interaction between model capability and task domain: when the model can diagnose its own errors (dialogue, sentiment reversal, code optimization), self-refinement works; when it cannot (math reasoning for GPT-3.5, or any task for Vicuna-13B), it fails. This domain-specific boundary condition provides a more nuanced framework than prior blanket claims that LLMs can or cannot self-improve.

As a consequence of this reframing, certain research directions become more attractive: prompt engineering for structured, rubric-based feedback formats; developing methods to verify or improve the accuracy of model-generated self-feedback without external supervision; characterizing the capability threshold at which self-evaluation becomes reliable; and combining self-feedback with external verification signals for domains (like math) where self-diagnosis is insufficient. Conversely, directions that become less urgent include training separate critic or refiner models for domains where the base model already possesses adequate self-evaluation—SELF-REFINE shows that a single prompted model can perform both roles at inference time with no training cost, making supervised critic training unnecessary for many applications.

Follow-Up Research This Work Enables

Characterize the capability threshold for reliable self-feedback. The Vicuna-13B experiment (Appendix G) establishes that SELF-REFINE fails entirely below some capability level, but provides only a single data point with a single prompt format. A systematic scaling study would instantiate SELF-REFINE across a range of open-source models of varying sizes (7B, 13B, 34B, 70B parameters) and instruction-tuning recipes, using prompts optimized for each model class rather than reusing the GPT-family prompts. The key measurement would be the feedback accuracy rate—what fraction of feedback instances correctly identify an actual problem in the output—as a function of model scale and instruction-tuning methodology. The hypothesis is that feedback accuracy exhibits a thresholding behavior (near-zero below some scale, rapidly improving above it), which would establish a minimum viable model size for deployment. The Vicuna-13B result may partially reflect prompt-format mismatch rather than intrinsic capability limits, so prompt adaptation (e.g., reformatting the multi-aspect rubric to match Vicuna's conversational training distribution) is essential to the experimental design. A strong study would also test whether the threshold differs across tasks—feedback accuracy on objective tasks like code optimization may emerge at smaller scales than on subjective tasks like dialogue quality assessment.

Combine SELF-REFINE with external verification for math and formal reasoning. The Math Reasoning Oracle experiment (Appendix H.1) demonstrates the principle: when an external source correctly identifies that an error exists, SELF-REFINE's refinement mechanism improves performance (GPT-3.5: +4.8%). The natural extension is to replace the Oracle with a practically obtainable signal—execution of the generated code against test cases, symbolic verification of mathematical steps, or unit tests for code optimization. The three-way interaction to study is: (1) does the model generate more accurate self-feedback when an external verifier first flags that an error exists (without specifying where or why)? (2) Does SELF-REFINE then refine more effectively when that external signal triggers the refinement loop? (3) Can the model learn to request external verification when its own confidence is low, creating an adaptive loop that only incurs verification cost on ambiguous cases? GSM-8k with Python execution-based answer checking would be the natural starting point. The paper's finding that "ChatGPT feedback for 94% instances is 'everything looks good'" (Section 3.3) suggests that external flagging is the minimum necessary intervention—without it, the refinement loop never engages.

Ablate the contribution of full history retention and multi-aspect scoring to identify the minimal viable SELF-REFINE. The paper motivates two design choices theoretically—full history retention (Equation 4 vs. Equation 3) to "learn from past mistakes and avoid repeating them" (Section 2), and multi-aspect numerical scoring to handle non-monotonic quality (Section 4, Table 10)—but neither is empirically ablated. A minimal-SELF-REFINE experiment would compare four conditions: (a) full SELF-REFINE (history + multi-aspect + score-based selection), (b) no history (only most recent output and feedback in context), (c) single-aspect feedback (one aggregate score instead of dimension-by-dimension scoring), (d) last-output selection (always return the final refinement instead of the highest-scoring iteration). The experiment would be run on tasks where these mechanisms are hypothesized to matter most—Acronym Generation (for score-based selection, given the observed non-monotonicity in Table 10), Dialogue Response (for multi-aspect scoring, given the 10-dimension rubric), and Code Optimization (for history retention, since improvements are monotonic and history may be unnecessary). The result would identify which components of SELF-REFINE are load-bearing and which are incidental, enabling a simpler, cheaper deployment that omits unnecessary complexity.

Test whether SELF-REFINE feedback can be reused to train a persistent improvement model. SELF-REFINE treats each input independently—there is no learning across examples. But the feedback generated during SELF-REFINE runs constitutes a growing dataset of (input, flawed-output, diagnosis, improved-output) tuples generated without human annotation. A natural follow-up asks: can this self-generated data be used to fine-tune the base model, such that the model internalizes the improvement behavior and produces better first-pass outputs without iterative refinement at deployment time? The experiment would run SELF-REFINE on a training set, collect all FEEDBACK-REFINE pairs, and use them for supervised fine-tuning of the same base model (training the model to map flawed-output + feedback → improved-output, or more ambitiously, to generate the improved output directly from the input, bypassing the intermediate flawed generation). The test would compare (a) base model with SELF-REFINE at deployment time (the current approach), (b) fine-tuned model from SELF-REFINE data, single-pass, and (c) fine-tuned model with SELF-REFINE applied on top. The paper's Code Optimization results—where SELF-REFINE generates specific algorithmic feedback like "use the formula (n*(n+1))/2 instead of the for loop" (Figure 2)—suggest that the feedback encodes actionable optimization knowledge that could generalize if distilled into the model parameters, potentially reducing the need for iterative refinement on future inputs.

Stress-test SELF-REFINE on adversarial or systematically deceptive self-feedback. The paper's error analysis (Section 4) shows that 94% of failures are feedback-driven—the model generates inaccurate diagnoses or inappropriate fix suggestions, and the refiner obediently implements them. This raises a concerning failure mode: if the model's feedback is not just inaccurate but systematically wrong in a particular direction, SELF-REFINE could amplify rather than correct errors. For example, on a sentiment reversal task, if the model's feedback consistently underestimates the intensity of negative sentiment needed, each iteration could produce progressively milder reversals that drift away from the target. A stress-test experiment would introduce different types of feedback perturbation—random noise in numerical scores, systematic bias toward a particular quality dimension (e.g., always prioritizing "safety" over "engagement" in dialogue), or deliberately incorrect error localization—and measure how SELF-REFINE's output quality degrades. The Dialogue Response error analysis (Table 12, Appendix H) already shows that the refiner ignores good feedback 25% of the time and introduces new problems 20% of the time—this experiment would quantify whether these failure rates compound across iterations when feedback is systematically misleading. The result would inform whether SELF-REFINE requires a feedback quality check or fallback mechanism before deployment in high-stakes settings.

Evaluate SELF-REFINE against compute-normalized parallel sampling baselines. The paper's MULTI comparison (Figure 6, Appendix H) uses 4 parallel samples versus SELF-REFINE's up-to-9 LLM calls—a 2.25× cost asymmetry that makes the comparison uninterpretable as a compute-normalized test. The critical missing experiment is: at equal total LLM call budget (e.g., 9 calls each), does SELF-REFINE's iterative refinement outperform (a) generating 9 independent samples and selecting the best via the same multi-aspect scoring used in SELF-REFINE's FEEDBACK step, or (b) generating 9 independent samples and selecting via majority voting (for tasks with objective answers) or a trained verifier? For Math Reasoning, majority voting over 9 samples is a standard strong baseline that SELF-REFINE does not compare against. For Code Optimization, execution-based verification (running all generated programs and keeping the fastest correct one) is an even stronger baseline. For subjective tasks like Dialogue Response, using the multi-aspect rubric from SELF-REFINE's FEEDBACK to score 9 independent samples and pick the highest-scoring one uses the same evaluation mechanism as SELF-REFINE but without the iterative refinement—is the iterative component adding value, or is it just a more expensive way to explore the output space? This experiment would definitively answer the question the paper raises in Section 4 ("Can we just generate multiple outputs instead of refining?") with the compute normalization that the existing MULTI comparison lacks.

Practical Applications and Downstream Use Cases

Cost-efficient quality improvement for black-box API deployments. The most immediate practical use case for SELF-REFINE is improving output quality from closed-source LLM APIs (GPT-4, Claude, Gemini) without access to model weights for fine-tuning. A company building a customer-facing dialogue system on GPT-4 can implement SELF-REFINE entirely client-side—the same API is called with different prompts for INIT, FEEDBACK, and REFINE—and obtain outputs that are preferred by human evaluators 54.0% vs. 16.0% over direct generation (Table 15, Appendix M), a more-than-3× preference advantage. For sentiment-sensitive applications like review generation or content moderation, SELF-REFINE's 32.4% absolute improvement on Sentiment Reversal with GPT-4 (Table 1) translates to output that more reliably meets the target sentiment intensity. The cost is up to 9× the per-input API calls versus single-pass generation, but for applications where (a) output quality directly impacts revenue (e.g., marketing copy, customer support responses) and (b) the cost of poor output exceeds the additional API expense, this tradeoff is easily justified. The paper does not provide dollar-cost estimates, but at GPT-4 pricing circa early 2023 (~$0.03–0.06 per 1K tokens), the additional cost per output would be on the order of cents to tens of cents, which is negligible for many commercial applications.

Data generation for instruction-tuning and distillation pipelines. The paper's finding that SELF-REFINE produces higher-quality outputs across diverse tasks—outperforming human annotators on some Code Readability metrics (Table 14, Appendix L) and approaching human-level code optimization (36.0% vs. 38.2% human reference; Table 8, Appendix F)—makes it a compelling tool for generating training data. A practitioner training a smaller, deployable model can use SELF-REFINE with GPT-4 to generate high-quality outputs on their training set, then fine-tune the smaller model on these (input, refined-output) pairs. The SELF-REFINE loop automatically produces a quality gradient—initial outputs, intermediate refinements, and final high-scoring outputs—which could be used for preference-based training methods like DPO. The paper's demonstration across 7 tasks provides templates (prompts, rubrics, scoring dimensions) that reduce the upfront effort of designing a SELF-REFINE data generation pipeline for new domains. The primary cost is the GPT-4 API calls for FEEDBACK and REFINE on the training set, which is a one-time expense per task—once the training data is generated, the distilled model handles deployment queries at much lower cost.

Iterative document and content revision in collaborative writing tools. SELF-REFINE's design—initial draft, multi-aspect feedback with specific suggestions, iterative refinement with revision history—maps directly onto the workflow of collaborative writing tools like Google Docs, Notion, or specialized technical writing assistants. A user writes an initial draft (replacing INIT); SELF-REFINE provides structured feedback on dimensions like clarity, specificity, engagement, and tone (replacing a human editor's first-pass review); the user or the model revises the draft based on that feedback; the cycle repeats. The paper's Dialogue Response Generation results (Section M) and the website generation example (Appendix I) demonstrate that the same rubric-based feedback approach transfers across content types. The website generation case study—where SELF-REFINE iteratively improves HTML/CSS/JS from a basic layout to a polished page (Figures 7–10, Appendix I)—shows the potential for structured artifact generation beyond text, though this is a qualitative demonstration rather than a systematic evaluation. The practical deployment would use the multi-aspect scoring to flag sections that need attention (e.g., "specificity: 1/3, engagement: 2/3") and suggest concrete edits, with the user retaining final approval over changes—mitigating the risk of bad feedback driving degradation, since the human-in-the-loop can reject unhelpful suggestions.

Quality assurance for constrained generation in regulatory or compliance settings. The Constrained Generation (CommonGen-Hard) task—generating sentences that incorporate 20–30 required concepts—demonstrates SELF-REFINE's ability to handle output constraints that are easily verifiable but hard to satisfy in a single pass. GPT-4+SELF-REFINE achieves 45.0% coverage versus 15.0% base (Table 1), a 3× improvement on what is essentially a checklist-completion task. This maps onto real-world applications where generated text must include or exclude specific terms, topics, or disclosures: medical report generation (must mention all relevant findings), legal document drafting (must include all required clauses), regulated marketing copy (must include mandatory disclosures while maintaining persuasive tone). The FEEDBACK step's explicit enumeration of missing concepts—"Concept Feedback: animal, ride" (Figure 25, Appendix S)—provides a transparent, auditable rationale for each revision that is valuable in compliance contexts where the reasoning behind output choices may need to be documented. The practical implementation would add a domain-specific constraint checklist to the FEEDBACK prompt, with the FEEDBACK step serving double duty as a compliance verification and an improvement guide.