Undesirable Memorization in Large Language Models: A Survey

🎯 Pitch

Large language models are not just learning patterns—they’re silently memorizing up to 1% of their training data verbatim, and this rate scales relentlessly with model size. This survey reveals that no single defense—from deduplication to differential privacy—can erase this privacy risk without degrading performance, forcing a fundamental trade-off that current techniques cannot resolve.

1. Executive Summary

This survey systematically organizes and analyzes the literature on undesirable memorization in Large Language Models—the tendency of models to store and reproduce verbatim or near-verbatim phrases from their training data—introducing a three-dimensional taxonomy spanning granularity (perfect, verbatim, approximate, entity-level, and content), retrievability (extractable vs. discoverable), and desirability (desirable knowledge retention vs. undesirable privacy leakage and copyright infringement). Drawing on 120 papers identified through a structured selection process, the survey examines measurement methodologies—string matching, exposure metrics, inference attacks like membership inference and extraction attacks, counterfactual memorization, and heuristic compression-based approaches—across primarily the Pythia and GPT-Neo model families trained on the PILE dataset, establishing that model capacity, data duplication, prompt length, and decoding strategies are the dominant factors driving memorization while deduplication, differential privacy, training interventions like goldfish loss, and post-training methods such as unlearning and model editing constitute the primary mitigation toolkit. The analysis reveals that verbatim memorization rates scale log-linearly with model size and that approximately 1% of training data becomes discoverably memorized at standard prefix lengths, establishing that current mitigation methods create a trade-off between privacy and utility rather than a complete solution—boundary conditions under which no single technique fully eliminates memorization risk without degrading performance.

2. Context and Motivation

The Core Problem: We Have Catalogued Memorization but Not Its Boundary Conditions

The fundamental gap this survey addresses is not that we don't know LLMs memorize—that has been well-documented since Carlini et al. (2021, 2023)—but rather that the literature on memorization has become fragmented across disparate research communities, with conflicting definitions, inconsistent measurement protocols, and no unified framework for understanding when memorization becomes undesirable, why it occurs under specific configurations, and how to mitigate it without destroying model utility. The paper diagnoses this fragmentation explicitly in Section I-A through its comparison with prior surveys (Table I), arguing that existing work either treats memorization as a subordinate concern within broader privacy surveys (Neel and Chang, 2023; Smith et al., 2023) or provides a general overview without the prescriptive research agenda that the current survey supplies (Hartmann et al., 2023).

This gap has concrete consequences. Researchers studying memorization through the lens of differential privacy use exposure metrics and $\epsilon$ -DP guarantees; researchers investigating copyright infringement measure verbatim reproduction rates against known texts; researchers focused on security rely on extraction attack success rates. Each community measures a different facet of the same underlying phenomenon, yet there is no systematic mapping between these measurements. A model judged "safe" by differential privacy standards may still reproduce copyrighted training passages; a model that passes extraction attack evaluations may still leak entity-level associations that violate privacy. The survey's three-dimensional taxonomy—granularity, retrievability, desirability—is explicitly designed to reconcile these perspectives by providing a common vocabulary and showing how different measurement methods capture different regions of this 3D space (Figure 3).

Why This Problem Matters: Privacy Breaches, Legal Liability, and Benchmark Contamination

The survey identifies three interconnected real-world consequences that make a systematic understanding of memorization urgent:

Privacy violations are already documented. The paper cites foundational work by Carlini et al. (2021) demonstrating extraction of personally identifiable information from GPT-2's training data, Nasr et al. (2023) showing scalable extraction from production systems including ChatGPT, and Lukas et al. (2023) demonstrating that fine-tuning with DP-SGD at commonly-used privacy budgets still leaks PII under stronger attacks. These are not hypothetical risks—Section II-A0d documents entity-level memorization where models associate names with phone numbers, emails, and addresses even when the exact training string is paraphrased. Kim et al. (2023) show emails and phone numbers exhibit higher extraction rates than addresses in OPT models, revealing that the type of sensitive information matters for risk assessment. The survey positions this as an urgency argument: memorization is not merely a theoretical curiosity but a demonstrated attack vector that scales with model capability.

Copyright and intellectual property liability is unresolved. Section II-C lists copyright violation as a core undesirable consequence, and the survey cites multiple works (Henderson et al., 2023; Cooper and Grimmelmann, 2025; Freeman et al., 2024) that examine the tension between fair use and verbatim reproduction of copyrighted training material. The practical stakes are high: the New York Times v. OpenAI lawsuit is cited as a landmark case where memorization is a central evidentiary concern (Freeman et al., 2024), and Mueller et al. (2024) analyze the quality and specificity dimensions of copyright-compliant memorization, showing that simple output filtering is insufficient because models can reproduce protected content with minor syntactic variations that evade string matching. The survey's taxonomy of granularity is directly relevant here: verbatim reproduction is the clearest legal violation, but approximate memorization (Section II-A0c) may be practically equivalent for copyright purposes while being much harder to detect at scale.

Benchmark contamination undermines evaluation validity. Section II-C notes that memorization of benchmark datasets leads to "overfitting and inflated performance on seen data," citing Elangovan et al. (2021) on data leakage in NLP evaluation and Bordt et al. (2024) on memorization of tabular data in LLMs. This is a reliability-of-science concern: if models memorize test sets, reported accuracy numbers do not reflect generalization capability. The survey's discussion of Salido et al. (2025) in Section V-B2 is particularly relevant—their "None of the Others" (NOTO) benchmark modification produces large accuracy drops, revealing that standard evaluations conflate reasoning with memorization. Without a rigorous framework for measuring when models rely on memorized rather than reasoned outputs, the entire evaluation ecosystem rests on potentially contaminated benchmarks.

Where Prior Approaches Fall Short

The survey identifies specific limitations along four axes that motivate its unifying framework:

Existing surveys treat memorization as a sub-problem, not the main focus. Table I explicitly enumerates this gap. Usynin et al. (2024) survey memorization in machine learning broadly, not LLMs specifically—missing LLM-specific dynamics like training data scale effects, tokenization impacts, and the discoverability-vs-extractability distinction that only matters for generative text models. Wei et al. (2024) cover memorization in deep neural networks, but their framework predates the LLM era and does not engage with instruction tuning, conversational agents, or retrieval-augmented generation as unique contexts. Neel and Chang (2023) and Smith et al. (2023) are privacy surveys where memorization is one section among many. Hartmann et al. (2023) is the closest predecessor, but the survey argues it does not distinguish clearly enough between desirable and undesirable memorization—a distinction that is conceptually central to the current paper's taxonomy—and does not provide the concrete, actionable future directions that Section VI of the current survey delivers.

Measurement protocols are inconsistent, making cross-study comparison nearly impossible. Table III is the survey's empirical evidence for this claim. It catalogs studies by their choice of model family (Pythia, GPT-Neo, GPT-J dominating), dataset (PILE dominant), decoding method (greedy dominant), prefix length (ranging from 32 to 450 tokens across studies), match length (32 to 1000 tokens), and which granularity/retrievability combination they test. The table reveals that approximately 60% of studies focus on discoverable memorization while 40% examine extractable, and similarly 60% examine verbatim while 40% split between approximate and entity-level. These are different things being measured with different protocols, yet results across these studies are often compared as though they capture the same phenomenon. The survey's taxonomy makes this fragmentation explicit so that future work can select measurement approaches matched to the specific risk being assessed (privacy vs. copyright vs. benchmark contamination) rather than defaulting to the most convenient metric.

Mitigation methods lack systematic comparison and boundary characterization. Section V catalogs a wide range of mitigation approaches—Deduplication (Lee et al., 2022), differential privacy via DP-SGD (Carlini et al., 2019; Li et al., 2022), training interventions like goldfish loss (Hans et al., 2024) and alternating teaching (Liu et al., 2023), post-training unlearning (Bourtoule et al., 2021; Chen and Yang, 2023), model editing (Ruzzetti et al., 2025), and decoding-based approaches (Ippolito et al., 2023)—but the survey's own Remark in Section V-C points out that no prior work systematically compares these methods under a unified evaluation protocol or characterizes their failure modes. Each method is evaluated on different models, different datasets, and different memorization metrics, making it impossible to determine whether deduplication is actually more effective than DP-SGD or whether goldfish loss generalizes across model scales. The survey does not resolve this—that would require new experiments—but it constructs the taxonomic scaffolding necessary to design such a comparison by mapping each method to the specific type of memorization it targets (verbatim vs. approximate, training-time vs. inference-time).

The desirability dimension is acknowledged but under-theorized. The survey argues that prior work oscillates between treating memorization as uniformly harmful (the privacy and security literature) or acknowledging its role in factual recall without analyzing the tension (the knowledge-retention literature). Section II-C0b lists transparent benefits—auditing/traceability, knowledge retention, linguistic pattern learning, alignment value injection—while Section II-C0a lists harms—privacy risk, copyright violation, bias perpetuation, benchmark contamination, reasoning degradation. The survey's key conceptual move is the Remark at the end of Section II-C: the same memorization behavior can be simultaneously beneficial to one stakeholder (users who want factual accuracy) and harmful to another (data subjects whose private information has been memorized). This is not resolved by current taxonomies that label memorization as simply "good" or "bad"—the survey argues for a contextual, multi-stakeholder analysis that prior work has not systematically provided.

How This Paper Positions Itself

The survey positions itself as a synthesizing framework rather than a collection of new experimental results, explicitly organized around five research questions (Section I-C) that structure the entire paper: (RQ1) definition, (RQ2) measurement, (RQ3) influencing factors, (RQ4) mitigation, (RQ5) open challenges. The paper's contribution is explicitly self-described in the conclusion as "organized, summarized, and discussed the existing scientific work related to undesirable memorization in LLMs"—with an emphasis on the undesirable qualifier that distinguishes it from Hartmann et al. (2023).

The positioning relative to prior surveys is structurally encoded in Section I-A and Table I. The survey argues that memorization is too important to be a sub-topic within general privacy surveys—the phenomenon deserves focused treatment because it spans privacy, security, copyright, reliability, and evaluation concerns that are typically siloed in separate research communities. The paper does not claim to resolve the tensions between these communities; rather, it builds the conceptual infrastructure—the 3D taxonomy, the factor-to-mechanism mapping in Table II—that makes those tensions visible and tractable.

The survey's most distinctive positioning move is its prescriptive research agenda in Section VI, which is structured around specific, under-explored contexts rather than general calls for "more research." The four contexts identified—conversational agents, RAG systems, multilingual LLMs, and diffusion language models—are not arbitrary: each is selected because it represents a production-relevant setting where memorization risks manifest differently than in the standard pretraining-and-evaluation paradigm that dominates existing work. For conversational agents, the survey notes that existing divergence attacks (Nasr et al., 2023) "are not powerful enough to stimulate training data reproduction," requiring novel attack methodologies. For RAG, Zeng et al. (2024) provide preliminary evidence that retrieval can both increase privacy risks (by exposing private documents in retrieved context) and decrease them (by reducing reliance on memorized parametric knowledge), and the survey calls for a systematic study of this dual effect. For multilingual models, Satvaty et al. (2025) and Luo et al. (2025) provide early evidence that memorization is language-dependent—low-resource languages exhibit different patterns—and the survey positions this as a fairness and equity concern. For diffusion language models, the survey notes that while Carlini et al. (2023) have demonstrated severe memorization in image diffusion models, the text domain is fundamentally different due to discreteness, and this gap needs to be filled.

The survey's theoretical spine is the claim that memorization is not a monolithic phenomenon but a spectrum—the deliberate term used in Section II's title—and that progress requires matching measurement methods, mitigation strategies, and risk assessments to the specific region of this spectrum that is relevant for a given application. This is more than a taxonomic exercise: it implies that the research community's current practice of using verbatim string matching on Pythia models with greedy decoding as a proxy for "memorization risk" is systematically missing forms of memorization (approximate, entity-level, content-level) that may be more practically harmful, and that mitigation methods validated only against verbatim extractable memorization may provide "a false sense of privacy" (to quote Ippolito et al., 2023, which the survey cites extensively in Section V-C3).

The survey also implicitly positions itself as a living document, noting in the Abstract that a dedicated GitHub repository will be "regularly updated to reflect the latest developments." This acknowledges the limitation that the paper itself identifies in its Limitations section: "a survey paper written in 2025 risks becoming outdated by 2026." By maintaining a continuously updated bibliography organized around its taxonomic dimensions, the survey aims to outlive its publication date as a community resource rather than a static snapshot.

3. Technical Approach

3.1 Reader Orientation

This is a survey paper—a systematic organization, synthesis, and analysis of existing research on memorization in LLMs, not a paper proposing a new model, algorithm, or system. The "system" being built is a conceptual framework: a three-dimensional taxonomy (granularity × retrievability × desirability) that classifies and relates 120 published studies, enabling researchers to understand what kind of memorization their measurement method captures, what factors drive it, and where mitigation strategies break down. The problem it solves is fragmentation: the research community has been studying memorization using inconsistent definitions, incompatible measurement protocols, and siloed mitigation approaches across privacy, security, copyright, and evaluation subfields, making cross-study comparison nearly impossible and masking the boundary conditions under which current solutions work.

3.2 Big-Picture Architecture (Diagram in Words)

The survey constructs a conceptual pipeline with five interconnected components, each mapping to one research question (RQ1–RQ5 from Section I-C). Information flows through these components not as data through a neural network, but as a cumulative argument where each stage builds on the definitions established in the previous stage:

Taxonomic Framework (RQ1: How is memorization defined?) — A 3D classification space defined by three orthogonal axes. This component defines the vocabulary that all subsequent analysis depends on. Input: the universe of memorization-related phenomena observed in LLMs. Output: a label in the 3D space for any observed instance (e.g., "verbatim × discoverable × undesirable," "entity-level × extractable × undesirable"). This establishes the spectrum concept—memorization is not binary.
Measurement Catalog (RQ2: How is memorization measured?) — A systematic inventory of quantification methods, each mapped to specific regions of the taxonomic space and characterized by practical constraints. Input: a measurement technique described in the literature. Output: its sensitivity profile (which types of memorization it detects, which it misses), computational cost, and assumptions. This catalog reveals that approximately 60% of studies measure discoverable memorization while 40% measure extractable, and that matching type-of-memorization to type-of-measurement is rare in practice.
Factor-to-Mechanism Mapper (RQ3: What factors contribute?) — A structured analysis categorizing 14+ influencing factors into three buckets (model-related, training pipeline, inference-time) with documented causal directions and effect magnitudes. Input: a factor (e.g., "data duplication," "model capacity," "prefix length"). Output: the mechanism by which it affects memorization (e.g., duplication causes more frequent exposure to specific token sequences during optimization, increasing their probability mass), the quantitative relationship where known (e.g., "log-linear scaling with model size" from Carlini et al., 2023), and interactions with other factors.
Mitigation Strategy Map (RQ4: How is undesirable memorization prevented?) — A taxonomy of countermeasures organized by intervention point in the ML lifecycle (data-level → training-time → post-training → inference-time), with each method characterized by the specific memorization types it targets, its empirical effectiveness ceiling, and its documented side effects on model utility. Input: a mitigation method. Output: its coverage in the taxonomic space (which regions of the 3D cube it addresses), its reported utility cost, and gaps (regions left unprotected).
Future Directions Engine (RQ5: What remains unexplored?) — A gap analysis that intersects the taxonomic framework with understudied deployment contexts (conversational agents, RAG, multilingual models, diffusion language models). Input: deployment context not yet systematically studied for memorization. Output: specific research questions and methodological challenges. This component is not a literature review—it is a prediction of where novel memorization behaviors will emerge and where existing mitigation strategies are likely to fail based on structural properties of the context.

How these components interact: the Taxonomic Framework (1) provides the vocabulary used by every subsequent component. The Measurement Catalog (2) maps measurement tools to taxonomic regions, revealing blind spots. The Factor-to-Mechanism Mapper (3) explains why certain taxonomic regions are over-represented in current studies (researchers choose configurations that maximize measurable memorization). The Mitigation Map (4) is evaluated against the Measurement Catalog—a mitigation method can only be validated for memorization types that can be measured. The Future Directions Engine (5) identifies deployment contexts where existing measurements cannot be applied (e.g., conversational agents where standard prompting-based extraction fails) and where new taxonomic regions become relevant (e.g., RAG where parametric memorization interacts with retrieval context).

3.3 Roadmap for the Deep Dive

First, the data selection pipeline (Section I-B)—because the 120 papers that constitute the survey's evidence base were not collected arbitrarily. Understanding the inclusion/exclusion criteria, the dual-source search strategy (Google Scholar + arXiv), and the rejection criteria explains why certain model families (Pythia, GPT-Neo), datasets (PILE), and measurement conventions (greedy decoding, 50-token match length) dominate Table III, and what systematic biases this may introduce into the survey's conclusions.
Second, the three-dimensional taxonomy (Section II)—granularity, retrievability, desirability—because this is the survey's central intellectual contribution and the vocabulary that every subsequent section uses. Each dimension is a spectrum with formally defined categories, and understanding the definitions (especially formal ones like "verbatim memorization," "extractable memorization," and "k-discoverable memorization") is prerequisite to understanding why different measurement methods produce different numbers.
Third, the measurement catalog (Section III)—because measurement is where the taxonomy meets the real world. A theoretically defined category (e.g., "approximate memorization") is only useful if it can be operationalized, and each measurement method (string match, exposure, MIA, extraction attacks, counterfactual memorization, heuristic compression) operationalizes different subsets of the taxonomy with different cost/reliability tradeoffs.
Fourth, the factor taxonomy (Section IV)—because the factors that cause memorization determine what a mitigation strategy must counteract. Understanding that duplication amplifies memorization non-linearly (10 appearances produce 1000× more frequent generation) and that model size scales memorization log-linearly provides the mechanistic rationale for why deduplication and differential privacy target what they target.
Fifth, the mitigation taxonomy (Section V)—organized by intervention point because intervention timing determines what information is available (data-level methods have access to the full corpus; post-training methods only have the trained model; inference-time methods cannot modify weights). Each method's capabilities and limitations follow from its position in this pipeline.
Sixth, the gap analysis (Section VI)—the prescriptive component that synthesizes information from all prior sections. The four contexts identified (conversational agents, RAG, multilingual models, diffusion models) are selected because they simultaneously (a) are high-stakes deployment scenarios, (b) have structural properties that make standard measurement protocols inapplicable, and (c) currently lack systematic memorization studies. The reasoning for each context draws directly on the taxonomy: for example, conversational agents require novel measurement because existing extractable memorization attacks are calibrated for base models, not instruction-tuned chatbots with safety filters.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a survey paper whose core idea is that memorization in LLMs is not a single phenomenon but a multi-dimensional spectrum, and that systematically organizing the literature along three orthogonal axes—granularity (what is remembered), retrievability (how it can be accessed), and desirability (whether it causes harm)—enables researchers to match measurement methods, causal explanations, and mitigation strategies to the specific kind of memorization they care about, rather than relying on one-size-fits-all approaches that create "a false sense of privacy."

Data Selection Pipeline: Building the Evidence Base

The survey's conclusions are only as reliable as the papers it includes, and Section I-B documents a structured, multi-stage selection process designed to capture the full scope of LLM memorization research while filtering out work that is tangentially related or methodologically unvalidated. The process is explicitly not a systematic review with pre-registered inclusion criteria applied automatically—it is a hybrid approach that combines algorithmically retrieved papers with expert-curated foundational works and reference-chained adjacent-area papers, reflecting the reality that memorization research spans multiple communities with inconsistent keyword usage.

Stage 1: Foundational seed papers. The process begins with "14 papers" identified as "the most cited early studies by recognized researchers in the field." These are not named individually in Section I-B—they must be inferred from the references cited throughout the survey and the categorization in Table III—but they include the highly-cited works that established the core definitions and empirical findings: Carlini et al. (2019) for the exposure metric and secret sharer framework, Carlini et al. (2021) for the first large-scale extraction attack on GPT-2, Carlini et al. (2023) for the systematic quantification of discoverable memorization across model scales, Kandpal et al. (2022) for the duplication-memorization link, and Lee et al. (2022) for deduplication's effectiveness. These seed papers serve two functions: they establish the conceptual core of the survey (providing the definitions around which the taxonomy is built), and they provide the bibliographic entry points for the reference chaining step.

Stage 2: Keyword-based retrieval from two sources. The survey searches two repositories using distinct but overlapping queries, with specific syntactic constraints designed to balance recall and precision:

Google Scholar search: papers that include the term "Memorization" in their title and include "Language Model" in their body. The title constraint is a precision filter—papers with "memorization" in the title are almost certainly about the phenomenon, not merely mentioning it in passing. The body constraint ("Language Model") distinguishes LLM memorization from memorization in biology, cognitive science, or other machine learning contexts. This query returned 460 papers.
arXiv search: papers that include both "Memorization" AND "Privacy" in their abstract, and "language model" in their body. The dual-abstract constraint is a higher-precision filter that targets papers where memorization is studied specifically as a privacy concern—which is the survey's primary focus (undesirable memorization). A keyword that appears in an abstract is guaranteed to be central to the paper's contribution, unlike a keyword that merely appears in the body text. This query returned 185 papers, with 115 papers overlapping with the Google Scholar results.

The two queries are intentionally asymmetric. The Google Scholar query casts a wider net, capturing any memorization-focused paper on language models (including those studying memorization for performance, knowledge retention, or generalization). The arXiv query is privacy-specific. Together, they ensure coverage of both the broad memorization literature and the narrower privacy-focused subset. The overlap count (115 papers) confirms that many papers appear in both, but the non-overlapping sets (approximately 345 from Google Scholar only, approximately 70 from arXiv only) capture different slices of the literature—for example, memorization-for-performance papers that don't mention privacy in the abstract (Google Scholar only) and privacy papers where "memorization" appears in the abstract alongside "privacy" but the paper's title uses different terminology (arXiv only).

Stage 3: Manual filtering with explicit rejection criteria. The initial pool of 460 + 185 = 645 papers (with 115 duplicates removed, yielding approximately 530 unique papers) is manually filtered using two criteria: (1) remove papers that are "non-relevant" or "out of scope for 'undesirable memorization'" or with a "focus on deep learning" rather than LLMs specifically; (2) remove papers that "were neither published at a scientific venue nor had any citations." The second criterion is a quality floor—it excludes preprints that have not passed peer review and have not influenced the field (zero citations), acknowledging that the arXiv-to-publication pipeline includes many papers that never receive community validation. A citation count of zero is a noisy signal (new papers on important topics may not yet be cited), but combined with "unpublished," it captures work that has not entered the research conversation.

This manual filtering process is documented in Figure 1, which shows the narrowing: 530 papers (combined pool) plus 14 seed papers = 544, filtered to 62 papers through manual review, then expanded back to 120 papers through reference chaining and adjacent-area inclusion. This means that more than half of the final paper set (approximately 58 papers) entered through reference chaining rather than keyword search—a pattern that is typical for surveys on emerging topics where terminology is not yet standardized.

Stage 4: Reference chaining and adjacent-area inclusion. The survey explicitly includes works "on data extraction, membership inference, and other forms of data leakage, which intersect with the broader concept of memorization." These papers were identified through (1) tracing citations backward from the core 62 papers to find precursor work, (2) tracing citations forward to find subsequent work that builds on the core papers, and (3) consulting "sections of existing survey articles" to identify papers that those surveys considered central. This stage is crucial because many measurement methods (membership inference attacks, extraction attacks) were originally developed in the security and privacy communities without explicit framing as "memorization" measurement, yet they operationalize the same underlying concept—the model's retention and potential disclosure of training data. Excluding these adjacent papers would artificially narrow the measurement catalog (Section III) to only methods that self-describe as memorization measures, missing tools that are widely used for memorization assessment in practice.

The final corpus of 120 papers is what the survey analyzes. The rejection rate from the initial keyword-based pool (~530 papers) to the core filtered set (62 papers) is approximately 88%, indicating that keyword search alone captures many tangentially related papers—likely because "memorization" as a term appears in passing in many deep learning papers without being the central focus, or because biology papers that coincidentally mention "language model" in their body are captured by the Google Scholar query. The reference-chaining expansion from 62 to 120 papers indicates that the keyword-based search alone would have missed nearly half the relevant literature, validating the hybrid approach.

The Three-Dimensional Taxonomy: Granularity, Retrievability, Desirability

The survey's central intellectual contribution is the taxonomy presented in Section II and visually summarized in Figure 3. This taxonomy is not merely a classification exercise—it is a matching framework that reveals when measurement methods, mitigation strategies, and risk assessments are aligned (or misaligned) with the specific type of memorization they need to address.

The three dimensions are introduced in sequence in Section II (granularity in II-A, retrievability in II-B, desirability in II-C), but they are interdependent: a complete characterization of a memorization instance requires a value on each axis. The "Spectrum of memorization" framing in the section title signals that each axis is continuous or ordinal, not binary. This is a deliberate design choice—binary classification ("memorized vs. not memorized") collapses important distinctions that matter for downstream decisions about whether a risk is acceptable.

The Granularity Dimension (Section II-A)

Granularity answers: what has the model remembered? The dimension has five ordered levels, from most specific (and easiest to detect) to most abstract (and hardest to unambiguously attribute to memorization):

Perfect memorization (Section II-A0a) is an idealized upper bound, defined formally by Kandpal et al. (2022) as: "A model is said to perfectly memorize its training data if the generation frequencies of sequences are the same as their appearances in the training data." The operational meaning is that sampling from the model's output distribution is statistically indistinguishable from sampling uniformly from the training corpus. This definition serves purely as a reference point—no real LLM achieves perfect memorization (real models generate novel sequences and misorder training data frequencies), but the concept provides a theoretical ceiling against which actual memorization rates can be compared. The survey uses it to illustrate that "memorization" is not binary but a degree of approximation to this perfect-memorization ideal.

Verbatim memorization (Section II-A0b) is the most commonly studied form and the one most closely associated with undesirable outcomes (privacy leakage, copyright violation). The formal definition from Carlini et al. (2021) is:

Let $\text{LM}(p)$ denote the output of a language model when prompted with $p$ . A string $s$ from the training set is defined as verbatim memorized if there exists a prompt $p$ such that $\text{LM}(p) = s$ .

where $p$ is the prompt (any string that causes the model to output $s$ ), $s$ is the training data string being checked for memorization, $\text{LM}(p)$ is the model's generated output given prompt $p$ , and the equality $=$ means exact string match between the generated output and the training string.

What it computes: This definition checks whether, for a given training string $s$ , there exists any prompt $p$ that makes the model reproduce $s$ character-for-character. The prompt $p$ can be any string—it doesn't have to be a prefix of $s$ —so the search space for $p$ is combinatorially large. In practice, researchers approximate this by testing prompts that are prefixes of $s$ (often the first 50 tokens), which makes the definition operationally similar to discoverable memorization (Section II-B) when the prefix comes from the training data.

Why this form: The existential quantifier ("there exists a prompt $p$ ") makes this a worst-case definition—the model is considered to have memorized $s$ if any adversary could ever elicit it, regardless of how unlikely the prompt is in normal use. This is appropriate for security analysis (where adversaries craft prompts adversarially) but overestimates practical privacy risk (where typical users don't prompt with prefixes of private documents). A less conservative definition—requiring memorization only when likely prompts reproduce $s$ —would downplay cases where an adversary deliberately exploits the model but would better reflect typical-use risk.

The survey also presents a stricter variant, Exact memorization (Tirumala et al., 2022), defined as:

Let $c = (p, s)$ be a context where $p$ is an incomplete block that is completed by the sequence $s$ . The context $c$ from the training set is considered to be exactly memorized by the language model $\text{LM}$ if $\text{Argmax}(\text{LM}(p)) = s$ .

where $c$ is a training context (prefix-suffix pair from the training data), $p$ is the prefix (incomplete block), $s$ is the ground-truth continuation from the training data, and $\text{Argmax}(\text{LM}(p))$ is the greedy decoding output—the single sequence that maximizes the model's conditional probability, rather than a sampled output.

What it computes: This is the greedy-decoding version of verbatim memorization—the model reproduces the training suffix when forced to choose the single most likely token at each step (no sampling). This is stricter than Carlini et al.'s definition if the model only reproduces $s$ under greedy decoding but not under sampling, and it is narrower in scope because the prompt $p$ must be the exact prefix from the training context, not an arbitrary prompt.

Why this form: Requiring greedy decoding (Argmax) rather than any possible sampling outcome eliminates the stochasticity of the sampling process as a confound. If a model outputs $s$ under greedy decoding, that means $s$ is the most likely continuation under the model's learned distribution, which is stronger evidence of memorization than occasional reproduction under high-temperature sampling. However, this also means the metric misses memorization that is latent—sequences that have high probability but are not the maximum probability continuation—which could still be extracted through techniques like beam search or nucleus sampling with carefully tuned parameters.

A critical operational detail that the survey highlights: "these definitions do not impose any restrictions on the length of the prompt $p$ or the generated output $s$ . However, if the generated text is too short, it may not be appropriate to classify it as memorization." In practice, the community has converged on a 50-token match length convention (both prefix and suffix are typically 50 tokens), as documented in Table III. This is a pragmatic threshold: 50 tokens is long enough that random chance reproduction is astronomically unlikely (the number of possible 50-token sequences in a typical vocabulary of 50,000 tokens is $50000^{50}$ , making collisions effectively impossible without memorization), but short enough that the measurement is computationally feasible (you can check millions of 50-gram suffixes against a training corpus). Different studies have used different lengths (see Table III: prefix lengths range from 32 to 450 tokens, match lengths from 32 to 1000 tokens), and these choices affect measured memorization rates—longer matches are harder to extract but provide stronger evidence of memorization when found.

Approximate memorization (Section II-A0c) extends the verbatim concept to allow for minor variations. The formal definition (from Ippolito et al., 2023) is:

A suffix $s$ for prefix $p$ is labeled as approximately memorized if for generation $g = \text{LM}(p)$ , $\text{Sim}(g, s) > \delta$ , where $\text{Sim}(\cdot, \cdot)$ is a textual similarity metric.

where $s$ is the ground-truth training suffix, $p$ is the prefix used to prompt the model, $g$ is the model's generated continuation, $\text{Sim}(\cdot, \cdot)$ is a similarity function (Ippolito et al. use BLEU, Duan et al. use token-wise Levenshtein distance, Peng et al. use Minhash based on Jaccard similarity), and $\delta$ is a threshold (Ippolito et al. use 0.75 for BLEU, chosen "based on qualitative analyses of the samples").

What it computes: Instead of requiring exact character-for-character match, this definition counts a training example as memorized if the model's output is similar enough to the training suffix, where "similar enough" is defined by a threshold on a text similarity metric. The threshold is the key hyperparameter: set it too low and unrelated text will be incorrectly flagged as memorized (false positives); set it too high and genuinely memorized-but-paraphrased text will be missed (false negatives).

Why this form: The motivation, as the survey explains, is that verbatim memorization is "too confined" and "does not capture the subtler forms of memorization." Two sentences that differ by punctuation, spelling variations (American vs. British English), or trivial word substitutions would be considered completely different by exact string match, but a human judge would recognize them as the same memorized content. Ippolito et al. (2023) demonstrate empirically that "the measurement of memorization can increase by a factor of two compared to only considering verbatim memorization." However, the survey notes the limitation in Ippolito et al.'s own characterization: "this definition can lead to both false positives and false negatives when compared to human judgment, indicating a potential direction for future investigations." The choice of similarity metric and threshold is application-specific—BLEU with a 0.75 threshold works for the prose generation tasks Ippolito et al. studied, but code generation might require different similarity functions (e.g., abstract syntax tree match) because minor syntactic variations in code can be semantically identical.

Entity-level memorization (Section II-A0d) shifts from string-level to relational memorization. The formal definition from Zhou et al. (2023) is:

Let $s$ be a training sample containing a set of entities $\mathcal{M}$ . A prompt $p$ can be constructed to include a strict subset of the entities $\mathcal{N} \subset \mathcal{M}$ . The model is said to show entity memorization if, when prompted with $p$ , it outputs a response containing some entities in $\mathcal{M} - \mathcal{N}$ .

where $s$ is a training sample (e.g., a document mentioning multiple entities), $\mathcal{M}$ is the complete set of entities mentioned in $s$ , $\mathcal{N}$ is a proper subset of $\mathcal{M}$ used to construct the prompt $p$ (the "clue" entities), and $\mathcal{M} - \mathcal{N}$ are the entities not in the prompt but present in the training sample (the "target" entities).

What it computes: This definition detects cases where the model reveals associations between entities that were co-mentioned in training data, even if the exact phrasing differs. The key insight is that prompting with a subset of linked entities (e.g., "John Smith") causes the model to output other entities linked to them (e.g., "012345678") through patterns learned during training, not through direct string memorization. The survey's example illustrates this clearly: "John Smith's phone number is 012345678" appears in training; if an adversary prompts with "I called John Smith yesterday and entered this number into my phone:" and the model outputs "012345678," this is entity-level memorization even though the generated sentence is novel—the model has learned and disclosed the association between the name and the number.

Why this form: Entity-level memorization captures privacy risks that slip through verbatim and approximate memorization checks. An exact string match test would not flag this as memorization (the output sentence is new), yet the privacy violation is equally severe. The survey notes that Kim et al. (2023) found this sensitivity varies by PII type—"emails and phone numbers show higher extraction rates compared to addresses"—likely because emails and phone numbers have distinct formats that serve as strong retrieval cues, while addresses have more common surface forms. This dimension matters practically because privacy regulations (GDPR, CCPA) protect personal data regardless of how it is formatted—a model that reveals a phone number through entity-association memorization is violating privacy even if it never reproduces the exact training sentence.

Content memorization (Section II-A0e) is the broadest and least formally defined category, encompassing "reproducing or inferring general content from the training data, rather than focusing on exact strings." The survey lists sub-types: factual memorization (reproducing factual knowledge in different words), conceptual memorization (recalling underlying concepts without matching original phrasing), and knowledge memorization (retaining information learned from training data). Unlike the previous categories, content memorization is not formally defined with a mathematical condition—it is characterized by what it captures that the other categories miss. The survey notes this category has both desirable implications (generalization, reasoning) and undesirable ones (propagating misinformation from unverified training data, as highlighted by Bender et al., 2021).

The Retrievability Dimension (Section II-B)

Retrievability answers: how can the memorized content be accessed? This dimension captures the practical distinction between information that is stored in the model's parameters (latent memorization) and information that can be extracted by an adversary with limited knowledge.

Extractable memorization (Section II-B0a) is defined by Carlini et al. (2023) from the adversary's perspective:

Let $\text{LM}(p)$ denote the output of a language model when prompted with $p$ . An example $s$ from the training set $\mathcal{S}$ is extractably memorized if an adversary (without access to $\mathcal{S}$ ) can construct a prompt $p$ that makes the model produce $s$ (that is, $\text{LM}(p) = s$ ).

where $s$ is a training example, $\mathcal{S}$ is the full training set (which the adversary cannot access), $p$ is a prompt constructed by the adversary without knowledge of $\mathcal{S}$ , and the output matches $s$ exactly.

What it computes: This definition operationalizes the real-world threat model where an attacker does not know what was in the training data but wants to extract it. The adversary must guess or discover prompts that trigger reproduction. The survey explains two practical challenges: "designing prompts that best elicit memorization in a model" (the adversarial prompt engineering problem) and "verifying if the model output is indeed from the training data" (the attribution problem—without access to $\mathcal{S}$ , the adversary must verify through external means like Google search, as Carlini et al. (2021) did for GPT-2, or assume correctness based on statistical properties of the output).

Why this form: This captures the risk of memorization more accurately than discoverable memorization because it models what an actual attacker can do. Discoverable memorization (giving the model its own training prefixes) represents an upper bound on what could be extracted if the attacker had perfect knowledge of the training data—but real attackers don't have that knowledge. The survey reports that Carlini et al. (2021) confirmed memorization of only "about 0.00001% of GPT-2's training data" through extractable methods with manual Google search verification, while Nasr et al. (2023) found "0.1% to 1%" using automated large-scale verification with a combined dataset proxy for the training data. The 100× to 10,000× difference in measured rates between these studies reflects the difference in verification methodology (manual vs. automated, strict vs. proxy-based) and model scale (GPT-2 vs. Pythia/GPT-Neo), not necessarily a difference in underlying memorization—underscoring why measurement methodology matters.

Discoverable memorization (Section II-B0b) is defined more directly from the training data perspective:

Let $\text{LM}(p)$ denote the output of a language model when prompted with $p$ . For a context $c = (p, s)$ from the training set $\mathcal{S}$ , we say that $s$ is discoverably memorized if $\text{LM}(p) = s$ .

where $c = (p, s)$ is a context (prefix-suffix pair) that appears in the training data $\mathcal{S}$ , $p$ is the prefix from that training context, $s$ is the suffix that followed $p$ in training, and the model's output when given the training prefix $p$ exactly matches the training suffix $s$ .

What it computes: This checks a counterfactual: if we give the model the beginning of a sequence it was trained on, does it reproduce the exact continuation from the training data? This is the most straightforward operationalization of memorization—it directly tests whether the model has internalized specific sequences from its training data. The $k$ -discoverable variant (from Biderman et al., 2023) adds a constraint: "string $s$ is said to be $k$ -discoverable if $s$ is discoverable AND $p$ is consisted of $k$ tokens." Different values of $k$ provide different lenses on memorization depth—small $k$ (e.g., 5 tokens) tests whether the model can complete very short prompts with training continuations (easier, higher measured rates), while large $k$ (e.g., 50 tokens) tests reproduction with substantial context (harder, but more convincing evidence of verbatim storage).

Why this form: This definition provides an upper bound on memorization—"Nasr et al. (2023) assume that discoverable memorization provides an upper bound for data extraction" because if the model cannot reproduce training data even when given the exact training prefix, it certainly cannot reproduce it under weaker prompts. However, the survey notes the practical limitation: "this method requires exhaustively interacting with the model by inputting different prompts and comparing the model output to the training data." The number of possible prefixes is combinatorially large, so in practice, researchers sample prefixes (e.g., from a held-out subset of the training data) and measure discoverability rates on that sample, then extrapolate. Carlini et al. (2023) found that "LLMs discoverably memorize roughly 1% of their training datasets" when prompting with about 50 tokens of context, with Anil et al. (2023) and Kudugunta et al. (2023) corroborating this rate on PaLM and MADLAD-400 respectively. This 1% figure is a central empirical result cited throughout the survey.

The survey further highlights a critical finding from Nasr et al. (2023) comparing extractable and discoverable memorization for the same model (GPT-Neo 6B): "some sequences are both discoverably and extractably memorized; some sequences are discoverably memorized but not extractably memorized, and vice versa; the overlap between these two types of memorization provides insights into the model's information retention and retrieval mechanisms." This means discoverable memorization is neither a superset nor a subset of extractable memorization—they measure partially overlapping but distinct phenomena. A sequence might be discoverable (the model reproduces it when given the training prefix) but not extractable (no adversarial prompt without training-data knowledge triggers reproduction), indicating the information is stored but not accessible through general prompts. Conversely, a sequence might be extractable but not discoverable—perhaps the adversarial prompt triggers a different retrieval pathway than prefix completion. This non-containment relationship is important: mitigation methods validated only against discoverable memorization may not protect against real-world extractable attacks.

A crucial clarification in the survey's Remark at the end of Section II-B addresses a potential confusion: "one might perceive extractable/discoverable memorization and other concepts within the granularity dimension as equivalent. However, it is important to distinguish them. Extractable and discoverable memorization emphasize the methods used to retrieve memorized samples. In the case of granularity of memorization, the retrieval method is irrelevant; the primary concern is what granularity of the data has been memorized, irrespective of how it is produced by the model." In other words: granularity describes what is stored (exact string, entity relationship, general concept); retrievability describes how it can be accessed (with knowledge of training prefixes or without). Each granularity level can be combined with either retrievability regime—for example, "entity-level × extractable" means an adversary without training data access can extract entity associations, while "entity-level × discoverable" means entity associations are retrievable only when prompted with specific training prefixes. Current studies rarely distinguish these combinations systematically.

The Desirability Dimension (Section II-C)

Desirability answers: does this memorization cause harm? This is the most context-dependent and stakeholder-relative dimension, and the survey explicitly frames it as introducing nuance rather than providing a definitive classification.

Undesirable memorization (Section II-C0a) is characterized by five concrete risk categories:

Privacy risks: models may "inadvertently memorize and potentially reveal sensitive personal information present in the training data" (citing Henderson et al., 2023).
Security vulnerabilities: memorized "confidential information like API keys or passwords could lead to security breaches" (citing Carlini et al., 2019; Ramaswamy et al., 2020; Huang et al., 2024).
Copyright violation: memorized copyrighted material "may lead to legal challenges, especially if the model can reproduce substantial portions of protected works" (citing Henderson et al., 2023; Mueller et al., 2024; Freeman et al., 2024; Cooper and Grimmelmann, 2025).
Bias and fairness: "based on the distribution of the data, memorization could introduce bias issues in the model output."
Performance degradation: two sub-types—"overperformance on the benchmarks" where memorized test sets inflate apparent capability (Elangovan et al., 2021; Bordt et al., 2024), and "performance degradation on specific contexts" where "models lean on memorized content" for tasks requiring reasoning, producing incorrect outputs due to "reliance on outdated memorized knowledge" (Du et al., 2025; Prashanth et al., 2024; Hong et al., 2025).

Desirable memorization (Section II-C0b) is characterized by four benefit categories:

Transparency and auditing: "when models retain recognizable traces of their training data, users and auditors can directly test whether specific information was included in the training corpus," enabling data-provenance checks, accountability, and verification of whether personal data was used.
Knowledge retention: "LLMs are often deliberately designed to retain facts, concepts, and general knowledge to enhance their performance in tasks like question answering and information retrieval" (citing Chen et al., 2023; Lu et al., 2024).
Language generation: "deliberate memorization of linguistic patterns, grammar rules, and vocabulary is crucial for the model's ability to generate coherent and contextually appropriate text."
Alignment goals: "intentional memorization can be utilized in the AI alignment phase to inject desired behaviors and values into the model" (citing Ouyang et al., 2022).

The survey's critical theoretical move is the Remark at the end of Section II-C that reframes desirability from a property of the memorization instance to a property of the stakeholder-task-context tuple:

"While the distinction between undesirable and desirable memorization is conceptually useful, the practical implications are more nuanced. Many effects of memorization depend on who is evaluating the model (e.g., model developers, end-users, data subjects, copyright holders) and in which context the memorized content appears... A behavior that benefits general users can simultaneously disadvantage copyright holders: memorization that enables auditing and data-provenance checks for users may expose sensitive information from data subjects; memorization that stabilizes alignment behaviors may also reinforce existing biases; memorization that improves factual recall may increase the risk of generation of false information due to outdatedness. In this sense, even when memorization provides functional advantages, it always carries an undesirable risk."

This is a structural claim about memorization: there is no such thing as purely desirable memorization, because the same model capacity that enables factual recall can be exploited to extract private information, and the same training data that teaches useful patterns also encodes harmful biases. The desirability dimension is therefore not a clean partition but a tension that must be managed through application-specific risk assessments and mitigation strategies that are themselves evaluated against multi-stakeholder criteria.

The three dimensions are visually represented in Figure 3 as a cube where each axis corresponds to one dimension. The survey notes that the axes are orthogonal in principle (any combination is theoretically possible), but in practice, the literature exhibits strong correlations: verbatim memorization is most often studied in the extractable × undesirable corner; content memorization is most often studied in the discoverable × desirable corner. The empty regions of the cube (e.g., "verbatim × extractable × desirable" or "entity-level × discoverable × desirable") represent under-explored combinations that the survey flags implicitly as research gaps—cases where memorization might provide transparency benefits (desirable) without privacy risk if the memorized content is non-sensitive public knowledge, but where no measurement methodology exists to isolate that specific combination.

Measurement Methods Catalog: Operationalizing the Taxonomy (Section III)

Section III maps five classes of measurement methods to the taxonomic dimensions, explaining what each method actually measures, what it misses, and how its operational constraints create systematic blind spots. The survey does not introduce new measurement methods; it organizes existing ones by their position in the taxonomy and their practical limitations.

String match (Section III-A) is the most direct operationalization of verbatim memorization. The method involves "exhaustively interacting with the model by inputting different prompts and comparing the model output to the training data," then computing "attack success rate... by dividing the portion of memorized text by the size of the training data." The survey notes the obvious limitation: "since there are infinite combinations of tokens that one can feed the model, this method usually falls short in providing the accurate amount of memorization, rather it can provide a good approximation on memorization lower bound" (citing Nasr et al., 2023; Hayes et al., 2025; Carlini et al., 2019). This is inherently a lower-bound estimator—the true memorization rate could be higher than what string matching finds because (a) the search over prompts cannot be exhaustive, (b) the search over training data cannot be exhaustive (for practical datasets), and (c) the exact-match criterion misses approximate memorization. The survey notes that exact match is "too sensitive to small perturbations in the generation"—a single token difference between the model's output and the training text results in a false negative—which is why approximate memorization (Section II-A0c) and approximate match metrics (using BLEU, Levenshtein, or other similarity functions) were introduced to "capture more memorized samples by bypassing negligible changes in the outputs" (Ippolito et al., 2023).

Table III documents the dominant conventions for string match in practice: 50-token match length, 50-token prefix length, and greedy decoding as the standard generation method. The survey does not argue that these conventions are optimal—only that they are consistent enough to enable comparison across studies that use them, and that deviations from these conventions (different prefix lengths, different decoding methods) produce different measured rates that cannot be directly compared.

Exposure metric (Section III-B) is a more nuanced quantification proposed by Carlini et al. (2019) that measures not just whether a sequence is reproduced but how strongly the model favors it over random alternatives. The formal definition:

$\text{exposure}_\theta(s[r]) = \log_2 |\mathcal{R}| - \log_2 \text{rank}_\theta(s[r])$

where $\theta$ represents the model parameters, $s[r]$ is a "canary" sequence—a known secret string injected into training data with a random identifier $r$ , $\mathcal{R}$ is the space of all possible random identifiers (the set from which $r$ was drawn uniformly), $\text{rank}_\theta(s[r])$ is the rank of the target canary $s[r]$ when all possible canaries (with different $r$ values) are sorted by the model's perplexity—lower perplexity means the model considers the sequence more likely, so lower rank means stronger memorization.

What it computes: The exposure metric measures how much more likely the model considers the actual training canary compared to a random guess. $\log_2 |\mathcal{R}|$ is the log of the total number of possible canaries—if identifiers are drawn from a space of size $|\mathcal{R}|$ , a random guess would have rank $|\mathcal{R}|/2$ on average, corresponding to an exposure of $\log_2 |\mathcal{R}| - \log_2(|\mathcal{R}|/2) = 1$ . High exposure (e.g., 10+ bits) means the training canary is ranked far higher than random chance would predict, indicating strong memorization. Exposure of 0 means the training canary is ranked no better than random—no detectable memorization.

Why this form: The log-ratio design ensures that exposure is measured in bits of information—intuitively, exposure of $b$ bits means the model's output distribution over canaries contains roughly $b$ bits of information about which specific canary was in the training data, out of the $\log_2 |\mathcal{R}|$ total bits needed to uniquely identify it from the random space. The "canary extraction test" (Carlini et al., 2019) uses this metric by inserting known secret numbers as canaries into the training data and measuring their exposure after training. The advantage over string matching is that exposure is a continuous measure—it captures not just whether the canary is reproducible but how much the model's internal probability distribution favors it, which provides a more graded assessment of memorization risk. The survey also notes Helali et al. (2020)'s "d-exposure" variant for discriminative tasks where perplexity-based ranking is not applicable.

Inference attacks (Section III-C) are categorized into two types that serve as proxy measures for memorization:

Membership Inference Attacks (MIA, Section III-C0a) aim to "determine whether a specific data point was part of the model's training set" (Shokri et al., 2017). The survey explains that MIA on LLMs "typically involves determining the model's confidence in a given text and using it as an indicator of whether the text was part of the training data. This is often done using the log-likelihood of the sequence tokens" (citing Mattern et al., 2023; Shachor et al., 2024; Shejwalkar et al., 2021; Jagannatha et al., 2021; Wang et al., 2022; Song and Shmatikov, 2019; Wang et al., 2024; Wei et al., 2025; Xie et al., 2024; Satvaty et al., 2025; Zhang et al., 2025). The operational logic: if a model assigns higher probability to a sequence that was in its training data than to similar sequences that were not, then the model has "memorized" that sequence in a way that is detectable by an adversary. MIA accuracy (the fraction of training vs. non-training sequences correctly classified) becomes a proxy for memorization—higher MIA accuracy implies more memorization, because the model's output distribution carries more information about training set membership.

Extraction attacks (Section III-C0b) "attempt to extract specific pieces of information from the model that were present in its training data" through various prompting strategies (Nasr et al., 2023; Hayes et al., 2025; Lukas et al., 2023; Wang et al., 2024; Ishihara, 2023; Parikh et al., 2022). Unlike MIA, which only determines whether a sequence was in training, extraction attacks aim to recover the content itself. The extraction success rate (fraction of prompts that yield verifiable training data) serves as a measure of extractable memorization.

The survey notes that "models that are more susceptible to these attacks are generally considered to have higher levels of memorization," establishing inference attack success as a proxy measurement. The limitation is that attack success depends on both the model's memorization and the attacker's skill—a weak attack may fail to extract memorized information that a stronger adversary could recover, leading to underestimation. Different studies use different attack methodologies (varying prompt construction, decoding strategies, and verification procedures), making cross-study comparison of absolute extraction rates unreliable.

Counterfactual memorization (Section III-D) proposed by Zhang et al. (2023) measures memorization by "how the presence or absence of a sample of the dataset affects the performance of the model on the same sample." The methodology creates subsets of a larger dataset, fine-tunes LMs on each subset, partitions subsets into IN (containing the target sample) and OUT (not containing it), and "test[s] and report[s] the performance on the IN and OUT group of models by averaging." The difference in the model's ability to generate or score the target sample between IN and OUT conditions is the counterfactual memorization measure.

What it computes: For each target sample $x$ , the counterfactual memorization is the difference in the model's loss or generation probability on $x$ when trained with $x$ (IN) versus without $x$ (OUT). A large difference means the model's behavior on $x$ is strongly dependent on having seen $x$ during training—strong memorization. A small difference means the model would have learned to handle $x$ similarly even without seeing it—generalization rather than memorization.

Why this form: This definition has formal similarities to differential privacy (where a mechanism's output distribution must be nearly identical on neighboring datasets differing by one sample), but it is an empirical measurement rather than a theoretical guarantee. The survey notes that Zhang et al.'s experiments on 400 trained models revealed that "counterfactually memorized data are generally unconventional texts such as all-caps, structured formats (i.e., tables or bullet lists), and multilingual texts"—datapoints that are rare or out-of-distribution relative to the overall training distribution are more likely to be memorized counterfactually because the model cannot generalize to them from other training examples.

Heuristic methods (Section III-E) represent an alternative measurement paradigm based on information-theoretic properties of memorized outputs:

Adversarial Compression Ratio (ACR) from Schwarzschild et al. (2024) "evaluates whether a string from the training data is memorized by determining if it can be elicited with a significantly shorter adversarial prompt," using the GCG algorithm (Zou et al., 2023) to find the most compressed prompt. The intuition connects to Kolmogorov complexity: if a long training string can be reproduced from a very short prompt, the model must have stored the string internally (the prompt doesn't contain enough information to specify the output otherwise). The method is adversarial because it optimizes the prompt for compression, not for naturalness—the found prompt may be nonsensical to humans but effective at triggering memorized output.

Soft token sparsity from Feng et al. (2025) "operate[s] on the premise that LLMs generate memorized content with higher confidence and determinism." When generating memorized sequences, the model's next-token probability distribution is sharply peaked (low entropy, high sparsity) because the model "knows" the exact continuation from training. When generating novel content, the distribution is flatter (higher entropy, lower sparsity). Token sparsity thus serves as a proxy for detecting memorized generations without requiring comparison to the training data—useful when the training data is not available for verification.

Factor Taxonomy: What Drives Memorization (Section IV)

Section IV categorizes 14+ influencing factors into three groups—model-related, training pipeline, inference-time—with specific quantitative relationships documented where the literature provides them. The survey summarizes these in Table II with "Key findings" and representative citations for each factor.

Model capacity (Section IV-A1): The dominant quantitative finding is the log-linear relationship between model size and memorization from Carlini et al. (2023): "the relationship between model size and memorization grows consistently on a log-linear scale, with larger models memorizing a greater portion of the data." Tirumala et al. (2022) additionally show that "larger models not only memorize more but do so faster in the training process." The relationship is specifically log-linear—each doubling of model parameters increases memorization by a multiplicative factor rather than an additive amount. The mechanism is intuitive: larger models have more parameters to store individual training examples as part of their learned representations.

The survey also references theoretical work on memorization capacity: Mahdavi et al. (2024) show that "the number of memorized sequences scales with the number of heads and context size under specific assumptions," and Kim et al. (2023) "quantify the theoretical lower bound of memorization capacity of Transformers on sequence-to-sequence mappings." These theoretical results provide lower bounds—they prove how much memorization is at minimum possible, not how much actually occurs in practice.

An important qualification from Carlini et al. (2023): "while memorization increases with model size, it does not necessarily correlate with improved performance... by comparison of models with similar capacities but differing performance levels because of their architectures." This decouples memorization from capability—a larger model that memorizes more is not necessarily better at generalization, and a smaller model that memorizes less may perform equally well on downstream tasks if it learns better abstractions.

Training data characteristics (Section IV-B1): The central finding is the non-linear effect of duplication from Kandpal et al. (2022): "a sequence appearing 10 times in the training data is, on average, generated approximately 1000 times more frequently than a sequence that appears only once." The survey highlights this as a super-linear relationship—10× more duplication leads to 1000× more generation, not 10×. This steep scaling means that even modest deduplication can substantially reduce memorization rates: Lee et al. (2022) show "models trained on deduplicated data would produce memorized text ten times less frequently" (in their no-prompt generation experiment).

Additional data-characteristic findings include Tirumala et al. (2022)'s observation that "nouns and numbers are memorized significantly faster than other parts of speech, likely because they serve as unique identifiers for specific samples"—content words that carry high information content about the specific training instance are memorized more readily than function words that are generic across many examples. Duan et al. (2024) show that "data with lower z-complexity leads to faster decreases in training loss, as more compressible patterns are memorized more quickly"—simpler sequences (repeated patterns, structured formats) are easier for the model to store verbatim.

Prashanth et al. (2024) categorize memorization into three types: "recitation (memorization of highly duplicated sequences), reconstruction (generation using learned templates or patterns), and recollection (memorization of rare, non-template sequences)." The survey notes this as evidence that "memorization is a multi-faceted phenomenon influenced by duplication, template patterns, and rarity, requiring nuanced analysis"—a single factor analysis (e.g., measuring only the effect of duplication) misses the distinct mechanisms driving different memorization types.

Training process dynamics (Section IV-B2): Kandpal et al. (2022) and Zhang et al. (2023) show that "memorization grows consistently with the number of training epochs," which is expected as more epochs mean more exposures to each training example. Jagielski et al. (2023) add a timing effect: "examples seen during the earlier stages of training are less prone to memorization and rather they are forgotten over time." This suggests a non-uniform memorization schedule during training—early-exposed examples may benefit from a "first-mover disadvantage" where the model's representations are still too general to encode them verbatim, while later-exposed examples are encoded once the model has developed the capacity for precise storage. Leybzon and Kervadec (2024) observe a U-shaped pattern: "higher memorization rates happen early and late in training, with lower rates mid-training," suggesting that "placing sensitive data in the middle stages of training could reduce its vulnerability to extraction attacks"—a counterintuitive finding with practical implications for data ordering in privacy-sensitive training pipelines.

Forgetting mechanisms (Section IV-B3): Tirumala et al. (2022) observe that forgetting "follows an exponential degradation, reaching a constant value baseline" that "scales with the model size." The exponential decay means that recently seen examples are disproportionately more memorable—memorization decays quickly after exposure but asymptotes to a non-zero baseline of residual memorization. Larger models have a higher baseline, meaning they retain more information indefinitely even after many subsequent training steps.

Jagielski et al. (2023) address forgetting through worst-case measurement: "standard image, speech, and language models do indeed forget examples over time, though non-convex models might retain data indefinitely in the worst case." The "worst case" qualifier is important—average-case forgetting (most examples decay) does not guarantee that every example is forgotten, and an adversary specifically targeting the most-retained examples might still extract information that appeared to be forgotten on average.

Fine-tuning and transfer learning (Section IV-B4): Mireshghallah et al. (2022) evaluate "how different fine-tuning methods—full model, model head, and adapter fine-tuning—vary in terms of memorization and vulnerability to privacy attacks. Their research, using membership inference and extraction attacks, finds that head fine-tuning is most susceptible to attacks, whereas adapter fine-tuning is less prone." Head fine-tuning (freezing the model body and only training classification heads) preserves the base model's memorization, while adapter fine-tuning (adding small trainable modules) may disrupt or overwrite some memorized information.

Zeng et al. (2024) find that "fine-tuned memorization varies significantly depending on the task"—tasks like summarization, dialogue, question answering, and machine translation exhibit different memorization patterns after fine-tuning—and that "multi-task fine-tuning can mitigate memorization risks more effectively than single-task fine-tuning," likely because training on diverse tasks prevents the model from overfitting to the surface form of any single task's training data.

Input and prompting strategies (Section IV-C1): The primary finding is that "longer prompts increase the likelihood of triggering memorized sequences" (Carlini et al., 2021; Kandpal et al., 2022; McCoy et al., 2023). Longer prompts provide more context that narrows the model's output distribution toward specific training continuations. Prompt engineering and prefix tuning (Li and Liang, 2021) have been deliberately used to maximize extraction: Ozdayi et al. (2023) use prompt tuning for "controlling memorized content extraction rates," Wang et al. (2024) introduce "dynamic, prefix-dependent soft prompt approach" that "outperforms previous techniques in extracting memorized content," and Kassem et al. (2024) show that "instruction-tuned models can reveal as much or more training data as base models" through instruction-based prompt optimization. Weller et al. (2024)'s "according-to prompting" explicitly directs "LLMs to ground responses in previously observed text," further increasing extraction.

Decoding methods (Section IV-C2): The survey notes that "greedy decoding" is the dominant choice for extraction studies (Table III confirms this) because it "maximize[s] the regeneration of training data" (Carlini et al., 2021). The mechanism: greedy decoding always selects the single most probable token, and if the training continuation is the most probable under the model's distribution, greedy decoding will reproduce it deterministically. Sampling-based methods (temperature, top-k, nucleus, typical sampling) introduce stochasticity that reduces reproduction rates but may better reflect what an attacker without control over decoding hyperparameters could extract. Yu et al. (2023) experiment with multiple decoding schemes and use "an auto-tuning method... to find the optimal decoding method that yields the maximization of training data reproduction," further confirming that decoding strategy is a lever for controlling (and measuring) memorization rather than a fixed property of the model.

Tokenization (Section IV-A2): Kharitonov et al. (2022) demonstrate that "increasing the sub-word vocabulary significantly affects the model's ability and inclination to memorize training data. Furthermore, models with larger vocabularies are more likely to reproduce training data when given specific prompts." The mechanism: larger vocabularies reduce sequence lengths (each token covers more characters, so fewer tokens are needed to encode a given text), and shorter sequences are easier to memorize because the model sees fewer token-level decisions where it could diverge from the training continuation.

Explainability and interpretability findings (Section IV-A3): The survey documents a rapidly growing subfield studying where and how memorization is represented inside LLMs. Key findings organized by layer analysis: Huang et al. (2024) find that "memorization requires significant repetition, increases in later checkpoints, and is tied to distributed model states and general language modeling capabilities"; Haviv et al. (2023) show that "memory recall follows a two-step process: early layers promote the correct token in the output distribution, while upper layers amplify confidence... memorized information is primarily stored and retrieved in early layers"; Dankers and Titov (2024) find "memorization is a gradual and task-dependent process rather than localized to specific layers... deeper layers contribute more to memorization when models generalize well to new data"; Stoehr et al. (2024) show "memorization in LLMs, while distributed across layers, is driven by distinct gradients in lower layers and influenced by a low-layer attention head focusing on rare tokens." By neuron-level and circuit-level analysis: Chen et al. (2024) identify "clustering of sentences with different memorization scores in the embedding space and observe an inverse boundary effect in entropy distributions"; Huang et al. (2025) show "memorization and generalization in LLMs are encoded in largely distinct subsets of neurons" and that intervention on these neurons "can be selectively amplified or suppressed without substantially affecting overall performance"; Lasy et al. (2025) demonstrate that "verbatim memorization in LLMs is driven by specific neural circuits, with distinct subgraphs responsible for initiating versus sustaining memorized sequences, revealing mechanistic pathways underlying exact data recall."

These findings collectively establish that memorization is not a diffuse property of the entire model but is localized in identifiable components (early layers, specific attention heads, distinct neuron subsets), opening the possibility of targeted interventions (pruning, editing, suppressing) that reduce memorization without degrading general language capabilities—a theme the survey returns to in the mitigation section (Section V-C2 on model editing).

4. Key Insights and Innovations

Innovation 1: A Three-Dimensional Taxonomy That Reveals Memorization Is Not One Phenomenon But a Space of Distinct Risks

The field's dominant assumption before this survey was that memorization could be treated as a roughly unitary concept—models either "memorized" training data or they didn't, and measuring memorization meant counting how often generated text matched training text verbatim. This assumption is visible in the fact that approximately 60% of studies surveyed (Table III) measure verbatim discoverable memorization using string matching with 50-token prefixes and greedy decoding, as though this protocol captures the phenomenon exhaustively.

This survey makes the fundamental conceptual move of decomposing memorization into three orthogonal, independent axes—granularity (what is remembered, from perfect reproduction down to content-level associations), retrievability (how it can be accessed, extractable vs. discoverable), and desirability (whether it causes harm, which the survey argues is inherently stake holder-relative). The taxonomy is not merely a classification exercise. It is a diagnostic instrument: it reveals that different research communities have been studying different regions of this 3D space without recognizing it, producing incompatible findings that appear contradictory only because they were measuring different things.

Consider the granularity axis alone. Verbatim memorization (exact string match) captures privacy risks from exact document reproduction but misses entity-level associations—a model revealing that "John Smith" is associated with phone number "012345678" through a novel sentence that never appeared in training. Approximate memorization captures near-verbatim reproductions with minor variations (BLEU > 0.75, per Ippolito et al., 2023) and doubles measured memorization rates compared to exact match, yet two sentences differing by a single punctuation mark would be flagged as "not memorized" by exact match and "memorized" by approximate match. Content memorization captures factual recall in paraphrased form, which is desirable for question-answering but problematic when the facts are outdated or false (Bender et al., 2021). These are not subtle gradations of the same thing—they are qualitatively different phenomena with different detection requirements, different risk profiles, and different mitigation responses.

The retrievability axis makes a distinction that prior work often collapsed: extractable memorization (adversary without training-data access can trigger reproduction) is a real-world threat model, while discoverable memorization (giving the model its own training prefix) is a laboratory upper bound. Nasr et al. (2023) demonstrated that for the same model (GPT-Neo 6B), some sequences are discoverably memorized but not extractably memorized, and vice versa—the two categories are not subsets of each other. A mitigation strategy validated only against discoverable memorization may therefore provide what Ippolito et al. (2023) call "a false sense of privacy" because it leaves extractable memorization intact.

The desirability axis introduces the survey's most theoretically sophisticated move: the Remark at the end of Section II-C argues that desirability is not a property of the memorization instance but of the stake holder-task-context tuple. Model developers, end-users, data subjects, and copyright holders evaluate the same memorization behavior differently. Memorization that enables auditing and data-provenance checks for users (desirable transparency) simultaneously exposes sensitive information from data subjects (undesirable privacy violation). The survey's structural claim is that memorization "always carries an undesirable risk" even when it provides functional advantages—there is no purely desirable memorization, only memorization whose benefits outweigh its risks in a specific deployment context.

Figure 3 visualizes this as a 3D cube, and the survey's implicit argument is that most of the cube is empty. The literature clusters in the verbatim × discoverable × undesirable corner because measurement methods are easiest there (string matching against known training data). Regions like entity-level × extractable × undesirable (Kim et al., 2023's finding that PII extraction rates vary by type) and content × discoverable × desirable (factual recall for QA) are sparsely populated but represent practically critical risk-benefit tradeoffs. The taxonomy thus functions as both a map of what we know and a gap-finding instrument—any combination of axis values without substantial literature is a research opportunity.

This is a fundamental reframing, not an incremental improvement. Prior surveys (Hartmann et al., 2023; Neel and Chang, 2023) acknowledged that memorization has multiple facets but did not provide a systematic, orthogonal decomposition with formal definitions for each category. The survey's taxonomy makes it possible to ask questions that the prior literature could not even formulate precisely: "Is this mitigation method effective against entity-level extractable memorization, or only against verbatim discoverable memorization?" "Does this measurement protocol capture the type of memorization that creates copyright liability in this specific jurisdiction?" The taxonomy transforms memorization from a vague umbrella term into a precise, multi-dimensional risk assessment framework.

Innovation 2: Systematic Documentation of Measurement Fragmentation as a First-Class Finding

The survey could have simply catalogued measurement methods—string matching, exposure metric, membership inference, extraction attacks, counterfactual memorization, adversarial compression, soft token sparsity—and called it a literature review. What makes this innovation distinctive is that the survey treats the inconsistency between these methods as a central finding rather than a methodological nuisance to be smoothed over.

Table III is the key piece of evidence. It catalogs 27 studies by their choice of model family, dataset, decoding method, prefix length, match length, and which granularity/retrievability combination they measure. The table reveals striking systematic patterns: Pythia and GPT-Neo dominate as model families because they are open-source and available at multiple training checkpoints; the PILE dataset dominates because these models were trained on it; greedy decoding dominates because it maximizes reproduction rates; 50-token and 32-token match lengths are the two competing conventions. These are not arbitrary choices—each is optimized for the specific measurement goal of the study—but the cumulative effect is that measurements from different studies are comparing fundamentally different things.

Specifically: approximately 60% of studies measure discoverable memorization (prompting with training prefixes) while 40% measure extractable (prompting without training-data knowledge). These measure different retrievability dimensions. Approximately 60% measure verbatim memorization (exact string match) while 40% split between approximate and entity-level. These measure different granularity dimensions. Prefix lengths range from 32 to 450 tokens. Match lengths range from 32 to 1000 tokens. Decoding methods include greedy, temperature, top-k, top-p, nucleus, and beam search. A study using discoverable-verbatim-greedy-50token on Pythia-6.9B and a study using extractable-approximate-nucleus-100token on GPT-Neo-2.7B are measuring different regions of the taxonomic cube with different instrument sensitivities, yet their reported "memorization rates" are often compared as though they capture the same underlying quantity.

The survey does not resolve this fragmentation—that would require a massive multi-model, multi-metric benchmark study that does not exist—but it diagnoses it with sufficient precision that future work can be designed to avoid the comparison errors that the current literature makes. The taxonomy provides a vocabulary for specifying precisely what a measurement method captures: "This study measures verbatim × discoverable × undesirable memorization with greedy decoding and 50-token match length on Pythia models trained on PILE." A study described this way can be compared to other studies with the same specification and recognized as non-comparable to studies with different specifications. This is a methodological infrastructure contribution—the survey builds the conceptual tools that enable rigorous comparison where none was previously possible.

The significance of this contribution extends beyond academic tidiness. When differential privacy researchers report that DP-SGD with ε = 8 reduces memorization to undetectable levels, but extraction-attack researchers report that the same ε leaves models vulnerable to black-box attacks (as shown by Lukas et al., 2023), the apparent contradiction is partially resolvable by recognizing that they are measuring different types of memorization with different detection sensitivities. The taxonomy provides the conceptual framework for articulating which risk a given ε guards against and which risks remain, enabling more honest and precise privacy claims.

Innovation 3: The "Desirability Is Always Stakeholder-Relative" Thesis

Prior work on memorization operated under an implicit binary: memorization is bad (the privacy and security literature) or memorization is sometimes good (the knowledge-retention literature, e.g., Ranaldi et al., 2023; Chen et al., 2023; Lu et al., 2024). The survey's Section II-C0a and II-C0b enumerate the harms and benefits respectively, but the critical intellectual move comes in the Remark that closes Section II-C:

"Many effects of memorization depend on who is evaluating the model... A behavior that benefits general users can simultaneously disadvantage copyright holders: memorization that enables auditing and data-provenance checks for users may expose sensitive information from data subjects; memorization that stabilizes alignment behaviors may also reinforce existing biases; memorization that improves factual recall may increase the risk of generation of false information due to outdatedness. In this sense, even when memorization provides functional advantages, it always carries an undesirable risk."

This is a theoretical claim, not an empirical finding. It asserts that memorization is inherently dual-use in a stronger sense than the standard "technology can be used for good or ill" framing. The claim is structural: the same model capacity that enables desirable factual recall (Section II-C0b, knowledge retention) is the capacity that an adversary exploits to extract private information (Section II-C0a, privacy risks). You cannot have one without the other because they are not separate mechanisms—they are the same mechanism evaluated by different stakeholders under different threat models. A model with zero memorization would have zero factual knowledge (undesirable for users); a model with perfect factual knowledge would have maximally extractable training data (undesirable for data subjects). The desirability dimension is therefore not a classification axis but a tension axis—every point on the memorization spectrum represents a tradeoff between competing stakeholder interests.

This thesis has practical consequences that the survey does not fully develop but implies. Privacy-preserving training methods (differential privacy, deduplication) that reduce memorization globally may harm performance on knowledge-intensive tasks more than necessary because they treat all memorization as equally undesirable. A stakeholder-aware mitigation strategy would target specific types of memorization (verbatim extractable PII) while preserving others (paraphrased factual knowledge), but this requires measurement methods that can distinguish these types—which brings us back to the taxonomy's role as a precision instrument rather than a blunt classification tool.

This is a fundamental reframing of the desirability question. Prior work acknowledged the existence of both benefits and harms but treated them as separate phenomena (memorization-for-privacy-risk vs. memorization-for-knowledge). The survey's thesis that they are the same phenomenon viewed from different perspectives is a conceptual unification that changes how mitigation strategies should be designed and evaluated.

Innovation 4: A Gap Analysis That Identifies Entire Deployment Contexts Where Memorization Is Unstudied

Section VI of the survey is not a generic "future work" section listing topics that could benefit from more research. It is a targeted gap analysis that identifies four specific deployment contexts—conversational agents, retrieval-augmented generation (RAG), multilingual LLMs, and diffusion language models—where structural properties of the context make existing memorization measurement and mitigation methods inapplicable or unreliable.

What makes this intellectually distinctive is the reasoning for each gap. The survey does not simply observe that these contexts are under-studied (which would be true of many topics). It argues that each context has specific structural features that defeat the assumptions underlying the standard measurement protocols that dominate Table III:

Conversational agents: Instruction-tuned, safety-filtered chatbot models are the dominant production deployment modality, yet Nasr et al. (2023) found that existing "divergence attacks are not powerful enough to stimulate training data reproduction" from these models. The standard extractable memorization measurement (prompt with prefixes, check for verbatim match) was designed for base models that complete text freely. Conversational agents are trained to refuse requests that appear to be seeking training data and to respond in a dialogue format rather than completing prompts. The measurement protocol fails not because conversational agents don't memorize—they are trained on the same data as base models—but because the adversarial access pathway has been blocked by alignment training. Whether this blocking is robust or can be circumvented by more sophisticated attacks (as Kassem et al., 2024, demonstrate with instruction-based prompt optimization) is unknown, and the survey positions this as a measurement-methodology problem requiring novel attack design.
Retrieval-Augmented Generation (RAG): The survey identifies a dual effect that makes RAG memorization qualitatively different from parametric memorization alone. On one hand, RAG can reduce reliance on memorized parametric knowledge because the model can retrieve facts at inference time rather than storing them (potentially reducing undesirable memorization). On the other hand, RAG introduces a new attack surface: the retrieval corpus may contain private documents that the model incorporates into its generation, and the model may reproduce retrieved content verbatim even if it wasn't in the pretraining data (a new form of "memorization" that is context-dependent rather than parameter-encoded). Zeng et al. (2024) provide preliminary evidence of both effects, but no systematic study exists. The survey's framing makes clear that standard pretraining memorization metrics (which measure parametric storage) are the wrong instrument for RAG systems, where memorization can arise from the interaction between retrieval and generation rather than from parameter storage alone.
Multilingual LLMs: Satvaty et al. (2025) and Luo et al. (2025) provide early evidence that memorization is not language-uniform—lower-resource languages exhibit different memorization patterns. The survey connects this to the broader fairness concern: if memorization mitigation methods are developed and validated primarily on English (where most research is conducted, as shown by the PILE dominance in Table III), they may systematically under-protect or over-restrict low-resource languages where data scarcity creates different memorization dynamics. The structural feature that makes this a distinct gap is data imbalance: the same model may see English data hundreds of times more frequently than low-resource language data during training, and the memorization consequences of this imbalance are unexplored.
Diffusion Language Models: The survey notes that Carlini et al. (2023) demonstrated severe memorization in image diffusion models—more severe than in GANs—but that text-domain diffusion models are fundamentally different because "the discrete nature of the text domain" means that "an analysis of the general vision diffusion models may not be applicable." The structural feature here is the continuous-to-discrete gap: image diffusion models operate in continuous pixel space where memorization can manifest as near-exact reconstruction with imperceptible noise, while text diffusion models operate in discrete token space where small perturbations produce semantically different outputs. The survey positions this as an open question requiring independent investigation rather than assuming that findings from the vision diffusion literature transfer.

Each of these four gaps is selected because it simultaneously (a) represents a high-stakes deployment scenario (conversational agents are the dominant LLM interface; RAG is the standard architecture for knowledge-intensive applications; multilingual models serve billions of non-English speakers; diffusion models are an emerging generative paradigm), (b) has structural properties that break the assumptions of existing measurement and mitigation approaches, and (c) currently has zero or near-zero systematic memorization research. This is a prescriptive research agenda grounded in the taxonomy—the survey uses its framework not just to organize what exists but to predict where the next wave of memorization problems will emerge and why current tools will be insufficient.

The significance of this innovation is that it converts Section VI from a token "future work" list into a diagnostic tool for prioritizing research investment. A funding agency or research group can use the taxonomy to identify which contexts have the largest unaddressed risk (by intersecting the contextual gap with the taxonomy's desirability dimension—privacy risks in conversational agents for customer service are more urgent than copyright risks in diffusion language models for art generation, for example) and which measurement tools need to be developed first (extractable memorization attacks for conversational agents before mitigation strategies can be validated).

5. Experimental Analysis

Evaluation Methodology

This section is fundamentally different in structure from the experimental analyses section of the reference example. The paper being analyzed is a survey paper—it does not run original experiments. Instead, it organizes, compares, and interprets other researchers' experiments. The "experimental analysis" must therefore be an analysis of what the aggregated literature shows, not what the survey itself tested. The "methodology" here covers how the survey selected and organized the studies it analyzes; the "results" are the patterns that emerge from that organization; the "ablations" are the methodological choices within individual cited studies that affect measured outcomes; and the "critical assessment" evaluates whether the survey's taxonomic claims are supported by the evidence it presents.

Dataset. The survey does not use a single dataset. The analyzed papers primarily use the PILE (an 825 GB English text corpus used to train Pythia and GPT-Neo models), with secondary datasets including C4, RealNews, LM1B, Wiki40B, the Enron email corpus, RedPajama, RefinedWeb, and Dolma. Table III documents dataset choices per study. The dominance of PILE is functionally a methodological constraint: because Pythia and GPT-Neo families are the most-studied models (due to being open-source with multiple training checkpoints), and those families were trained on PILE, the empirical findings on memorization rates and factors are disproportionately calibrated to English text with PILE's specific duplication and content characteristics. The survey acknowledges this indirectly by calling for multilingual memorization research (Section VI-C) but does not quantify how PILE dominance might bias conclusions.

Base model(s). The analyzed literature concentrates on the Pythia and GPT-Neo model families (Table III), with secondary appearances by GPT-2, GPT-J, OPT, LLaMA, PaLM, GPT-3, T5, and instruction-tuned variants (Alpaca, Vicuna, ChatGPT). Pythia and GPT-Neo are chosen by researchers—not by the survey—because they are open-source and available at intermediate training checkpoints, enabling analysis of memorization dynamics during training (Tirumala et al., 2022; Biderman et al., 2023). Model scales range from 125M to 175B parameters. The survey notes a log-linear relationship between model size and memorization (Carlini et al., 2023) across the studied range, but the concentration on specific model families means that architectural differences in memorization behavior (e.g., attention pattern differences, training objective variations) are under-explored.

Metrics. The survey catalogs six measurement approaches, each operationalizing different regions of the taxonomic cube:

String match rate: fraction of generated suffixes that match training data exactly, reported as a percentage of training data memorized (e.g., ~1% discoverable memorization at 50-token prefix; Carlini et al., 2023).
Exposure: log-ratio measure (bits) comparing training canary rank to random expectation; Carlini et al. (2019) define it as exposure = log₂|R| − log₂ rank(s[r]).
Membership inference accuracy: fraction of training vs. non-training sequences correctly classified by an adversary (Shokri et al., 2017; Mattern et al., 2023).
Extraction attack success rate: fraction of adversarial prompts that yield verifiable training data (Nasr et al., 2023).
Counterfactual memorization: difference in model loss/probability on a sample between IN and OUT training conditions (Zhang et al., 2023); reported as performance differential across 400 trained models.
Heuristic measures: Adversarial Compression Ratio (ACR; Schwarzschild et al., 2024) and soft token sparsity (Feng et al., 2025), reported as compression factors and entropy/sparsity values respectively.

The survey does not propose a unified metric or argue that one metric is superior—it documents that different metrics capture different taxonomic regions and that cross-metric comparison is methodologically invalid without careful alignment.

Baselines. Because this is a survey aggregating results across many studies, "baselines" here refers to the reference points that the surveyed literature uses. Key baselines include:

Perfect memorization (Kandpal et al., 2022): a theoretical upper bound where the model's generation distribution matches the training data distribution—used as a conceptual reference, not an empirically measured quantity.
Random/non-memorized baselines: in exposure studies, the expected rank of a sequence not in training is |R|/2 (exposure ≈ 1 bit); in MIA studies, chance-level classification accuracy (50% for balanced sets).
Non-deduplicated vs. deduplicated training: Lee et al. (2022) compare models trained on C4-Original vs. C4-NearDup vs. C4-ExactSubstr, finding 10× less memorization in no-prompt generation for deduplicated models.
Greedy vs. sampling decoding: Carlini et al. (2021) establish greedy decoding as the standard extraction method; studies that compare decoding methods (Yu et al., 2023) use greedy as the reference point.
No mitigation vs. mitigation strategies: Section V compares deduplication, DP-SGD, training interventions (goldfish loss, alternating teaching), unlearning, and decoding-based methods against their respective no-mitigation baselines on task-specific metrics.

Generation budget / compute accounting. The survey does not standardize compute accounting across studies because the underlying studies use incompatible cost models. String-matching studies measure cost in number of model queries (number of prompts tested), with typical scales of millions of 5-token blocks (Nasr et al., 2023). Exposure studies use the number of canary sequences evaluated (typically hundreds to thousands). MIA studies measure cost in number of shadow models trained (often hundreds, as in Carlini et al., 2023's counterfactual work with 400 trained models). The survey does not attempt to normalize these into a common FLOPs or query budget—it documents the heterogeneity as part of its measurement fragmentation finding (Section III, Table III).

Cross-validation / statistical protocol. The survey does not conduct experiments and therefore does not employ cross-validation. The surveyed papers vary in their statistical rigor. Zhang et al. (2023) train 400 models for counterfactual memorization measurement, providing per-sample confidence estimates. Carlini et al. (2023) report memorization rates with model-scale comparisons across multiple checkpoints. Most discoverable memorization studies use fixed evaluation splits (e.g., held-out subsets of PILE) without cross-validation. The survey does not systematically assess the statistical reliability of individual studies—it presents their findings at face value and focuses on conceptual synthesis rather than meta-analytic effect-size estimation. The absence of confidence intervals or uncertainty quantification in most cited studies is an implicit limitation of the evidence base that the survey itself inherits.

Main Quantitative Results

The survey's "results" are the cross-study patterns that emerge from its taxonomic organization. Since the paper aggregates 120 studies without running new experiments, the following is organized by research question axis (matching the survey's own RQ structure) and reports the headline quantitative findings that the survey itself highlights as empirically established.

How Much Memorization Occurs? (Measuring the Phenomenon)

Discoverable memorization converges to approximately 1% of training data at standard measurement settings. Carlini et al. (2023) report that "LLMs discoverably memorize roughly 1% of their training datasets" when prompted with approximately 50 tokens of context (Section II-B0b). This rate is corroborated by Anil et al. (2023) on PaLM and Kudugunta et al. (2023) on MADLAD-400. The 1% figure is specifically for verbatim discoverable memorization (defined in Section II-B0b: LM(p) = s where (p, s) is from the training set) with 50-token prefixes. The survey does not provide absolute counts—the 1% means that when sampling training contexts and checking whether the model reproduces the exact suffix, approximately 1 in 100 suffixes match.

Extractable memorization rates vary by orders of magnitude depending on verification methodology. Carlini et al. (2021) confirmed memorization of "about 0.00001% of GPT-2's training data" through extractable methods with manual Google search verification (Section II-B0a). Nasr et al. (2023) found "0.1% to 1%" using automated large-scale verification with a combined dataset proxy for the training data on Pythia, RedPajama, and GPT-Neo models. The 100× to 10,000× gap between these numbers reflects differences in verification methodology (manual vs. automated, strict original-data matching vs. proxy-dataset matching), model scale (GPT-2 1.5B vs. models up to 12B), and prompt engineering sophistication—not necessarily a 100× difference in underlying memorization. The survey presents both numbers without attempting to reconcile them, treating the methodological difference as part of the measurement fragmentation finding.

Approximate memorization doubles measured rates compared to exact match. Ippolito et al. (2023) define approximate memorization using BLEU > 0.75 similarity to training data and find that "the measurement of memorization can increase by a factor of two compared to only considering verbatim memorization" (Section II-A0c). The exact factor depends on the similarity threshold (δ = 0.75 for BLEU) and the specific generation task—the survey reports this as an illustrative magnitude rather than a universal constant. The practical significance: if verbatim matching finds ~1% memorization, approximately 2% of generations may be substantially similar to training data.

Entity-level PII extraction rates vary by information type. Kim et al. (2023) demonstrate that "emails and phone numbers show higher extraction rates compared to addresses" in OPT models (Section II-A0d). The survey does not report exact extraction-rate numbers for each PII type—only the relative ordering. This finding establishes that memorization risk is not uniform across information categories: structured identifiers with distinctive formats (emails, phone numbers) are more extractable than natural-language entities (addresses), likely because their format serves as a stronger retrieval cue.

Exposure metrics quantify memorization strength in bits. Carlini et al. (2019) define the exposure metric (Equation 1, Section III-B) and use canary extraction tests to measure how much a model's internal probability distribution favors training canaries over random alternatives. The metric is computed as exposure = log₂|R| − log₂ rank(s[r]). High exposure (e.g., 10+ bits) indicates strong memorization. The survey does not report specific exposure values aggregated across studies—exposure depends on canary design (randomness space size |R|, insertion frequency, training duration) and is reported per-experiment rather than as a population-level statistic.

What Factors Drive Memorization? (RQ3 Findings)

Memorization scales log-linearly with model size. Carlini et al. (2023) find that "the relationship between model size and memorization grows consistently on a log-linear scale" (Section IV-A1, Table II). Each doubling of model parameters approximately multiplies memorization by a constant factor, rather than adding a constant amount. The survey does not report the specific multiplication factor (which would depend on dataset, training regime, and measurement method) but presents log-linearity as the robust structural finding. Tirumala et al. (2022) additionally find that "larger models not only memorize more but do so faster in the training process."

Data duplication amplifies memorization super-linearly. Kandpal et al. (2022) report that "a sequence appearing 10 times in the training data is, on average, generated approximately 1000 times more frequently than a sequence that appears only once" (Section IV-B1). This is a super-linear relationship: 10× duplication → 1000× generation frequency. This quantitative relationship is central to the survey's argument that deduplication is an effective mitigation strategy (Section V-A). Lee et al. (2022) corroborate this with their finding that "models trained on deduplicated data would produce memorized text ten times less frequently" in no-prompt generation (Section IV-B1). The specific numbers: the C4 dataset contains "up to 13.6% near duplicates and up to 19.4% exact duplicates" (Section V-A), and training on deduplicated variants reduces memorization by a factor of approximately 10.

Deduplication reduces memorization by a factor of 10 in no-prompt generation, but prompted memorization is more persistent. Lee et al. (2022) train a 1.5B parameter GPT-2 model on three dataset variants (C4-Original, C4-NearDup, C4-ExactSubstr) and evaluate 50-token exact match memorization. In the no-prompt setting (generating text without any prefix), "the no-prompt experiment generations show 10× less memorization in de-duplicated trained models" (Section V-A). In the prompted setting when the prompt is sampled from duplicate examples, "the model trained on C4-Original generates the true exact continuation over 40% of the time," and the deduplicated-trained models still generate ground truth "more often when the prompt is sampled from the duplicate examples, suggesting that more harsh de-duplication algorithms are needed to prevent memorization" (Section V-A). No exact percentage for the deduplicated models in the prompted condition is reported—only that they outperform the original but still show elevated rates on duplicate-origin prompts.

Length of prompt context increases memorized output probability. Carlini et al. (2021), Kandpal et al. (2022), and McCoy et al. (2023) all find that "longer prompts increase the likelihood of triggering memorized sequences" (Section IV-C1). The survey does not report a specific quantitative relationship (e.g., per-token increase in extraction probability). Carlini et al. (2023) report that memorization scales with "prefix context length" as one of three factors alongside model size and data repetition, but the survey's citation of this is qualitative (Section II-B0b: "there is a log-linear correlation between data extraction and model size, repetition of data, and prefix context length").

Memorization and forgetting follow structured temporal dynamics during training. Jagielski et al. (2023) demonstrate that "examples seen during the earlier stages of training are less prone to memorization and rather they are forgotten over time" (Section IV-B2). Tirumala et al. (2022) observe that forgetting "follows an exponential degradation, reaching a constant value baseline" that "scales with the model size" (Section IV-B3). Leybzon and Kervadec (2024) observe a U-shaped pattern: "higher memorization rates happen early and late in training, with lower rates mid-training" (Section IV-B2). No specific decay constants or baseline values are reported—the survey presents the qualitative dynamics (exponential decay, U-shaped emergence, size-dependent baseline) as the established findings.

Language and content type affect memorization rates. Tirumala et al. (2022) find that "nouns and numbers are memorized significantly faster than other parts of speech" (Section IV-B1). Zhang et al. (2023) find that "counterfactually memorized data are generally unconventional texts such as all-caps, structured formats (i.e., tables or bullet lists), and multilingual texts" (Section III-D). Duan et al. (2024) find that "data with lower z-complexity leads to faster decreases in training loss" and "strings of varying complexity exhibit distinct memorization curves, with lower-complexity strings being memorized more easily even for smaller repeats, following a log-linear relationship in memorization probability" (Section IV-B1). The survey presents these as qualitative patterns (which content types are more memorized) rather than providing quantitative comparisons (how much more).

Fine-tuning effects vary by method and task. Mireshghallah et al. (2022) evaluate full model, model head, and adapter fine-tuning, finding that "head fine-tuning is most susceptible to attacks, whereas adapter fine-tuning is less prone" (Section IV-B4). Zeng et al. (2024) find that "fine-tuned memorization varies significantly depending on the task" and "multi-task fine-tuning can mitigate memorization risks more effectively than single-task fine-tuning" (Section IV-B4). No specific memorization rates per fine-tuning method are reported—the findings are presented as relative comparisons within each study.

Tokenization impacts memorization. Kharitonov et al. (2022) demonstrate that "increasing the sub-word vocabulary significantly affects the model's ability and inclination to memorize training data" and "models with larger vocabularies are more likely to reproduce training data when given specific prompts" (Section IV-A2). The proposed mechanism: larger BPE vocabularies reduce sequence lengths, decreasing the number of token decisions needed to reproduce a training string. No specific vocabulary-size-to-memorization-rate mapping is reported.

How Can Memorization Be Mitigated? (RQ4 Findings)

Differential privacy reduces memorization but at utility cost. Carlini et al. (2019, 2021) demonstrate that adjusting the DP-SGD privacy parameter ε "can reduce the exposure of memorized data to a level that makes it indistinguishable from any other data. However, this comes at the cost of reduced model utility and a slower training process" (Section V-B1). Li et al. (2022) show that "with carefully chosen hyperparameters and downstream task objectives, fine-tuning pretrained language models with DP-SGD can yield strong performance on a variety of NLP tasks at privacy levels" and "some of their fine-tuned models even outperform non-private baselines and models trained under heuristic privacy approaches" (Section V-B1). Lukas et al. (2023) provide the cautionary counter-finding: even at commonly-used ε values, "the model may not withstand stronger privacy attacks, potentially compromising its effectiveness" (Section V-B1). No specific ε-to-utility tradeoff curves are reported—the survey presents the existence of a tradeoff and the existence of configurations where utility loss is acceptable, without quantifying the Pareto frontier.

Deduplication is one of the most effective and efficient mitigation methods. The survey explicitly states: "Data deduplication has been widely shown to be one of the most effective and efficient methods to mitigate memorization" (Answer to RQ4, Section V). This assessment is based on Lee et al. (2022)'s finding of 10× memorization reduction in no-prompt generation and "models trained on deduplicated data would produce memorized text ten times less frequently" (Section IV-B1). The survey does not compare deduplication quantitatively against other methods like DP-SGD—it asserts effectiveness in the concluding summary based on the consistent literature consensus that duplication is a primary driver and that deduplication directly addresses this mechanism without modifying the training algorithm or model architecture.

Training interventions provide targeted memorization control without full DP overhead. The goldfish loss (Hans et al., 2024) is described as "a subtle modification to the next-token training objective where randomly sampled subsets of tokens are excluded from the loss computation. This prevents models from memorizing complete token sequences, significantly reducing extractable memorization while maintaining downstream performance" (Section V-B3). The alternating teaching method (Liu et al., 2023) uses "a teacher-student framework where multiple teachers trained on disjoint datasets supervise a student model in an alternating fashion to reduce unintended memorization" and demonstrates "superior privacy-preserving results on the LibriSpeech dataset while maintaining minimal utility loss when sufficient training data is available" (Section V-B3). Memorization sinks (Ghosal et al., 2025) isolate "memorized content into dedicated neurons, preventing it from interfering with general language knowledge" (Section V-B3). No quantitative memorization reduction rates are reported for these methods—the findings are presented as proof-of-concept demonstrations.

Unlearning and model editing enable post-hoc removal of memorized content. For unlearning, the survey cites SISA framework (Bourtoule et al., 2021), lightweight unlearning layers (Chen and Yang, 2023), In-Context Unlearning (Pawelczyk et al., 2024), and DeMem (Kassem et al., 2023) as demonstrating that specific information can be removed without full retraining. For model editing, Private Memorization Editing (Ruzzetti et al., 2025) "detects memorized PII in a trained LLM and edits the model's internal memory to remove that PII, thereby reducing the risk of privacy leakage while preserving the model's general language capabilities," and weight-pruning-based methods (Ni et al., 2025) provide "a unified framework to tune a model's memorization: by gradient-guided pruning, one can suppress or amplify memorization depending on privacy vs. utility needs" (Section V-C2). No memorization-reduction percentages are reported.

Decoding-based methods provide modest memorization reduction at inference time. Ippolito et al. (2023) propose MemFree Decoding, which "significantly lower[s] generation similarity with the training data" but is vulnerable to bypass through "a simple style transfer in prompts" (Section V-C3). Borec et al. (2024) investigate nucleus sampling and find that "while increasing the nucleus size slightly reduces memorization, it only provides modest protection" (Section V-C3). The survey does not report specific memorization-reduction percentages for these decoding-based methods, only the qualitative finding that they provide limited protection that can be circumvented.

Hard memorization (verbatim) vs. soft memorization (similarity-based) respond differently to mitigation. Borec et al. (2024) distinguish between "hard" memorization (verbatim reproduction) and "soft" memorization (measured by ROUGE similarity), finding that nucleus sampling may suppress exact reproductions but leave high-similarity paraphrasing intact. This is significant because approximate memorization (Section II-A0c) captures content that decoding-based methods may miss—a user examining output for privacy violations might see text that is clearly derived from training data even if no 50-gram matches exactly.

Ablation Studies and Robustness Checks

The survey does not perform original ablation studies. However, it documents methodological choices in cited studies that function as ablations—comparisons where one variable is changed while others are held fixed to isolate its effect. The following catalogs these as they appear across the surveyed literature, organized by the factor being isolated.

Decoding method (greedy vs. temperature vs. top-k vs. nucleus vs. typical vs. beam search): Yu et al. (2023) systematically compare decoding schemes and use auto-tuning to find the optimal method for maximizing training data reproduction. Carlini et al. (2021) compare greedy decoding against decaying temperature and top-n sampling, finding that top-n is more successful at extraction than greedy alone. The survey reports these as evidence that "greedy decoding is dominantly employed" (Table II) because it "maximizes the likelihood of generating memorized sequences" (Table III footnote), but that other methods can achieve higher extraction under specific conditions. The practical implication: measured memorization rates are sensitive to the decoding strategy used during evaluation, and a model evaluated only with greedy decoding may appear less memorizing than it actually is under an optimized adversarial decoding scheme.

Prefix length (32 vs. 50 vs. 100+ tokens): Table III documents that prefix lengths range from 32 to 450 tokens across studies, with 50 tokens being the dominant convention. Carlini et al. (2023) show that longer prefixes increase extractable memorization rates, establishing that prefix length is not a neutral methodological choice—it shifts measured memorization upward. The survey does not report a per-token increase rate but notes that this relationship is one of the three factors (with model size and data repetition) contributing to memorization variance.

Match length (32 vs. 50 vs. 64 vs. 1000 tokens): Table III documents match-length variation from 32 to 1000 tokens. Longer match-length requirements reduce measured memorization rates (longer exact matches are rarer) but provide stronger evidence of memorization when found (longer matches are less likely to occur by chance). The survey does not quantify the match-length-to-memorization-rate relationship.

Similarity metric and threshold (for approximate memorization): Ippolito et al. (2023) use BLEU with a 0.75 threshold chosen "based on qualitative analyses of the samples" (Section II-A0c). Duan et al. (2024) use token-wise Levenshtein distance with an unspecified threshold. Peng et al. (2023) use Minhash with Jaccard similarity for efficient approximate matching. The choice of metric and threshold determines what fraction of near-matches are classified as memorized. Ippolito et al. note that their threshold "can lead to both false positives and false negatives when compared to human judgment" (Section II-A0c), indicating that approximate memorization measurement is threshold-sensitive in ways that are not yet calibrated against human perception of copying.

Deduplication granularity (exact vs. near-duplicate vs. no deduplication): Lee et al. (2022) compare three training data conditions: C4-Original, C4-NearDup, and C4-ExactSubstr. In no-prompt generation, both deduplication variants reduce memorization by 10×. In prompted generation with duplicate-origin prompts, the C4-Original model reproduces the exact continuation over 40% of the time, and the deduplicated models still show elevated rates on these prompts. The survey interprets this as evidence that "more harsh de-duplication algorithms are needed to prevent memorization" (Section V-A), suggesting that existing deduplication methods (exact substring matching, MinHash-based near-duplicate detection) leave residual memorization when prompts come from the surviving duplicate-adjacent data.

Fine-tuning method (full model vs. head vs. adapter): Mireshghallah et al. (2022) compare these three fine-tuning approaches and find that head fine-tuning is most susceptible to privacy attacks while adapter fine-tuning is least prone (Section IV-B4). The survey presents this as an ablation that isolates the effect of parameter modification scope on memorization retention: head fine-tuning preserves the base model's memorized information intact, while adapter fine-tuning introduces new parameters that may overwrite or interfere with stored memorized content.

PRM aggregation strategy (min vs. prod vs. last): The survey does not contain this ablation—it appears in the reference example but is not part of the memorization survey paper. This is a structural difference between the two papers being analyzed.

Verifier type (PRM vs. ORM) for revision model outputs: The survey does not contain this ablation—the memorization survey does not study verifiers, process reward models, or revision models. These concepts belong to the LLM test-time compute scaling literature and are absent from the memorization taxonomy.

Context inclusion for revision verifier: Not applicable to this survey paper.

Oracle vs. predicted difficulty bins: Not applicable. The memorization survey does not use difficulty estimation, oracle bins, or predicted bins. These are concepts from the test-time compute scaling paper in the reference example.

Majority voting as selection mechanism: Not applicable. The memorization survey does not study answer selection mechanisms for reasoning tasks.

ReST^EM revision model optimization: Not applicable. The survey does not study revision models, ReST^EM, or reinforcement-learning-based optimization of language model outputs.

Counterfactual memorization sensitivity to data characteristics: Zhang et al. (2023) find that counterfactually memorized data are "generally unconventional texts such as all-caps, structured formats (i.e., tables or bullet lists), and multilingual texts" (Section III-D). This is effectively an ablation over text type within a fixed measurement methodology (counterfactual comparison across 400 trained models), isolating which data characteristics cause memorization to be more dependent on specific training exposure rather than generalizable patterns.

Language as a memorization factor: Satvaty et al. (2025) and Luo et al. (2025) provide evidence that "lower-resource languages show different memorization patterns when compared to the English language" and that "memorization patterns are shaped by cross-lingual similarities rather than data volume alone" (Section VI-C). This is an ablation over language within the same model (multilingual models trained on multiple languages), isolating language identity as a factor independent of model architecture.

Adversarial prompt optimization method: Kassem et al. (2024) compare direct prompting against instruction-based prompt optimization, finding that "instruction-tuned models can reveal as much or more training data as base models" when prompted optimally (Section IV-C1). This ablates prompt engineering strategy, showing that alignment training (instruction tuning) does not eliminate memorization—it only changes which prompts are needed to access it.

Critical Assessment

Do the Aggregated Empirical Results Support the Survey's Central Organizational Claim?

The survey's primary intellectual contribution is that memorization is not a unitary phenomenon but a three-dimensional spectrum (granularity × retrievability × desirability), and that organizing the literature along these axes reveals systematic mismatches between measurement methods, causal explanations, and mitigation strategies. This is a taxonomic claim rather than a causal or predictive one—it asserts that the proposed categorization is useful for organizing the literature and identifying research gaps, not that memorization is caused by specific factors with specific magnitudes.

The taxonomic dimensions are well-motivated by the literature the survey catalogs. The granularity dimension (Section II-A) is supported by formal definitions from multiple independent research groups: Kandpal et al. (2022) for perfect memorization, Carlini et al. (2021) for verbatim, Ippolito et al. (2023) for approximate, Zhou et al. (2023) for entity-level, and Petroni et al. (2019) and related work for content-level. These are not the survey's inventions—they are definitions extracted from the literature and organized along a spectrum of increasing abstraction. The retrievability dimension (Section II-B) similarly draws on established distinctions: extractable memorization from Carlini et al. (2023) and discoverable memorization from Nasr et al. (2023), with the non-containment relationship empirically demonstrated by Nasr et al. (2023)'s observation that "some sequences are both discoverably and extractably memorized; some sequences are discoverably memorized but not extractably memorized, and vice versa" (Section II-B0c). This non-containment is the strongest piece of evidence that the two retrievability categories are genuinely distinct rather than one being a subset of the other—the survey's taxonomy would be undercut if extractable were simply a subset of discoverable, but the empirical evidence shows they are partially overlapping but independent regions of the space.

However, the desirability dimension is asserted rather than empirically supported. Section II-C distinguishes desirable and undesirable memorization by listing benefits (transparency, knowledge retention, language generation, alignment) and harms (privacy risks, security vulnerabilities, copyright violation, bias, benchmark contamination, reasoning degradation). The survey's theoretical move—that desirability is stakeholder-relative and that "even when memorization provides functional advantages, it always carries an undesirable risk" (Section II-C Remark)—is a philosophical position rather than an empirical finding. None of the 120 surveyed papers quantifies the tradeoff between, say, the benefit of factual recall accuracy and the cost of privacy leakage for the same model. The survey does not cite studies that measure how much factual recall degrades when privacy protections are applied, or how much privacy risk increases when factual recall is optimized. The desirability dimension is therefore a useful conceptual framing for organizing normative questions about memorization, but it is not empirically validated in the way that the granularity and retrievability dimensions are.

The fragmentation finding (Table III) is genuine and well-documented, but the survey may overstate its consequences. Table III clearly shows that different studies use different models, datasets, decoding methods, prefix lengths, and match lengths, measuring different combinations of granularity and retrievability. The survey's claim that this makes cross-study comparison "nearly impossible" is supported by the documented heterogeneity. However, the survey does not demonstrate that these methodological differences actually change the qualitative conclusions of the studies that use them. For example, if two studies using different prefix lengths both find that deduplication reduces memorization by a factor of ~10×, then the methodological difference is less consequential than the survey implies. The survey identifies measurement fragmentation but does not assess whether it changes the rank ordering of findings (e.g., whether deduplication is always better than DP-SGD across measurement settings) or merely shifts absolute numbers. This is a missing meta-analytic dimension—the survey organizes studies but does not test whether their conclusions are robust to measurement protocol variations.

Are the Factor-to-Mechanism Claims Well-Supported by the Cited Evidence?

The log-linear model-size-to-memorization relationship is well-established but comes with important caveats. Carlini et al. (2023) is the primary citation for this claim, and the survey presents it as a robust finding (Section IV-A1, Table II). However, the cited finding is based on GPT-Neo model variants (125M to 6B parameters) and Pythia models (70M to 12B parameters) trained on the PILE dataset. Whether log-linearity extends to models significantly outside this range (e.g., GPT-3 at 175B, GPT-4 at estimated 1.7T parameters) is not established—extrapolation beyond the studied scale range is speculative. Additionally, the survey cites Carlini et al.'s finding that "memorization does not necessarily correlate with improved performance" by comparing "models with similar capacities but differing performance levels because of their architectures" (Section IV-A1). This architectural sensitivity means that the log-linear relationship is not a universal law—it holds for the model families studied under their training regimes but may differ for models with different architectures, training objectives, or data mixtures. The survey does not discuss this scope limitation.

The super-linear duplication-memorization relationship (10× duplication → 1000× generation) is striking but from a single study. Kandpal et al. (2022) is the sole citation for this specific quantitative claim (Section IV-B1). The survey does not report whether this relationship has been replicated on different model families, datasets, or duplication patterns. A 10× → 1000× relationship implies an exponent of approximately 3 (since 10^3 = 1000), which would mean memorization scales as duplication raised to the power ~3. This is a large exponent, making the finding practically important but also making replication particularly valuable before it is treated as an established fact. The survey presents this as a "central finding" without noting that replication evidence is absent from the surveyed literature.

The 1% discoverable memorization rate is converged upon by multiple independent studies—an unusually strong consensus for this literature. Carlini et al. (2023) on GPT-Neo, Anil et al. (2023) on PaLM, and Kudugunta et al. (2023) on MADLAD-400 all report approximately 1% discoverable memorization at ~50-token prefixes. This convergence across different model families and datasets is the strongest empirical anchor in the survey. However, all three measurements use the same fundamental methodology (prompt with training prefixes of ~50 tokens, check for exact suffix match) and similar model scales (billions of parameters). The 1% figure may be an artifact of this specific measurement protocol rather than a fundamental constant. A model with different training dynamics (e.g., trained for fewer epochs, or with a different data mixture) might exhibit different rates. The survey acknowledges that discoverable memorization provides "an upper bound" (Section II-B0b), so the 1% should be interpreted as a ceiling under current measurement conventions, not an estimate of how much training data is actually at risk of extraction in practice.

The finding that "earlier stages of training are less prone to memorization" comes from Jagielski et al. (2023), but Leybzon and Kervadec (2024) observe a U-shaped pattern—higher early, lower mid, higher late. These two findings are in tension: Jagielski et al. find early training is protective, while Leybzon and Kervadec find early training has higher memorization. The survey presents both without resolution (Section IV-B2), noting only that Leybzon and Kervadec's U-shape suggests "placing sensitive data in the middle stages of training could reduce its vulnerability" (Section IV-B2). The inconsistency may arise from different definitions of "early training" (first few epochs vs. first few steps within an epoch), different model scales, or different memorization measurement methods. The survey does not attempt to reconcile these findings, which is appropriate for a survey but leaves a genuine empirical uncertainty in the literature that readers should be aware of.

Does the Mitigation Section (Section V) Provide Actionable Guidance?

The survey's Answer to RQ4 singles out deduplication as "one of the most effective and efficient methods." This recommendation is supported by the quantitative evidence (Lee et al., 2022: 10× reduction) and the mechanistic link between duplication and memorization (Kandpal et al., 2022: super-linear scaling). However, the survey's own evidence base also shows that deduplication is incomplete—prompted memorization on duplicate-adjacent data persists (Lee et al., 2022: models trained on deduplicated data still generate ground truth "more often when the prompt is sampled from the duplicate examples"). The survey's recommendation should therefore be understood as: deduplication substantially reduces memorization but does not eliminate it, and residual memorization remains accessible through adversarial prompts.

The caution about differential privacy (DP) is well-calibrated. The survey notes both the formal guarantee (Carlini et al., 2019: exposure reduced to indistinguishable-from-random levels) and the practical limitations (Li et al., 2022: utility cost that can be mitigated with careful hyperparameter tuning; Lukas et al., 2023: DP-trained models may not withstand stronger attacks). The survey does not recommend a specific ε value or provide a utility-memorization tradeoff curve, which is the information a practitioner would actually need. This is a limitation of the survey's evidence base—the cited studies report on different models, tasks, and ε values, and the survey does not synthesize them into actionable guidance beyond "select hyperparameters wisely."

The survey does not compare mitigation methods head-to-head. This is the most significant practical limitation. A model developer deciding between deduplication, DP-SGD, and goldfish loss cannot use this survey to answer: which method gives the best memorization reduction for a given utility budget? The survey catalogs each method's existence, its mechanism, and its documented effect sizes within the studies that introduced it, but these studies use different models, datasets, evaluation metrics, and threat models. The Remark in Section V-C acknowledges this: "no single method fully eliminates memorization, and careful combination is needed to balance privacy, reasoning, and performance." This is true but underspecified—the survey does not provide evidence about which combinations work, how much memorization remains after combining methods, or what the resulting utility cost is.

What Experiments Would Strengthen the Literature the Survey Catalogs?

The survey implicitly identifies several missing experiments through its gap analysis (Section VI), but the following are additional experiments that would directly address weaknesses in the current evidence base:

A multi-metric memorization benchmark: No study in the survey simultaneously measures verbatim exact match, verbatim approximate match, entity-level extraction, and content-level factual recall on the same set of models. Such a study would quantify how much these different granularity levels correlate (does a model that memorizes verbatim also memorize entity associations more?) and would validate or refute the survey's claim that different measurement methods capture distinct phenomena.
A head-to-head mitigation comparison under standardized conditions: Train models (same architecture, same dataset, same training budget) with deduplication-only, DP-SGD-only, goldfish-loss-only, and combinations thereof. Measure memorization using all granularity and retrievability combinations. Measure downstream task performance on both knowledge-intensive tasks (where memorization helps) and reasoning tasks (where memorization may hurt). This would produce the utility-memorization tradeoff curves that practitioners need and would validate whether combining methods produces super-additive protection or diminishing returns.
Scale extrapolation studies: Current empirical evidence on model-scale effects covers models up to ~12B parameters (Pythia, GPT-Neo). Do the log-linear trends continue to 70B, 175B, and beyond? Does deduplication's 10× reduction factor hold at larger scales? Are there scale thresholds where memorization qualitatively changes (e.g., from verbatim to approximate as models learn to paraphrase)? These questions are unanswered by the surveyed literature.
Longitudinal memorization studies: The survey notes that "a survey paper written in 2025 risks becoming outdated by 2026" (Limitations). A study that tracks memorization rates across model releases over time (GPT-2 → GPT-3 → GPT-4, or the LLaMA series) would reveal whether newer models memorize more (due to larger scale) or less (due to improved training practices, deduplication, safety filtering) and at what rate the phenomenon is evolving with the technology.

Summary: What the Evidence Actually Establishes vs. What the Survey Claims

The survey makes three structural claims that define its contribution:

Memorization is a multi-dimensional phenomenon. The evidence for this is strong for the granularity and retrievability dimensions, which are formalized with definitions extracted from the literature and supported by empirical demonstrations that different measurement methods capture non-overlapping subsets of memorized content (Nasr et al., 2023's extractable-vs-discoverable non-containment). The desirability dimension is conceptually motivated but empirically unvalidated—no study quantifies the tradeoff between benefits and harms of the same memorization behavior for different stakeholders.
The literature is fragmented, and this fragmentation masks important distinctions. Table III provides strong documentation of methodological heterogeneity. The survey demonstrates that different studies measure different things but does not quantify whether this heterogeneity changes qualitative conclusions or only absolute numbers. The fragmentation claim is well-supported as a characterization of the literature; the claim that this fragmentation makes findings incomparable is plausible but not demonstrated through counterfactual analysis.
Mitigation strategies target different parts of the memorization spectrum and have different failure modes. The survey catalogs mitigation methods and their documented effects, but does not provide head-to-head comparisons or utility-memorization tradeoff curves. The claim that "careful combination is needed" is supported by the observation that no single method fully eliminates memorization, but the survey does not provide evidence about which combinations work or how to select them. The actionable guidance for practitioners is therefore limited to: (a) deduplicate training data (strong evidence of effectiveness), (b) consider DP-SGD but be aware of utility costs and residual vulnerability to strong attacks, (c) use training interventions or post-training unlearning for targeted removal, and (d) do not rely on decoding-based methods alone. The survey's primary practical value is in its diagnostic framework—it enables practitioners to identify which type of memorization they care about and which measurement methods are appropriate for assessing it—rather than in its prescriptive recommendations, which remain at the level of general strategies rather than specific recipes.

6. Limitations and Trade-offs

Fragmentation Analysis Demonstrates Heterogeneity, Not Whether It Matters

The assumption or constraint. The survey's central diagnostic claim is that the memorization literature is so methodologically fragmented that cross-study comparison is "nearly impossible" (Section 1, implicit throughout Section III and Table III). Table III documents that different studies use different model families (Pythia, GPT-Neo, GPT-J dominating), different datasets (PILE dominating), different prefix lengths (32–450 tokens), different match lengths (32–1000 tokens), different decoding methods (greedy, temperature, top-k, nucleus, beam search), and different granularity/retrievability combinations. The presence of heterogeneity is well-established. However, the survey does not test whether this heterogeneity actually changes the conclusions of the studies that exhibit it. The implicit assumption is that measurement fragmentation invalidates cross-study comparison.

The consequence. A practitioner reading the survey might conclude that no reliable knowledge exists about memorization because every study measures something different. But this overstates the case. If five studies using five different prefix lengths all find that deduplication reduces memorization by a factor of approximately 10×, or that memorization scales log-linearly with model size, then the methodological heterogeneity is less consequential than the survey implies—the qualitative findings are robust even if the exact numbers differ. The survey does not provide this kind of meta-analytic robustness check. It catalogs the differences but does not assess whether findings are sensitive to methodological choices. This leaves a gap between "the literature uses heterogeneous methods" (true and well-demonstrated) and "the literature's findings are therefore unreliable or incomparable" (a stronger claim that is asserted rather than tested).

What evidence exists in the paper. Table III is the primary evidence for the existence of fragmentation. Section III catalogs measurement methods with their different operationalizations of the taxonomic dimensions. The survey notes that Nasr et al. (2023) found some sequences are discoverably memorized but not extractably memorized, and vice versa (Section II-B0c)—this is direct evidence that two measurement methods capture non-overlapping sets, which supports the claim that fragmentation has empirical consequences. However, this is a single comparison between two specific methods on one model. No equivalent comparison exists for, say, the effect of changing prefix length from 32 to 50 tokens, or changing decoding from greedy to nucleus, on the rank ordering of mitigation strategies. The fragmentation diagnosis is well-supported; the claim that fragmentation invalidates cross-study conclusions is less so.

Mitigation status. The survey does not attempt to resolve this limitation—it acknowledges it implicitly by presenting Table III as a descriptive inventory rather than a meta-analytic synthesis. The limitation is structural to the survey methodology: aggregating 120 heterogeneous studies without re-running experiments means the survey cannot test whether heterogeneity matters for conclusions. Section VI's call for future work on specific contexts (conversational agents, RAG, multilingual models, diffusion models) is partially a response to this—by identifying understudied contexts, the survey implicitly argues that even the fragmented literature is concentrated on a narrow slice of the problem space. But the survey does not propose or call for a large-scale multi-metric benchmark that would directly test the robustness of existing findings to measurement protocol variation.

The Desirability Dimension Is Philosophically Framed, Not Empirically Grounded

The assumption or constraint. Section II-C distinguishes desirable from undesirable memorization by listing benefits (transparency/auditing, knowledge retention, language generation, alignment value injection) and harms (privacy leakage, security vulnerabilities, copyright violation, bias perpetuation, benchmark contamination, reasoning degradation). The survey's most theoretically ambitious move is the Remark closing Section II-C:

"Even when memorization provides functional advantages, it always carries an undesirable risk."

This is presented as a structural claim about memorization—that the same model capacity enabling factual recall is the capacity an adversary exploits for extraction, making purely desirable memorization impossible. The survey treats this as a conclusion from the aggregated literature.

The consequence. The claim that memorization is inherently dual-use is untested by any study the survey cites. No paper in the surveyed corpus (120 papers) quantifies the tradeoff between, for example, the benefit of factual recall accuracy gained through memorization and the cost of increased privacy risk from that same memorization, measured on the same model under the same conditions. The survey does not cite evidence that when memorization is reduced (e.g., through deduplication or DP-SGD), factual recall degrades in a predictable way, or that when factual recall is optimized, privacy risk increases measurably. Without such evidence, the "always carries an undesirable risk" thesis is a philosophical framing rather than an empirical finding. It is plausible and intellectually productive—it motivates the survey's emphasis on balancing rather than eliminating memorization—but a practitioner seeking to decide whether a specific memorization reduction is "worth it" in utility terms will find no quantitative guidance.

What evidence exists in the paper. Section II-C catalogs benefits (citing Ranaldi et al., 2023; Chen et al., 2023; Lu et al., 2024; Ouyang et al., 2022; Bender et al., 2021) and harms (citing Henderson et al., 2023; Carlini et al., 2019; Ramaswamy et al., 2020; Huang et al., 2024; Mueller et al., 2024; Freeman et al., 2024; Cooper and Grimmelmann, 2025; Elangovan et al., 2021; Bordt et al., 2024; Du et al., 2025; Prashanth et al., 2024; Hong et al., 2025). But these citations establish that benefits and harms exist—not that they trade off against each other in the same model, or that the tradeoff is unavoidable, or that the same memorization instances simultaneously produce benefits and harms. The survey's Section VI-A ("Balancing memorization benefits and privacy risks") calls for "strategies that balance memorization benefits with privacy protection, developing techniques that safeguard sensitive data and intellectual property without significantly degrading accuracy or utility." This call for future work is an implicit acknowledgment that the balance has not yet been empirically characterized.

Mitigation status. The survey does not resolve this limitation—it explicitly defers it to future work (Section VI-A). The survey's contribution is to frame the tension clearly rather than to resolve it. The limitation is that the framing is presented as a conclusion from the literature when it is better understood as a hypothesis that the literature has not yet tested. A fairer characterization would be: the literature documents that memorization produces both benefits and harms in different contexts, and the survey hypothesizes that these are structurally linked, but no study has demonstrated this linkage empirically.

Practical Difficulty Estimation and Mitigation Selection Costs Are Unaccounted For

The assumption or constraint. Throughout the survey, measurement methods and mitigation strategies are described in terms of their effectiveness (memorization reduction rates, exposure reduction, attack success rate changes) without accounting for their operational cost. The survey does not provide a cost model—in terms of compute, human effort, or data requirements—for any of the approaches it catalogs. This is an implicit assumption that effectiveness comparisons are meaningful without cost normalization.

The consequence. A practitioner choosing between mitigation strategies cannot use this survey to answer cost-effectiveness questions. Consider the practical options the survey endorses: (a) data deduplication: requires running exact-match and MinHash algorithms over the entire training corpus before training (Lee et al., 2022: C4, RealNews, LM1B, Wiki40B datasets), which is a one-time preprocessing cost but computationally non-trivial for trillion-token corpora; (b) DP-SGD training: requires per-example gradient clipping and noise addition, which increases training time—the survey notes it "comes at the cost of reduced model utility and a slower training process" (Section V-B1) but does not quantify the slowdown; (c) training interventions like goldfish loss (Hans et al., 2024): requires modifying the training objective; (d) post-training unlearning: requires identifying what to unlearn and running unlearning procedures, with compute costs that scale with the amount of data to be removed; (e) decoding-based methods: have near-zero deployment cost but provide only "modest protection" that can be "circumvented" (Section V-C3). The survey provides no framework for comparing, say, the cost of 10× memorization reduction through deduplication vs. the cost of equivalent reduction through DP-SGD vs. the cost of post-hoc unlearning. A practitioner cannot determine whether deduplication's "one of the most effective and efficient methods" designation (Answer to RQ4) holds when total lifecycle cost is considered.

What evidence exists in the paper. The survey documents effectiveness in relative terms: deduplication yields 10× memorization reduction (Lee et al., 2022, Section V-A); DP-SGD can reduce exposure to indistinguishable-from-random levels (Carlini et al., 2019, Section V-B1); goldfish loss "significantly reduc[es] extractable memorization while maintaining downstream performance" (Section V-B3); decoding methods "only provide modest protection" (Borec et al., 2024, Section V-C3). But no study in the survey simultaneously reports both memorization reduction and the operational cost (FLOPs, wall-clock time, data requirements, engineering effort) of achieving it. The survey does not flag this as a missing dimension of comparison, even though cost-effectiveness is the information a practitioner needs to make a deployment decision.

Mitigation status. The survey does not address this limitation. Section VI does not call for cost-effectiveness analysis as a future direction. The "efficiency" claim in the Answer to RQ4 ("Data deduplication has been widely shown to be one of the most effective and efficient methods to mitigate memorization") uses "efficient" in a loose sense (it works well) rather than in a cost-normalized sense (it achieves the best memorization reduction per unit of compute or engineering effort). This is a gap between the survey's prescriptive recommendations and the evidence needed to operationalize them.

Evidence Base Is Concentrated on Two Model Families and One English Dataset

The assumption or constraint. Table III documents that Pythia and GPT-Neo are the dominant model families (used in 60%+ of studies measuring memorization dynamics), and the PILE is the dominant dataset (used in essentially all studies on these model families, since Pythia and GPT-Neo were trained on it). Secondary model families (GPT-J, OPT, LLaMA, PaLM, GPT-3, T5) and datasets (C4, RealNews, Enron, RedPajama, RefinedWeb, Dolma) appear in fewer studies. The survey's findings about memorization rates, factor effects, and mitigation effectiveness are therefore disproportionately calibrated to English text with PILE's specific duplication characteristics (Lee et al., 2022: "up to 13.6% near duplicates and up to 19.4% exact duplicates"), content distribution (web text, academic papers, code, books), and the architectural properties of autoregressive Transformer models trained with standard next-token prediction objectives.

The consequence. The survey's conclusions—that memorization scales log-linearly with model size, that 1% of training data is discoverably memorized at 50-token prefixes, that deduplication reduces memorization by 10×—are empirically supported for the specific model-dataset combinations studied but may not generalize to other architectures (encoder-decoder models like T5, mixture-of-experts models, retrieval-augmented models), other data distributions (non-English text, code, structured data, multilingual corpora), or other training paradigms (instruction tuning, RLHF, continual pretraining). The survey itself provides evidence for this concern: Section VI-C notes that "lower-resource languages show different memorization patterns when compared to the English language" (Satvaty et al., 2025) and that "memorization patterns are shaped by cross-lingual similarities rather than data volume alone" (Luo et al., 2025). Section VI-D notes that diffusion language models may have entirely different memorization behavior from autoregressive models due to "the discrete nature of the text domain." These observations suggest that the survey's headline empirical findings are domain-specific rather than universal, but the survey presents them as general findings about "LLMs" without sufficient qualification about their scope of empirical support.

Additionally, the concentration on open-source models means that findings may not apply to proprietary production models (GPT-4, Claude, Gemini), which may have undergone additional memorization-reduction procedures (safety filtering, deduplication, output monitoring) that are not documented publicly. The survey notes that Nasr et al. (2023)'s divergence attacks on conversational-aligned LLMs "are not powerful enough to stimulate training data reproduction" (Section VI-A), suggesting that production models may have memorization characteristics different from the open-source research models that dominate the literature.

What evidence exists in the paper. Table III provides the model-family and dataset concentration evidence. Section VI-C (multilingual models) and Section VI-D (diffusion language models) provide the survey's own acknowledgment that memorization may differ across contexts. Section VI-A (conversational agents) notes the gap between research models and production models. The survey's Limitations section acknowledges that "a survey paper written in 2025 risks becoming outdated by 2026" and that "the fast pace at which the field is evolving" is a concern, but frames this as a timeliness issue rather than a scope-of-evidence issue. The survey does not explicitly state that its empirical conclusions are calibrated to autoregressive English-language Transformer models trained on web-scale text corpora with standard objectives.

Mitigation status. The survey partially addresses this through the future directions in Section VI, which call for research on multilingual models (Section VI-C), diffusion language models (Section VI-D), conversational agents (Section VI-A), and RAG systems (Section VI-B). However, these are presented as extensions to the existing knowledge base, not as caveats about its current scope. A practitioner working with, say, a code generation model or a multilingual model in a low-resource language should understand that the survey's quantitative findings (1% memorization rate, 10× deduplication reduction) may not transfer to their setting. The survey would be stronger if it explicitly bounded the scope of its conclusions to the evidence base from which they are drawn.

The Survey Catalogs What Exists but Does Not Test Whether It Works Together

The assumption or constraint. Section V organizes mitigation methods by intervention point: data-level (deduplication), training-time (DP-SGD, promoting reasoning, training interventions like goldfish loss and alternating teaching), and post-training (unlearning, model editing, decoding-based methods). Each method is described with its mechanism and its documented effectiveness from the study that introduced or evaluated it. The survey's Remark in Section V-C acknowledges that "no single method fully eliminates memorization, and careful combination is needed to balance privacy, reasoning, and performance." This implies that combinations of methods are necessary in practice, and the survey positions itself as providing the taxonomic framework for designing such combinations.

The consequence. The survey provides no evidence about which combinations work, whether their effects are additive, super-additive, or sub-additive, or what the total utility cost of combined mitigation is. A practitioner who follows the survey's guidance might apply deduplication + DP-SGD + post-training unlearning, expecting each component to contribute its individually documented memorization reduction. But the survey provides no evidence that these methods compose without interference. For example, deduplication changes the data distribution, which may change the effective privacy budget needed from DP-SGD. DP-SGD adds noise to gradients, which may affect the effectiveness of subsequent unlearning procedures that rely on gradient-based editing. Decoding-based methods at inference time may interact with parameter-level mitigations in unexpected ways. The survey's call for "careful combination" is correct but underspecified—it does not describe what "careful" means or what failure modes to watch for.

This is not a criticism of the survey for failing to run experiments (it is a survey, not an empirical study). But it is a limitation of the state of the literature that the survey documents: of the 120 papers surveyed, none systematically studies combinations of mitigation methods under standardized conditions. The survey could make this gap more explicit rather than implying that the taxonomic framework is sufficient for designing combinations. Organizing methods by intervention point (data, training, post-training, inference) is useful for understanding their mechanisms, but it does not substitute for empirical evidence about their interactions.

What evidence exists in the paper. Section V presents each mitigation method in isolation, citing the original studies that introduced or evaluated it. The Remark in Section V-C is the only acknowledgment of the combination problem: "no single method fully eliminates memorization, and careful combination is needed to balance privacy, reasoning, and performance." No study in the survey tests a combination of methods (e.g., deduplication + DP-SGD, or goldfish loss + unlearning). The survey does not identify the absence of combination studies as a research gap in Section VI—the future directions focus on new contexts (conversational agents, RAG, multilingual models, diffusion models) rather than on methodological gaps within the existing mitigation literature.

Mitigation status. The survey does not address this limitation. Section VI does not call for combination studies as a future direction. A practitioner seeking to combine methods will need to look beyond this survey for guidance on interactions, side effects, and composition strategies.

The Survey Inherits the Statistical and Methodological Weaknesses of Its Source Literature

The assumption or constraint. The survey aggregates findings from 120 papers without systematically assessing the statistical rigor, sample sizes, or methodological validity of each study. The survey's conclusions are only as reliable as the studies it synthesizes. The survey makes no claim to have conducted a formal risk-of-bias assessment or to have weighted studies by evidence quality. The manual filtering criterion—removing papers that "were neither published at a scientific venue nor had any citations" (Section I-B)—is a minimum quality floor, not a rigorous vetting. A paper with one citation from a peer-reviewed venue passes this filter, but may still have methodological weaknesses (small sample sizes, non-standard evaluation, cherry-picked examples) that affect the reliability of its findings.

The consequence. Several of the survey's headline quantitative findings come from individual studies without documented replication: the 10× duplication → 1000× generation frequency relationship (Kandpal et al., 2022; Section IV-B1); the 0.00001% extractable memorization rate for GPT-2 (Carlini et al., 2021; Section II-B0a); the 38% correct-to-incorrect revision rate (not in this survey—this is from the reference example, but illustrates the pattern of single-study numbers being treated as established facts). The survey does not systematically distinguish between findings that have been replicated across multiple independent studies (e.g., approximately 1% discoverable memorization at 50-token prefixes, replicated by Carlini et al., 2023; Anil et al., 2023; and Kudugunta et al., 2023) and findings that rest on a single study. A reader cannot determine, from the survey's presentation, which quantitative claims are robust and which are provisional.

The small-sample problem is particularly acute for studies that use human verification. Carlini et al. (2021) confirmed memorization of about 0.00001% of GPT-2's training data via manual Google search verification. The sample size and verification methodology (manual search) make this a lower bound, but the survey reports it alongside automated large-scale extraction rates (0.1–1% from Nasr et al., 2023) without highlighting that the 100× to 10,000× difference may partially reflect measurement methodology rather than underlying memorization differences. A practitioner reading these numbers might incorrectly conclude that larger models (studied by Nasr et al.) memorize 100–10,000× more than GPT-2, when part of the gap is attributable to more exhaustive automated verification finding memorization that manual search missed.

What evidence exists in the paper. The survey does not include a study quality assessment, a risk-of-bias table, or a meta-analytic weighting scheme. Section I-B describes the paper selection process but not quality criteria beyond the citation/publication floor. Table III catalogs methodological choices (model, dataset, decoding, prefix length, match length) but does not assess whether studies using suboptimal choices (e.g., very short match lengths that may produce false positives) should be downweighted. The survey's conclusions are presented as consensus findings from the aggregated literature, but the aggregation is unweighted—every cited study contributes equally to the narrative synthesis regardless of its methodological rigor.

Mitigation status. The survey does not address this limitation. It does not claim to be a systematic review with pre-registered quality criteria, and its selection process is explicitly described as iterative and hybrid (Section I-B). This is appropriate for a narrative survey of a fast-moving field, but it means that the survey's conclusions should be understood as a curated expert synthesis rather than a systematic evidence synthesis with formal quality weighting. The distinction matters for readers who may treat the survey's quantitative claims (1% memorization, 10× duplication effect, log-linear scaling) as meta-analytic averages when they are actually headline findings from individual studies selected through a non-systematic process. The GitHub repository mentioned in the Abstract as a living document partially addresses the timeliness concern but does not address the quality-weighting concern.

7. Implications and Future Directions

How This Work Changes the Landscape

This survey does not propose a new algorithm, model, or measurement method. It is a conceptual diagnostic tool—a three-dimensional taxonomy that restructures how the research community thinks about memorization in LLMs. The magnitude of its contribution is a reframing, not a paradigm shift: the fundamental empirical findings (log-linear scaling with model size, ~1% discoverable memorization, super-linear duplication effects) were already established. What the survey changes is the organizing logic by which those findings are interpreted, compared, and acted upon.

Before this survey, the memorization literature was characterized by what the paper diagnoses as measurement fragmentation—Table III documents that studies used incompatible protocols (different prefix lengths, match lengths, decoding methods, and model families) while measuring different regions of the phenomenon (60% discoverable vs. 40% extractable, 60% verbatim vs. 40% approximate/entity-level) and treating their findings as though they captured the same underlying quantity. The survey resolves this contradiction not by proposing a unified metric, but by providing the vocabulary for articulating precisely which type of memorization a given method measures. A study described as measuring "verbatim × discoverable × undesirable with greedy decoding and 50-token match length on Pythia/PILE" can now be compared only to studies with the same specification, and recognized as non-comparable to studies measuring different taxonomic regions. This is a methodological infrastructure contribution: it makes rigorous comparison possible where none was previously available.

The survey's second reframing is the desirability thesis in Section II-C—the claim that memorization "always carries an undesirable risk" even when it provides functional advantages, because the same model capacity enabling factual recall is the capacity adversaries exploit for extraction. This is a philosophical reframing rather than an empirical finding (the survey does not cite studies quantifying the benefit-harm tradeoff), but it has concrete consequences for research prioritization. Under the prior implicit assumption that memorization could be categorized as simply "good" (knowledge retention) or "bad" (privacy leakage), mitigation research could aim for elimination of the bad while preserving the good. The survey's thesis implies this is structurally impossible—mitigation always involves trading off competing stakeholder interests, and the goal is not elimination but calibrated reduction matched to the specific risk profile of the deployment context. This changes the evaluation criteria for mitigation methods from "does it reduce memorization?" to "does it reduce the specific type of memorization that causes harm for the specific stakeholders in this deployment, at acceptable utility cost?"

The survey makes one research direction substantially more attractive: difficulty-aware, context-specific memorization assessment and mitigation. The taxonomy's core message is that uniform approaches fail because memorization is not one thing. This motivates research on type-specific mitigation—methods that target verbatim extractable PII memorization while preserving factual recall, or that suppress entity-level associations without degrading linguistic pattern learning. Conversely, the survey makes less attractive the research direction of developing ever-more-sensitive generic memorization metrics without matching them to specific risk models. A new metric that detects memorization at 0.001% instead of 1% provides marginal value if it does not distinguish between memorization of public knowledge (posing no privacy risk) and memorization of private PII (posing severe risk). The taxonomy provides the framework for designing metrics with discriminative validity—metrics that are sensitive to the types of memorization that matter for a given application and insensitive to types that do not.

The survey also implicitly resolves the tension between optimistic and pessimistic findings on memorization mitigation. Studies finding that deduplication reduces memorization by 10× (Lee et al., 2022) and studies finding that DP-SGD leaves models vulnerable to strong attacks (Lukas et al., 2023) appear contradictory only if memorization is treated as a single quantity. The taxonomy reveals that deduplication primarily targets verbatim × discoverable memorization (sequences that are reproduced when prompted with training prefixes), while extraction attacks target verbatim × extractable memorization (sequences reproducible without training-data knowledge). A method can be highly effective against one type while leaving another intact. This resolution does not make the tension disappear—it explains its source, enabling researchers to design mitigation strategies that address specific taxonomic regions rather than claiming universal effectiveness.

Follow-Up Research This Work Enables

A multi-metric memorization benchmark mapping the correlation structure of the taxonomic cube. The survey's central claim is that the three dimensions—granularity, retrievability, desirability—capture distinct aspects of memorization. But no study the survey catalogs simultaneously measures multiple granularity levels (verbatim, approximate, entity-level, content) and multiple retrievability regimes (extractable, discoverable) on the same set of models. A benchmark that evaluates 5–10 open-source models (spanning the Pythia, GPT-Neo, LLaMA, and OLMo families at scales from 125M to 70B parameters) on all 4×2 = 8 granularity × retrievability combinations would directly test whether these dimensions are orthogonal (measuring distinct phenomena) or redundant (highly correlated such that measuring one is sufficient). The survey's taxonomy predicts low-to-moderate correlations across dimensions—if correlations are instead very high, the taxonomy's practical value is reduced. This benchmark would also establish baseline rates for each taxonomic region, enabling future mitigation studies to report effectiveness per-region rather than on a single aggregate metric. The survey provides the measurement protocol specifications needed to standardize this benchmark (prefix lengths, match lengths, decoding methods per region, as catalogued in Table III).

A head-to-head mitigation comparison with per-taxonomic-region effectiveness reporting. The survey's Remark in Section V-C notes that "no single method fully eliminates memorization, and careful combination is needed," but provides no evidence about which combinations work or how their effects compose. A study training identical base models (same architecture, same pretraining data, same compute budget) under five conditions—(1) no mitigation, (2) deduplication only, (3) DP-SGD only at multiple ε values, (4) goldfish loss only, (5) deduplication + DP-SGD combined—and measuring memorization using the multi-metric benchmark described above would produce the utility-memorization tradeoff curves the survey cannot currently provide. Reporting effectiveness separately for verbatim × extractable (privacy-relevant), verbatim × discoverable (copyright-relevant), entity-level × extractable (PII-leakage-relevant), and content × discoverable (factual-recall-relevant) memorization would reveal whether mitigation methods are type-specific or type-general. The survey's catalog of influencing factors (Section IV) provides the candidate control variables (model scale, duplication rate, training duration, decoding method) that must be held constant or systematically varied to isolate mitigation effects.

Scale extrapolation: do the log-linear trends hold beyond 12B parameters? The survey's headline quantitative relationships—log-linear model-size-to-memorization scaling (Carlini et al., 2023), 10× duplication producing 1000× generation frequency (Kandpal et al., 2022), ~1% discoverable memorization at 50-token prefixes (Carlini et al., 2023; Anil et al., 2023; Kudugunta et al., 2023)—are established on models up to ~12B parameters trained on PILE-scale datasets. Whether these relationships continue, saturate, or change character at 70B, 175B, or larger scales is unknown. A study evaluating the Pythia scaling suite (70M to 12B) alongside LLaMA-2 variants (7B, 13B, 70B) or OLMo variants using identical measurement protocols (50-token prefix, greedy decoding, same match-length threshold) would test whether the log-linear trend extrapolates. The survey's factor taxonomy (Section IV-A1) identifies model capacity as the dominant factor, but the evidence for this is bounded in scale—a finding that memorization saturates at larger scales (e.g., because models transition from storing individual examples to learning more compressed abstractions) would fundamentally change the risk calculus for frontier models.

Conversational agent extraction attack methodology. The survey notes that Nasr et al. (2023)'s divergence attacks on conversational-aligned LLMs "are not powerful enough to stimulate training data reproduction" (Section VI-A), but it also cites Kassem et al. (2024)'s finding that instruction-based prompt optimization can reveal "as much or more training data as base models." These findings are in tension: either conversational agents are genuinely more resistant to extraction (the safety training works), or existing attacks are insufficiently adapted to the dialogue format. A study systematically varying attack methodology—base-model-style prefix prompting, instruction-based prompting (Kassem et al., 2024), role-playing scenarios, multi-turn dialogue extraction, and optimized soft prompting (Wang et al., 2024)—against the same conversational models (Alpaca, Vicuna, Llama-2-Chat, GPT-3.5-Turbo) using the same verification pipeline would resolve this tension. The survey's retrievability dimension (extractable vs. discoverable) provides the framework: conversational agents may suppress extractable memorization (adversary without training-data knowledge cannot trigger reproduction) while leaving discoverable memorization intact (the model still stores the information), and the attack methodology determines which regime is being tested. A finding that no attack methodology achieves non-trivial extraction from safety-trained models would validate current alignment approaches; a finding that specific attack types succeed would identify failure modes requiring targeted mitigation.

RAG memorization: does retrieval reduce parametric memorization or create new leakage vectors? The survey identifies a structural duality in RAG systems (Section VI-B): retrieval-augmented generation can reduce reliance on memorized parametric knowledge (because facts are retrieved rather than stored), but it introduces a new attack surface (the retrieval corpus may contain private documents that the model reproduces verbatim). A controlled study comparing three conditions—(1) base LLM without retrieval, (2) RAG with a public retrieval corpus, (3) RAG with a corpus containing injected canary documents—on both standard parametric memorization metrics (discoverable and extractable) and a new "contextual memorization" metric (does the model reproduce retrieved documents verbatim?) would characterize this dual effect. The survey's granularity dimension distinguishes parametric verbatim memorization (stored in weights) from contextual verbatim reproduction (read from retrieved documents), and a finding that RAG reduces the former while introducing the latter would have direct implications for deployment architecture (e.g., access control on the retrieval corpus becomes the primary privacy mechanism, making parametric memorization mitigation less critical).

Cross-lingual memorization measurement to test whether current findings are English-specific. The survey documents that prevailing memorization research is concentrated on English-language models trained on English data (Pythia/PILE, GPT-Neo/PILE), and it cites early evidence that "lower-resource languages show different memorization patterns" (Satvaty et al., 2025) and that "memorization patterns are shaped by cross-lingual similarities rather than data volume alone" (Luo et al., 2025). A systematic study measuring verbatim discoverable memorization rates across 10–20 languages with varying training data volumes in a single multilingual model (e.g., BLOOM, mT5, or Aya) would test whether memorization scales with per-language data volume (as the duplication-memorization relationship would predict), with linguistic structure (agglutinative vs. isolating languages may have different tokenization-driven memorization profiles per Kharitonov et al., 2022), or with cross-lingual transfer (high-resource language memorization enabling low-resource language generalization). The survey's taxonomy provides the standardized measurement protocol; the novelty is extending it across languages while controlling for model architecture and training regime (held constant within a single multilingual model).

Practical Applications and Downstream Use Cases

Privacy auditing for LLM deployments using the taxonomic framework to scope assessments. Organizations deploying LLMs that process user data—customer support chatbots, enterprise document summarization, medical record analysis—need to assess memorization risk before deployment. The survey's taxonomy provides the scoping instrument: an auditor can specify which granularity levels are relevant (entity-level for PII protection, verbatim for trade-secret protection, content-level for proprietary-knowledge protection) and which retrievability regime to test against (extractable for external-attacker threat models, discoverable for internal-audit or data-subject-access-request scenarios). Currently, audits relying on generic verbatim match rates may report low memorization while missing entity-level associations that violate GDPR's data protection requirements—the survey's documentation that different measurement methods capture different types of memorization (Section III, Table III) directly motivates type-specific auditing protocols. Concretely: an auditor evaluating a medical chatbot would measure entity-level × extractable memorization (can patient names be linked to conditions without training-data access?) rather than verbatim × discoverable memorization (can exact training sentences be reproduced when prompted with training prefixes?), because the former corresponds to the actual privacy threat while the latter overestimates practical risk in a conversational deployment.

Training data curation decisions informed by the duplication-memorization super-linear relationship. The survey's headline quantitative finding—10× duplication producing approximately 1000× generation frequency (Kandpal et al., 2022, Section IV-B1)—provides a concrete cost-benefit framework for deduplication decisions. For datasets where a small fraction of documents (say, 1% of the corpus) appears with high duplication (10+ occurrences), these documents will dominate memorized outputs by a factor of ~1000× compared to singly-occurring documents, despite representing only a small fraction of training data volume. A data curator can use this to prioritize deduplication effort: near-deduplication of the most-duplicated 1% of the corpus may eliminate a disproportionate fraction of memorization risk while preserving most of the training data (and thus most of the model's knowledge). Lee et al. (2022)'s finding that C4 contains up to 19.4% exact duplicates and 13.6% near duplicates (Section V-A) provides the scale calibration—for web-scale datasets, this "duplicate-heavy tail" is substantial enough that targeted deduplication is a high-leverage intervention. The practical process is: (1) run exact-match and MinHash deduplication on the corpus, (2) identify the duplication frequency distribution, (3) set a deduplication threshold based on the super-linear scaling relationship to achieve a target memorization reduction while retaining a target data volume.

Model release decisions incorporating extractable memorization risk as a key metric. When open-sourcing LLMs (as Meta did with LLaMA, EleutherAI with Pythia, and AllenAI with OLMo), the releasing organization typically evaluates performance benchmarks (MMLU, HellaSwag, HumanEval) and safety benchmarks (TruthfulQA, ToxiGen). The survey's retrievability dimension—specifically the distinction between discoverable and extractable memorization—implies that model release should also include an extractable memorization audit as a standard component of the release card. A model that shows 1% discoverable memorization (the standard reported figure) but only 0.001% extractable memorization (using best-available extraction attacks) poses a fundamentally different risk profile from a model with comparable discoverable rates but 0.1% extractable rates, yet these numbers are never reported in standard model cards. The survey provides the measurement protocol specifications: prompt the model with prefixes drawn from a held-out test corpus (discoverable), and with adversarially optimized prompts constructed without training-data access (extractable), reporting both numbers. A finding of low extractability relative to discoverability would inform downstream users that the model can be safely deployed with standard prompting interfaces, while a finding of high extractability would signal that additional output filtering, rate limiting, or monitoring is needed.

When to Prefer Which Mitigation Strategy

The survey does not articulate a clear tradeoff between named alternative mitigation methods based on comparative evidence—it catalogs methods rather than comparing them. The following decision framework is therefore inferred from the survey's organization and the individual study findings it reports, not presented as an explicit tradeoff the paper itself makes:

Prefer deduplication as a first-line mitigation when you have access to the training corpus before training begins. Lee et al. (2022) demonstrate approximately 10× memorization reduction in no-prompt generation (Section V-A), and Kandpal et al. (2022) establish the mechanistic link (super-linear duplication-memorization scaling). Deduplication addresses memorization at its source—removing the duplicated data that drives disproportionate memorization—rather than modifying the training algorithm or model architecture. It has no direct utility cost beyond the data removed, and the removed data is by definition redundant (duplicates). However, the survey notes that prompted memorization persists on deduplicated models when prompts come from duplicate-adjacent data (Lee et al., 2022: the model still generates ground truth "more often when the prompt is sampled from the duplicate examples"), so deduplication is partial rather than complete protection. The operational cost is a one-time preprocessing step (exact-match and MinHash algorithms over the full corpus).
Consider DP-SGD when formal privacy guarantees are required (e.g., training on sensitive user data under regulatory requirements) and you accept the utility cost. Carlini et al. (2019) demonstrate that adjusting ε can reduce exposure to indistinguishable-from-random levels, and Li et al. (2022) show that careful hyperparameter tuning can preserve competitive performance on NLP tasks. The tradeoff is well-characterized: lower ε provides stronger privacy guarantees at higher utility cost. However, the survey cautions (citing Lukas et al., 2023) that models trained with commonly-used ε values may not withstand stronger extraction attacks, so the effective privacy protection depends on ε selection and attack methodology. DP-SGD is appropriate when the threat model requires provable (rather than empirical) privacy guarantees, and when the deployment can tolerate the increased training time and potential accuracy degradation.
Use training interventions (goldfish loss, alternating teaching, memorization sinks) for targeted memorization reduction without full DP overhead when you can modify the training objective but need to maintain stronger utility than DP-SGD allows. The survey reports that these methods can "significantly reduc[e] extractable memorization while maintaining downstream performance" (Section V-B3), but the evidence comes from individual proof-of-concept studies (Hans et al., 2024 for goldfish loss; Liu et al., 2023 for alternating teaching; Ghosal et al., 2025 for memorization sinks) without head-to-head comparison or replication. These methods are appropriate for research and development settings where formal privacy guarantees are not required but empirical memorization reduction is desired. The survey does not provide guidance on when to choose one intervention over another—selection currently depends on which method's mechanism (token-dropping, teacher-student, neuron isolation) best matches the deployment constraints.
Apply post-training unlearning or model editing for removing specific memorized content from an already-trained model when retraining is infeasible and the content to be removed is identifiable. The survey cites methods that enable selective removal without full retraining (Section V-C1, V-C2), but notes that these methods "rely on identifying memorized content, risk unintended side effects, and cannot fully address hidden memorization or reasoning influences" (Section V-C Remark). The practical constraint is that unlearning requires knowing what to unlearn—if you cannot identify all instances of memorized PII in the model's training data, you cannot remove them. This makes these methods appropriate for targeted removal (e.g., removing a specific individual's data upon request) rather than general memorization reduction.
Do not rely on decoding-based methods (MemFree decoding, nucleus sampling) as the sole mitigation. The survey reports that these methods provide "only modest protection" (Borec et al., 2024) and that MemFree decoding can be "bypass[ed]" through "a simple style transfer in prompts" (Ippolito et al., 2023). They may be useful as a defense-in-depth layer applied at inference time on top of training-time or data-level mitigations, but the survey's evidence indicates they are insufficient as a standalone strategy.