Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs

ArXiv: 2508.06601

🎯 Pitch

Simply removing dangerous text from training data makes LLMs over 10× more resistant to adversarial fine-tuning than any post-training safety method, without degrading general performance. But this defense can be bypassed by providing the missing knowledge at inference time—so it must be paired with other safeguards.

---

1. Executive Summary

This paper investigates whether filtering dual-use text from pretraining data can build tamper-resistant safeguards into open-weight LLMs, using biothreat proxy knowledge as a representative test case. Training multiple 6.9B-parameter Pythia-architecture models from scratch on the DCLM dataset with a multi-stage filtering pipeline (a keyword blocklist escalated to a fine-tuned ModernBERT classifier), the authors find that filtered models resist adversarial fine-tuning attacks for up to 10,000 steps and 300M tokens of biothreat-related text—improving tamper-resistance by over an order of magnitude over post-training baselines like Circuit-Breaking and Latent Adversarial Training—with no observed degradation to unrelated capabilities on MMLU, PIQA, LAMBADA, and HellaSwag. The paper further demonstrates that while pretraining data filtering and Circuit-Breaking techniques are complementary (filtering prevents internalized knowledge acquisition while Circuit-Breaking impairs in-context retrieval), no defense withstands staged attacks combining fine-tuning with retrieval augmentation, establishing that defense-in-depth strategies remain necessary but incomplete for managing open-weight model risks.

2. Context and Motivation

The Core Problem: Protecting Open-Weight Models from Post-Release Tampering

The paper addresses a specific, urgent tension at the heart of modern AI governance: open-weight models offer immense benefits for research transparency and decentralized access, but their weights—once released—can be modified arbitrarily by downstream actors to elicit harmful capabilities that the original developers tried to suppress. The paper opens with a quote from the International AI Safety Report that frames the stakes precisely: once model weights are available for public download, "there is no way to implement a wholesale rollback of all existing copies of the model" (Bengio et al., 2025). This is the fundamental governance problem—closed-weight API deployments can rely on input/output monitoring, content filters, and the ability to revoke access when misuse is detected. Open-weight deployments have none of these guarantees.

The commercial relevance of this problem is accelerating. The paper documents that frontier developers are increasingly raising alarms about their own models' potential for harmful use. Google DeepMind reported that upcoming revisions to Gemini 2.5 Pro could "reach the critical capability level [for harmful novice uplift]" in biological domains (Google AI / DeepMind Gemini Team, 2025). Anthropic activated its Safety Level 3 protocols for Claude Opus 4, stating they "could not rule out" the model's ability to significantly assist individuals "in obtaining, producing, or deploying CBRN weapons" (Anthropic, 2025). OpenAI precautiously classified ChatGPT Agent as "high capability in the Biological and Chemical domain" (OpenAI, 2025b). These are not hypothetical scenarios—frontier developers are implementing precautionary measures based on their internal capability evaluations.

But the open-weight dimension compounds this risk. The paper cites evidence that open-weight models' capabilities lag only six to twelve months behind closed-weight models (Cottier et al., 2024; Maslej et al., 2024), and they are released at an increasing rate (Bhandari et al., 2025). This means that within the near future, models with genuinely concerning dual-use capabilities—comparable to those that prompted the above safety protocol activations—will likely be available for unrestricted download and modification. The problem is not just speculative; it is a predicted near-term reality that the current science of safeguards is inadequately prepared to handle.

Why the Problem Has Proven Intractable: The "Wrapper" Hypothesis

The paper identifies a specific mechanistic hypothesis for why post-training safety techniques have consistently failed to produce tamper-resistant models. This hypothesis—quoted directly in the paper's opening from Jain et al. (2023)—states:

"Fine-tuning rarely alters the underlying model capabilities. A minimal transformation, which we call a 'wrapper', is typically learned on top of the underlying model capabilities, creating the illusion that they have been modified. Further fine-tuning on a task where such hidden capabilities are relevant leads to sample-efficient 'revival' of the capability."

This is not merely an empirical observation about past failures—it is a claim about the nature of what fine-tuning achieves. If safety fine-tuning only adds a shallow behavioral wrapper rather than erasing the underlying neural circuitry for harmful knowledge or capabilities, then any downstream actor with access to the weights can strip that wrapper away through further fine-tuning. The paper is motivated by the hypothesis that if a model never acquires dangerous knowledge during pretraining in the first place, there is no latent capability for fine-tuning to "revive." The degree of tamper-resistance achievable is then a function of how thoroughly the relevant training data can be identified and removed.

Where Post-Training Methods Fail

The paper provides a comprehensive accounting of the current state of tamper-resistance techniques and their consistent failure under sustained attack. Despite recent work on developing various defense mechanisms—including representation noising (Rosati et al., 2024), self-destructing models (Henderson et al., 2023), and various machine unlearning approaches—the paper states the consensus finding bluntly: "existing techniques can consistently be undone within several hundred fine-tuning steps or fewer" (citing Qi et al., 2024b; Huang et al., 2025; Che et al., 2025; Fan et al., 2025; Hu et al., 2024; Sheshadri et al., 2024; Łucki et al., 2024; Qian et al., 2025; Deeb & Roger, 2024).

How Circuit-Breaking (CB) works—and why it's insufficient as a standalone defense. The paper provides detailed technical exposition of two representative post-training baselines that it uses throughout the experiments. Circuit-Breaking (Zou et al., 2024) operates by training Low-Rank Adaptation (LoRA) adapters at multiple layers in the network (the paper uses layers 5, 10, 15, 20, 25, and 30) with a two-part loss: (1) preserve the neural activations induced by benign examples from a "retain" dataset, and (2) "reroute" the activations induced by harmful examples from a "forget" dataset so they become orthogonal to the originally induced activations. The aim is to scramble the neural processing of harmful inputs so that all linearly-encoded information about them is erased. Circuit-Breaking with Latent Adversarial Training (CB+LAT; Sheshadri et al., 2024) extends this by using an ensemble of latent-space adversaries at the same layers, performing four steps of adversarial optimization per training step to elicit and train against latent harmful capabilities that passive fine-tuning might miss. The key detail is that both approaches rewire the model's processing of harmful inputs without removing the underlying knowledge representations learned during pretraining.

The failure mode of these approaches is precisely what the "wrapper" hypothesis predicts. When CB or CB+LAT models are subjected to adversarial fine-tuning—fine-tuning specifically on biothreat-related scientific text (the WMDP-Bio Forget dataset)—the suppressed capabilities rapidly reemerge. Appendix E (Table 10) provides a systematic comparison of fine-tuning attacks across the literature, establishing that the paper's attacks (up to 10,000 steps, 300M tokens, 80,000 unique examples) are substantially larger in scale than those tested in prior work. The fact that filtered models resist even these extended attacks, while post-training approaches fail under far milder regimes, is the paper's central empirical contribution and the primary motivation for shifting the defense paradigm from post-training to pretraining.

The Specific Gap: No Robust Science of Open-Weight Model Risk Management

The paper identifies a larger gap beyond individual technique failures: "there is not yet a robust science of open-weight model risk management." This phrasing is significant because it positions the problem as requiring not just better methods but an organized body of knowledge similar to what exists for closed-weight deployments. The paper cites François et al. (2025) who note that "the science of open-weight model safety is nascent" despite the rising prominence of open-weight models.

The gap has multiple dimensions. First, no standardized framework exists for evaluating tamper-resistance. Different papers use different attack configurations, different model sizes, different data, and different metrics, making it impossible to compare claims across studies. The paper's Table 10 (Appendix E) documents this fragmentation explicitly, showing that prior work spans a range of 2 to 54,258 unique examples, 2 to 128 batch size, and 8 to 3,000 steps for fine-tuning attacks. Second, no prior work had demonstrated tamper-resistance beyond a few hundred fine-tuning steps at scales relevant to dual-use capabilities. The paper argues that demonstrating resistance to thousands of steps—an order of magnitude beyond the state of the art—is a necessary condition for any safety case based on robustness to tampering. Third, existing evaluations of capability suppression rely heavily on multiple-choice question answering benchmarks with known vulnerabilities to shortcut exploitation. The paper's own analysis (Section 2.4 and Appendix D.4) finds that simply selecting the longest answer on WMDP-Bio yields 46% accuracy—well above random chance (25%)—demonstrating that naive evaluation can dramatically overstate retained dangerous knowledge.

Conflicting Signals in the Literature: Why Data Filtering Is Controversial

The paper confronts a tension in the existing literature that complicates the straightforward narrative of "remove data, remove capability." Works by Maini et al. (2025) and Li et al. (2025a) studied filtering for toxic and harmful content at scales up to 2B parameters and found that "filtered models were sometimes less robust to certain types of input-space attacks than unfiltered safety-fine-tuned ones." The titles of these works capture their counterintuitive findings: "Safety is not about censorship" (Maini et al., 2025) and "Bad data may lead to good models" (Li et al., 2025a). Their shared hypothesis was that language models sometimes need to "understand" harmful behaviors in order to effectively resist exhibiting them—exposure to harmful training data provides the model with representations that make refusal and avoidance more learnable during safety fine-tuning.

The paper directly grapples with this tension in Section 6.2 and Appendix G, not by dismissing the prior findings but by proposing a domain-dependent hypothesis for when filtering works. The key distinction is between emergent propensities (toxicity, attempted compliance with harmful requests, aligning with a particular set of principles) and precise knowledge (scientific or engineering facts). The paper speculates that emergent propensities "do not require precise knowledge to be exhibited" and may benefit from exposure to harmful data during training to learn appropriate avoidance behavior. In contrast, precise knowledge "arises only from a small subset of training documents" and can be suppressed by removing those documents altogether. The paper's experiments on biothreat proxy knowledge—a domain of precise factual knowledge—support this hypothesis. The experiments in Appendix G on the models from Maini et al. (2025), testing jailbreaking resistance, support the complementary claim: filtering for toxic content does not provide durable safeguards against compliance with harmful requests, and may actually increase vulnerability to certain attacks.

Concurrent but Unspecified Industry Practice

The paper notes a striking gap between open research and industry practice. Several frontier model developers have publicly mentioned efforts to filter pretraining data for harmful content (Kamath et al., 2025 for Gemma 3; OpenAI's Deep Research System Card and GPT-OSS model card; Meta's Llama 4 announcement; Anthropic's Claude Opus 4 system card). However, the paper observes that none of these public communications "provides precise details about what data was filtered, how it was filtered, how much was filtered, or how the success of the filtering was evaluated." This opacity means that while data filtering may already be an industry standard practice, the scientific community lacks the open, reproducible models and evaluations necessary to systematically study its effectiveness, failure modes, and scaling properties.

The paper's release of a suite of 6.9B-parameter models with documented filtering decisions, training configurations, and evaluation protocols is explicitly positioned as filling this gap—providing a testbed where the causal impact of removing specific subsets of training data can be studied mechanistically.

How This Paper Positions Itself

The paper frames its contribution not as proposing a new method but as demonstrating a paradigm shift in where safety interventions occur in the model development pipeline. The title itself—"Deep Ignorance"—captures the core idea: that preventing knowledge acquisition during pretraining produces a fundamentally different kind of safety property than suppressing already-acquired knowledge through post-training.

The paper also explicitly avoids claiming that data filtering solves the open-weight safety problem. The limitations are enumerated clearly: filtering cannot prevent models from leveraging harmful knowledge provided in-context (Section 4.1), cannot suppress emergent propensities like toxicity (Section 6.2), and does not withstand combined fine-tuning + retrieval-augmentation attacks (Section 4.1). The positioning is deliberately incremental: data filtering is "a promising layer of defense" within a defense-in-depth strategy, not a standalone solution. The contribution is establishing that pretraining data curation can be a tractable and effective component of open-weight risk management, with specific empirical benchmarks for how effective it can be and under what conditions it fails.

3. Technical Approach

This is primarily an empirical systems paper whose core idea is that removing dual-use text from pretraining data prevents a model from acquiring dangerous knowledge in the first place, yielding fundamentally more tamper-resistant safeguards than post-training techniques that can only suppress already-learned capabilities.

3.1 Reader Orientation

The paper builds a system for filtering training data at scale to prevent language models from learning biothreat proxy knowledge during pretraining, then evaluates whether the resulting "ignorant" models resist downstream attacks that attempt to reintroduce that knowledge. The core hypothesis is that if a model never learns unsafe information during pretraining, fine-tuning attackers have nothing to "revive" — unlike post-training safeguards which, per Jain et al. (2023), merely learn a shallow behavioral wrapper over intact latent capabilities that subsequent fine-tuning easily strips away.

3.2 Big-Picture Architecture (Diagram in Words)

The system has six major components:

1. Blocklist Generator — uses Llama 3.3 70B to extract biorisk-relevant keywords from gold-standard proxy documents; produces a list of 6,178 terms used for fast, CPU-bound document scanning.

2. Multi-Stage Filtering Pipeline — processes every document in the training corpus through two stages: (a) a keyword blocklist that flags any document containing two or more prohibited terms, and (b) an escalated review by a fine-tuned ModernBERT classifier for flagged documents that determines whether the document actually contains biothreat proxy content.

3. Training Corpus Construction — takes the original DCLM dataset and produces filtered versions by removing documents identified by the filtering pipeline, with filtered documents optionally replaced by blocklist-escalated but classifier-approved documents to maintain a similar prevalence of scientific data.

4. Model Training — trains 6.9B-parameter Pythia-architecture transformers from scratch on either unfiltered data (Baseline), single-stage filtered data (Strong Filter), or two-stage filtered data (Weak Filter), using a staged approach of 500B tokens of pretraining followed by 50B tokens of annealing with domain-specific data upsampled.

5. Post-Training Safeguards (Circuit-Breaking baselines) — applies Circuit-Breaking (CB) with or without Latent Adversarial Training (LAT) to trained models, training LoRA adapters at layers 5, 10, 15, 20, 25, and 30 to reroute neural activations for harmful inputs while preserving activations for benign inputs.

6. Evaluation Framework — measures both general capabilities (MMLU, PIQA, LAMBADA, HellaSwag) and biothreat proxy knowledge (two curated subsets of WMDP-Bio designed to mitigate multiple-choice shortcut exploitation) under normal, input-space attack, latent-space attack, adversarial fine-tuning, and benign fine-tuning conditions.

Information flows as follows: the original DCLM dataset enters the filtering pipeline → each document passes through the blocklist stage (91.58% pass without escalation during pretraining) → escalated documents receive ModernBERT classifier scores → documents above threshold are removed → the remaining data is split into pretraining (500B tokens, 409.9M documents) and annealing (50B tokens, 89.1M documents) phases → models are trained on these corpora from scratch → post-training safeguards are optionally applied → models are evaluated under multiple attack regimes.

3.3 Roadmap for the Deep Dive

• First, the multi-stage filtering pipeline — how the blocklist is generated, how it operates, how the ModernBERT classifier is trained, and what the two-stage architecture costs in compute. This is the central engineering contribution and the mechanism that determines what data the models see.
• Second, the language model training configuration — the staged training approach, the dataset composition, why annealing matters for the specific evaluations used, and the two filtering variants (Strong vs. Weak) that create the experimental comparisons.
• Third, the baseline post-training safeguards — how Circuit-Breaking and Latent Adversarial Training work technically, why they were chosen as baselines, and what their known failure modes are. This establishes the benchmark against which filtering is compared.
• Fourth, the evaluation methodology — how biothreat proxy knowledge is defined and measured, the specific problems with unmodified WMDP-Bio (shortcut exploitation, choice-only answering), and the two curated evaluation subsets that address these problems. This is critical for understanding what the performance numbers actually mean.
• Fifth, the FLOPs accounting for filtering — the calculation showing that the entire filtering pipeline costs less than 1% of training compute, establishing that the approach is computationally tractable at scale.

3.4 Detailed, Sentence-Based Technical Breakdown

The Multi-Stage Filtering Pipeline: Design and Implementation

The filtering pipeline is designed to solve a specific engineering problem: identifying a tiny fraction of potentially harmful documents within a pretraining corpus of hundreds of millions of diverse documents, at a computational cost that does not dominate the training budget. The solution is a two-stage cascade that spends minimal compute on the vast majority of safe documents while concentrating classifier resources on the small fraction that matches surface-level indicators of relevant content.

Stage 1 — Keyword Blocklist Generation and Operation. The blocklist is generated through a two-round process using Llama 3.3 70B (Grattafiori et al., 2024a). In the first round, the model iterates through all 15,231 "Gold" proxy documents from the WMDP-Bio Forget training set — expert-curated scientific papers containing biothreat proxy knowledge — and extracts scientific keywords that are "unlikely to appear in general text." In the second round, the model reviews each candidate keyword to confirm that it is, in isolation, relevant to biorisk. This produces a final blocklist of 6,178 terms.

An additional hyperparameter controls the quality of this blocklist: the pos-ratio, defined as the percentage of Gold proxy documents in which a keyword appears, divided by the percentage of Gold general biology documents (from the WMDP-Bio bio Retain set, containing 66,360 general biology papers) in which the keyword occurs. Formally, for each candidate term $t$:

$\text{pos-ratio}(t) = \frac{|\text{proxy documents containing } t| / |\text{total proxy documents}|}{|\text{general biology documents containing } t| / |\text{total general biology documents}|}$

where the numerator is the prevalence in the forget set and the denominator is the prevalence in the retain set.

What it computes: a discrimination ratio measuring whether a term appears disproportionately in proxy documents versus general biology documents. A value of 1.0 means the term is equally common in both; higher values indicate terms that are specific to the proxy documents.

Why this form: this ratio directly addresses the filtering precision-recall tradeoff. Terms that appear frequently in both proxy and benign biology texts (e.g., "DNA polymerase") would generate many false positives if included in the blocklist, requiring expensive classifier review of numerous safe documents. By thresholding on pos-ratio, the blocklist preferentially includes terms that are diagnostic of proxy content — specifically, the authors grid-searched over this hyperparameter on their validation set (described in Appendix D.1.1) and settled on a threshold of 0.4, meaning terms are retained only if proxy documents account for at least 40% of their occurrences across the labeled corpus.

The resulting blocklist is used as a CPU-bound string-matching operation over every document in the training corpus. Documents containing fewer than two blocked terms (a design choice to reduce false positives from single accidental matches) are approved without further review. Documents containing two or more blocked terms are escalated to the second stage. This threshold of two terms reduces the escalation rate: during pretraining, 91.58% of documents pass the blocklist without escalation, and during annealing, 90.64% of documents pass. This means the expensive classifier stage processes only 8.42% and 9.36% of documents respectively, making the pipeline computationally efficient.

The blocklist achieves near-perfect recall at the cost of modest precision. On an evaluation set composed only of WMDP documents, the blocklist achieves 98.02% recall and 58.28% precision (F1 of 73.10), correctly identifying virtually all proxy documents while flagging many false positives. When evaluated on a more realistic mixture that includes Llama-labeled DCLM documents (representing the diversity of actual pretraining data), precision drops to 44.02% (F1 of 60.80) while recall remains at 98.21%. This precision-recall tradeoff is intentional: the paper "erred on the side of over-filtering to minimize false negatives," accepting a higher false positive rate in the belief that missing proxy documents (false negatives) is the more serious error for safety.

Stage 2 — ModernBERT Classifier Training and Operation. The second stage is a fine-tuned ModernBERT-Large model (Warner et al., 2024) that performs binary classification: given a document, predict whether it contains biothreat proxy knowledge. ModernBERT was chosen over earlier BERT variants (BERT, SciBERT) because its 8,192-token context window allows it to process longer document chunks, reducing the need for document splitting and enabling better contextual understanding.

The training dataset for this classifier (Table 4, Appendix D.1.1) contains 198,184 documents split across three sources:

1. Gold-labeled documents from WMDP: 15,231 proxy papers (forget set) and 59,041 general biology papers (bio Retain set), providing expert-verified positive and negative examples.
2. Augmented documents: Llama 3.1 8B Instruct is used to rewrite each gold document in three alternative formats — as a lecture transcript, an exam, or an article — using few-shot prompts with examples sourced from Claude 3.5 Sonnet. This produces 45,801 proxy augmentations and 18,575 general biology augmentations. This augmentation step addresses a critical limitation: the gold documents are exclusively scientific papers from PubMed, but pretraining datasets contain diverse formats including lecture transcripts, news articles, and instructional content. Without augmentation, a classifier trained only on paper-format documents might fail to recognize proxy knowledge presented in other formats.
3. Llama-labeled DCLM documents: approximately 75,000 documents from DCLM that were escalated by the blocklist are labeled by Llama 3.3 70B using an optimized prompt (Appendix I.8). This directly addresses distribution shift: the classifier is trained not only on clean PubMed papers but also on actual web documents from DCLM that the blocklist flagged, making its training distribution closer to the inference distribution.

The classifier is trained with full-parameter fine-tuning (not just a classification head) using the following hyperparameters:

• Optimizer: AdamW (Loshchilov & Hutter, 2017)
• Learning rate: $1 \times 10^{-4}$
• Weight decay: 0.01
• Batch size: 4 per device with gradient accumulation of 16 (effective batch size of 64)
• Training duration: 1 epoch
• Learning rate schedule: linear decay with no warmup
• Adam parameters: $\beta_1 = 0.9$, $\beta_2 = 0.999$, $\epsilon = 10^{-8}$
• Gradient clipping: maximum norm of 1.0
• Training precision: full FP32 (mixed precision disabled)

At inference time, documents that exceed ModernBERT's 8,192-token context window are processed in chunks; a document is flagged as proxy content if any chunk exceeds the classification threshold. The paper settled on a threshold of 0.0105, which was chosen through minimal hyperparameter optimization (the authors "opted to pursue minimal hyperparameter optimization to mitigate the risk of overfitting") and resulted in approximately 5% of annealing documents being filtered.

Strong vs. Weak Filter Configurations. The paper defines two filtering approaches that differ in whether the ModernBERT escalation stage is used:

• Strong Filter (single-stage): The blocklist operates alone. Any document containing two or more blocked terms is removed from the corpus without further review. This removes 8.42% of pretraining documents and 9.36% of annealing documents. The strong filter maximizes recall at the cost of lower precision — it removes everything the blocklist flags, including many false positives.
• Weak Filter (two-stage): Documents flagged by the blocklist are escalated to the ModernBERT classifier for review. Only documents that the classifier also flags (score > 0.0105) are removed. Filtered documents are replaced with documents that were escalated by the blocklist but approved by ModernBERT, to maintain a similar prevalence of scientific data after filtering. This removes 4.96% of annealing documents — approximately half as many as the strong filter — because many blocklist-flagged documents are cleared by the classifier.

The two configurations represent different points on the precision-recall spectrum, with the strong filter being more aggressive (fewer false negatives, more false positives) and the weak filter being more targeted. The paper experiments with applying different filtering strengths to different training phases — for example, the primary model used in Section 3 ("strong-filter-pt-weak-filter-anneal") uses strong filtering during the 500B-token pretraining phase and weak filtering during the 50B-token annealing phase.

Importantly, the paper also defines an Extra Weak Filter variant where the ModernBERT threshold is set to 0.5 instead of 0.0105, resulting in only 0.94% of pretraining documents and 2.02% of annealing documents being filtered. This variant, while not the main focus of experiments, demonstrates that the filtering aggressiveness can be continuously adjusted via the classifier threshold.

Filtering's Computational Cost. The paper provides a detailed FLOPs accounting (Appendix C) demonstrating that the filtering pipeline is computationally negligible relative to training. The total estimated FLOPs for creating and applying the most expensive filtering setup (end-to-end weak filtering applied to both pretraining and annealing) is calculated as:

$\text{FLOPS}_{\text{total}} = \text{FLOPS}_{\text{train}} + \text{FLOPS}_{\text{filter}}$

where $\text{FLOPS}_{\text{train}}$ is the total FLOPs for model training:

$\text{FLOPS}_{\text{train}} = 8.32 \times P \times D = 8.32 \times 6.86 \times 10^9 \times 5.50 \times 10^{11} \approx 3.14 \times 10^{22} \text{ FLOPs}$

Here $P = 6.86 \times 10^9$ is the number of non-embedding model parameters (the Pythia 6.9B architecture), $D = 5.50 \times 10^{11}$ is the total number of training tokens (500B pretraining + 50B annealing), and the constant 8.32 (rather than the standard Kaplan et al. (2020) factor of 6) accounts for activation checkpointing, which trades additional forward passes during backpropagation for reduced GPU memory.

The filtering FLOPs decompose into four components (Table 3):

• Llama 3.3 70B distillation for blocklist generation: $4.45 \times 10^{19}$ FLOPs
• Llama 3.3 70B synthetic data generation for classifier training: $1.33 \times 10^{20}$ FLOPs
• ModernBERT classifier training: $6.08 \times 10^{18}$ FLOPs
• Multi-stage filtering inference (pretraining): $6.92 \times 10^{19}$ FLOPs
• Multi-stage filtering inference (annealing): $7.77 \times 10^{18}$ FLOPs

Summing to $\text{FLOPS}_{\text{filter}} = 2.62 \times 10^{20}$ FLOPs. The percentage overhead is then:

$\frac{\text{FLOPS}_{\text{filter}}}{\text{FLOPS}_{\text{train}}} \times 100\% = \frac{2.62 \times 10^{20}}{3.14 \times 10^{22}} \times 100\% = 0.83\%$

What this computes: the total additional computation required for data filtering as a fraction of the computation already required for model training. For the weak filter configuration (multi-stage only during annealing, single-stage blocklist during pretraining), the overhead drops to 0.61%.

Why this form matters: this calculation demonstrates that data filtering is not merely effective but also tractable — it does not meaningfully increase the total cost of training. This addresses a potential practical objection: if filtering cost 20% or 50% of training compute, organizations might judge it economically infeasible. At less than 1%, the filtering overhead is comparable to typical data preprocessing costs and should not be a barrier to adoption. The paper notes that this fraction would likely decrease further with blocklist optimization (reducing the documents escalated to the classifier) and when training larger models (where the $3.14 \times 10^{22}$ training FLOPs denominator grows faster than the largely fixed filtering FLOPs numerator).

Qualitative Analysis of Filtered Data. While the paper does not measure classifier precision/recall on the actual pretraining corpus (due to the prohibitive cost of labeling), Appendix J provides a qualitative examination of what types of documents tend to be filtered. The vast majority of filtered documents come from Semantic Scholar (Pes2o), which is "unsurprising, as scientific documents comprise the majority of the training and evaluation datasets for our filters." Common filtered content includes biomedical, public health, and virology papers. False positives are observed in DCLM documents discussing economic and social impacts of the COVID-19 pandemic rather than technical biology, and Wikipedia articles about pandemics that are non-technical. The paper is transparent that "our filters likely have high false-positive rates," though the minimal impact on general capability benchmarks (Section 2.6, Figure 3) suggests that the removed content is a small enough fraction of the total corpus that the model can compensate through other training data.

Language Model Training Configuration

Staged Training Approach. The paper uses a two-phase training procedure: pretraining on 500B tokens of general web data, followed by annealing on 50B tokens of higher-quality domain-specific data. This staged approach (also used by OLMo-2 (OLMo et al., 2024) and Llama 3 (Grattafiori et al., 2024a)) improves performance on knowledge-intensive benchmarks by exposing the model to a higher concentration of academic, instructional, and domain-specific content in the final phase of training. The paper explicitly notes that the annealing phase refreshes the learning rate: a cosine schedule is applied over the full 500B-token pretraining phase, then reset and applied again over the 50B-token annealing phase (learning rate starts at $3.0 \times 10^{-4}$, decays to $1.2 \times 10^{-5}$ during pretraining, and resets to $3.0 \times 10^{-4}$ with decay to 0.0 during annealing; see Table 7).

Pretraining Dataset. The pretraining corpus uses a deduplicated version of DCLM (DataComp-LM, Li et al., 2024a), an English-language web corpus filtered for quality and diversity using model-based scoring. The deduplicated version, provided by ZyphraAI, contains approximately 500B tokens when tokenized with the GPT-NeoX tokenizer (the same tokenizer as Pythia and GPT-NeoX-20B), encompassing 409,935,485 individual documents.

Annealing Mixture Composition. The annealing mixture (Table 6) is designed to optimize performance on knowledge-intensive benchmarks while maintaining familiarity with instruction-following formats. The composition is:

• 50.00% DCLM (25B tokens, 20.5M documents): fresh DCLM data not seen during pretraining, maintaining broad web coverage
• 19.69% Instruction-following data (9.84B tokens): 16.87% from the Flan dataset (8.43B tokens, 28.6M documents) and 2.82% from StackExchange (1.41B tokens, 2.5M documents). The Flan dataset is explicitly noted as being "decontaminated against evaluation benchmarks," meaning the model should not have seen WMDP-Bio or other eval questions during training
• 30.31% Academic and reference data (15.16B tokens): 22.90% from Semantic Scholar's Pes2o dataset (11.45B tokens, 31.1M scientific papers), 7.37% from Wikipedia (3.68B tokens, 6.2M articles), and 0.04% from Camel datasets for biology, chemistry, and physics (0.01B tokens each, 20,000 documents each)

The paper explicitly states that this annealing mixture "contains a higher proportion of scientific content compared to OLMo-2" and was designed to "indirectly optimize for WMDP-Bio performance while maintaining realistic training parameters." In other words, the annealing data was deliberately enriched with biology-relevant content to ensure that baseline (unfiltered) models would perform well on the WMDP-Bio benchmark, creating a strong test for whether filtering could suppress that performance. This is a critical design choice: if the annealing mixture had contained very little biology content, it would be trivial for filtering to suppress WMDP-Bio performance (because the model never saw the relevant content anyway), and the results would not demonstrate meaningful capability suppression. By deliberately including substantive biology content — and then showing that filtering it away suppresses WMDP-Bio while preserving other capabilities — the experiment provides a stronger demonstration of targeted filtering effectiveness.

Model Architecture. All models use the Pythia 6.9B architecture (Biderman et al., 2023): a decoder-only transformer with 32 layers, hidden dimension of 4,096, 32 attention heads, GELU activation, LayerNorm normalization, rotary position encoding (applied to 25% of the hidden dimension), a sequence length of 2,048 tokens, and no weight tying. Training uses GPT-J residual style, flash attention, bfloat16 precision, and ZeRO stage 1 distributed training with activation checkpointing enabled. The models are trained on 128 Nvidia H100 GPUs with an effective batch size of 4,194,304 tokens (32 sequences per GPU × 2,048 tokens × 1 gradient accumulation step), taking approximately 15,632 GPU hours (about 5 days wall-clock time).

Optimization Configuration. The training uses the Adam optimizer with $\beta_1 = 0.9$, $\beta_2 = 0.95$, $\epsilon = 1.0 \times 10^{-8}$, weight decay of 0.1, and gradient clipping at 1.0. The learning rate follows a cosine schedule with a peak of $3.0 \times 10^{-4}$, 1% warmup steps, and decay to a minimum of $1.2 \times 10^{-5}$ during pretraining (reset with decay to 0.0 during annealing). Dropout is disabled (set to 0.0) throughout training.

Training Efficiency. The paper reports a Model FLOPs Utilization (MFU) of 0.56 when benchmarked against the theoretical peak H100 BF16 throughput of 989 TFLOPs/s, and an "achievable" MFU of 0.70 when benchmarked against the Maximum Achievable Matmul FLOPS (MAMF) of 794.5 TFLOPs/s. This indicates healthy training efficiency without significant bottlenecks.

Why no instruction fine-tuning? The paper explicitly "forgo[es] formal instruction fine-tuning because we focus on single-turn question answering evals, which do not require models to act as fluent chatbots or agents." This is a deliberate simplification: adding instruction fine-tuning would introduce an additional post-training stage that could interact with filtering effects in complex ways, making it harder to isolate the causal impact of data filtering. The annealing mixture's substantial fraction of instruction-following data (Flan, 16.87%) provides enough familiarity with question-answering formats — Appendix D.5 empirically verifies this, showing that adding explicit multiple-choice training to the filtering models improves WMDP-Bio performance by only approximately 2 percentage points (compared to 2 points for the unfiltered baseline), confirming that the annealing setup already provides sufficient test-task exposure.

Filtering Variants for Training. The paper trains models with five distinct filtering configurations (plus the unfiltered baseline), which are systematically tested in Section 2 and Section 3:

1. Unfiltered (Baseline): no filtering applied to either pretraining or annealing data
2. End-to-end Strong Filter: single-stage blocklist filtering applied to both phases, removing 8.42% of pretraining documents and 9.36% of annealing documents
3. Strong PT + Weak Anneal: strong filtering during pretraining, weak filtering (blocklist → ModernBERT) during annealing, removing 4.96% of annealing documents (this is the model with the best overall tamper-resistance in Section 3)
4. End-to-end Weak Filter: weak filtering applied to both phases
5. End-to-end Extra Weak Filter: weak filtering with ModernBERT threshold of 0.5 (instead of 0.0105), removing only 0.94% of pretraining and 2.02% of annealing documents

The paper also trains hybrid configurations (weak PT + strong anneal, strong filter PT-only, weak filter PT-only) that are reported in Appendix B (Table 2) but not in the main experiments. All filtering configurations are trained to the same total token count (550B) as the baseline, with filtered models training for slightly more than one epoch on their (smaller) datasets to match the same token volume. The models are not trained to competitive state-of-the-art performance; they were trained only until reaching 50% accuracy on the standard WMDP-Bio benchmark, with "remaining compute budget dedicated to filtering experiments."

Baseline Post-Training Safeguards: Circuit-Breaking and Latent Adversarial Training

The paper uses Circuit-Breaking (CB) and Circuit-Breaking with Latent Adversarial Training (CB+LAT) as the primary post-training baselines against which data filtering is compared. These are state-of-the-art capability suppression techniques, and the paper's demonstration that filtering substantially outperforms them under adversarial fine-tuning is the central empirical claim.

Circuit-Breaking (CB) — Detailed Mechanism. CB (Zou et al., 2024) operates by training Low-Rank Adaptation (LoRA) adapters (Hu et al., 2022) at multiple layers in the transformer. LoRA adapters are low-rank matrices that modify the model's linear transformations without changing the original weights, enabling parameter-efficient fine-tuning. The CB objective has two components:

1. Preservation loss: For examples from a benign "retain" dataset, the LoRA adapters are trained to preserve the original neural activations — the hidden states at each intervened layer should remain as close as possible to what the unmodified model would produce. This ensures that general capabilities are not degraded.

2. Rerouting loss: For examples from a harmful "forget" dataset, the LoRA adapters are trained to make the resulting activations orthogonal to the activations the unmodified model would have produced. The idea is to "scramble the neural processing of harmful examples so that all linearly-encoded information about them is erased" (Section 2.3).

The paper applies CB at layers 5, 10, 15, 20, 25, and 30 of the 32-layer Pythia architecture, following the layer selection from Zou et al. (2023a). This spans early, middle, and late layers of the network, intervening at multiple points in the processing pipeline rather than relying on a single-layer intervention that downstream layers might compensate for.

Circuit-Breaking with Latent Adversarial Training (CB+LAT). CB+LAT (Sheshadri et al., 2024) extends CB by adding an adversarial training component. At each training step, before updating the LoRA adapter parameters, the technique computes latent-space perturbations — small perturbations to the hidden activations at the intervened layers that are designed to make the model output the harmful behavior. Specifically, the paper uses "an ensemble of latent-space adversaries at layers 5, 10, 15, 20, 25, and 30 with 4 steps of adversarial optimization per training step." These perturbations are optimized using gradient descent to maximize the model's probability of generating harmful outputs. The LoRA adapters are then trained to resist these perturbed activations, in addition to the standard preservation and rerouting objectives.

The motivation for LAT is that standard CB may only suppress the "surface-level" neural processing of harmful inputs, leaving latent pathways that could be reactivated through careful optimization. LAT attempts to proactively elicit those latent pathways via adversarial perturbations, forcing the model to learn to suppress them. The paper reports (Section 2.3) that "CB and CB+LAT perform comparably overall but with CB+LAT tending to more strongly resist some input-space attacks."

Why CB Methods Were Chosen as Baselines. The paper selected CB and CB+LAT because they represent the state of the art in LLM capability suppression at the time of writing. They are among the most thoroughly evaluated post-training techniques, with documented performance against diverse attacks. Their failure mode — gradual degradation under sustained fine-tuning — is well-characterized and provides a clear benchmark for filtering to surpass. The paper also notes in Section 4.1 that CB and filtering are "in no way mutually exclusive," and that combining them yields improved robustness over either alone, making them not just baselines but components of a potential defense-in-depth strategy.

The "Wrapper" Problem in Practice. The key insight motivating the comparison between filtering and CB methods is the difference in what is being modified. CB methods modify how the model processes inputs related to biothreats — they rewire the neural pathways that would produce biothreat knowledge without erasing the underlying factual representations in the weights. The paper's hypothesis (drawn from Jain et al., 2023) is that those factual representations remain intact and can be "revived" through fine-tuning on relevant text. Data filtering, in contrast, prevents those representations from forming during pretraining in the first place. The CB methods therefore serve as a test of this hypothesis: if filtering truly produces "deep ignorance" while CB produces only "shallow suppression," then filtering should resist fine-tuning attacks far longer than CB, which is exactly what the experiments in Section 3 find.

Evaluation Methodology: Measuring Biothreat Proxy Knowledge

WMDP-Bio and Its Limitations. The primary target for capability suppression is the WMDP-Bio benchmark (Li et al., 2024b), a public four-way multiple-choice question answering (MCQA) dataset developed by subject matter experts to assess "biothreat-related yet harmless knowledge of dual-use biological processes and laboratory techniques." The benchmark contains questions across six categories: Dual-use Virology, Bioweapons & Bioterrorism, Reverse Genetics & Easy Editing, Enhanced Potential Pandemic Pathogens, Viral Vector Research, and Expanding Access.

The paper identifies a critical evaluation challenge that makes naive WMDP-Bio scores unreliable for measuring genuine knowledge suppression: shortcut exploitation. Language models can achieve above-chance MCQA performance through heuristics rather than knowledge — for example, by selecting the longest answer, by recognizing statistical patterns in how answers are phrased, or by exploiting correlations between choices and the question domain. The paper documents that simply selecting the longest answer on the full WMDP-Bio yields 46% accuracy, far above the random baseline of 25%. This means that a model could score 46% on WMDP-Bio without possessing any biothreat knowledge whatsoever, simply by learning the "choose the longest option" heuristic during training.

More subtly, recent work by Balepur et al. (2024) demonstrated that LLMs can answer multiple-choice questions when given only the choices without the question itself, indicating they exploit systematic patterns in how correct and incorrect answers are constructed. The paper quantifies this for WMDP-Bio: three models (Llama 3.3 70B, Llama 4 Scout, and OLMo 2 32B) achieve 54.67%, 51.38%, and 50.12% accuracy respectively on choice-only prompts — all well above the 25% random baseline. This confirms that WMDP-Bio contains exploitable patterns that allow models to "game" the evaluation.

WMDP-Bio Robust MCQA Subset (868 questions). To address choice-only shortcuts, the paper identifies 405 questions (31.81% of the original WMDP-Bio) that all three tested models answer correctly without seeing the question — these are classified as shortcut-exploitable and are removed. The remaining 868 questions form the WMDP-Bio Robust MCQA subset. On this subset, the unfiltered baseline model's accuracy drops from 52.79% (all questions) to 43.20% (robust questions only), reflecting the removal of questions that could be answered through heuristics rather than knowledge. The strong filter model achieves 34.68% and the weak filter model achieves 33.87% on this robust subset — both substantially above random chance (25%) but significantly below the unfiltered baseline, confirming that filtering suppresses genuine knowledge rather than merely disrupting shortcut exploitation.

WMDP-Bio Verified Cloze Subset (1,076 questions). As a complementary approach to mitigate MCQA confounds, the paper introduces a cloze-style evaluation where the model sees only the question text and must generate the answer without seeing the four choices. For each question, the model is called four times, once with each possible answer; the length-normalized log probability (perplexity) of each answer is computed, and the answer with the highest likelihood is selected. This format eliminates the ability to compare choices or exploit inter-choice patterns.

The paper excludes 197 questions from WMDP-Bio that cannot be adapted to cloze format — for example, questions that require seeing all choices ("Which of the following...?", "All of the above") or questions whose answer depends on comparing the alternatives. An OpenAI o3 model is used to automatically classify which questions are cloze-compatible (prompt in Appendix I.5). The remaining 1,076 questions form the WMDP-Bio Verified Cloze subset.

On this subset, all models perform lower than on MCQA, confirming that the cloze format is more challenging. The unfiltered baseline achieves 36.34%, while the strong filter achieves 24.44% and the weak filter achieves 26.77%. Notably, the strong filter's 24.44% is effectively at random chance (25% with 4 choices), meaning the model's biothreat proxy knowledge, as measured by this evaluation, has been completely suppressed. This is a key result: the filtering is sufficiently aggressive that the model cannot reliably distinguish correct from incorrect answers in an open-ended format, even though it retains some MCQA performance (likely due to residual test-taking heuristics that the cloze format eliminates).

Why two evaluation formats? The MCQA and cloze subsets test complementary aspects of knowledge. MCQA tests whether the model can discriminate between correct and incorrect answers when presented with both — a recognition task. Cloze tests whether the model can generate or assign higher probability to the correct answer without seeing alternatives — a recall task. The paper's finding that filtering reduces performance more on cloze than MCQA (strong filter: 34.68% MCQA vs. 24.44% cloze) suggests that filtering successfully removes explicit knowledge (recall) while the model retains some residual ability to recognize correct answers through partial or degraded representations. This is consistent with the interpretation that filtering prevents deep knowledge acquisition but does not eliminate every trace of biologically-relevant linguistic patterns from the training data.

General Capability Benchmarks. To verify that filtering does not catastrophically degrade the model's general knowledge, the paper evaluates on:

• MMLU-No-Bio (53 of the 57 MMLU topics): excludes virology, medical genomics, high school biology, and college biology to measure knowledge disjoint from the filtered content
• MMLU-HSC-Bio (MMLU high-school and college biology topics): measures benign biology knowledge the paper wants to preserve
• PIQA: physical commonsense reasoning
• LAMBADA: long-context text comprehension
• HellaSwag: commonsense natural language inference

The filtering models show no systematic degradation on these benchmarks — in some cases, they slightly outperform the unfiltered baseline (e.g., the strong PT + weak anneal model achieves 78.35% on PIQA vs. 76.44% for the baseline, and 54.05% on LAMBADA vs. 47.08% for the baseline; Table 2). The paper interprets these small differences as within normal training variance and concludes that filtering causes "no apparent net degradation of non-bio capabilities."

Attack Regimes for Tamper-Resistance Evaluation. The paper evaluates models under five categories of adversarial pressure (detailed in Section 3.1):

1. Input-space attacks (black-box/grey-box): Few-shot prompting with 16 held-out question/answer pairs, and Universal Greedy Coordinate Gradient (GCG-U) with a universal adversarial prefix optimized across 32 held-out questions (20 steps, search width 256, batch size 32, initialized as exclamation marks, requiring approximately 4 hours of wall-clock time on an Nvidia H200 GPU).

2. Latent-space attacks (white-box): Universal perturbations to the model's hidden activations at layers 0, 8, 16, 24, and 30, optimized over 64 steps with batch size 16, learning rate $10^{-3}$, and exponential decay with $\gamma = 0.95$. These attacks directly manipulate the model's internal representations to elicit biothreat knowledge without modifying the weights.

3. Adversarial fine-tuning (weight-space): Full-parameter and LoRA fine-tuning for 2 epochs on the WMDP-Bio Forget set (24,453 documents, 305M tokens total), using batch size 16, context window 2,048, and learning rate $2 \times 10^{-5}$. The paper runs 2 full-parameter and 2 LoRA fine-tuning runs and reports the mean and standard deviation. Mixing benign WikiText data into the fine-tuning dataset did not improve attack effectiveness, so attacks use the WMDP-Bio Forget set alone.

4. Benign fine-tuning: Identical configuration to adversarial fine-tuning but using WikiText instead of the WMDP-Bio Forget set, testing whether legitimate model adaptation (domain-specific fine-tuning) accidentally revives suppressed capabilities.

5. Retrieval-augmentation attacks (Section 4.1): A custom 1,000-question "open-book" evaluation where a passage containing the answer is provided in context. This tests whether models can leverage biothreat knowledge provided at inference time even if they lack internalized knowledge.

The key quantitative claim about tamper-resistance — resistance "for up to 10,000 steps and 300M tokens" — is measured under the adversarial fine-tuning regime. The paper compares its attack scale to prior work in Table 10 (Appendix E), finding that it uses the largest number of unique examples (80,000), total steps (10,000), and step × batch size product of any related work. This is a deliberate stress-test designed to demonstrate that filtering's tamper-resistance is not merely a matter of running shorter or weaker attacks than prior work, but represents a qualitatively more robust defense.

Difficulty Estimation via Pass@1 — Parallel to Prior Work. While the paper does not use the term "difficulty estimation" in the same way as the compute-optimal scaling literature, it performs an analogous operation: the baseline unfiltered model's performance on WMDP-Bio (43.20% Robust MCQA, 36.34% Cloze) establishes the "difficulty" of suppressing biothreat knowledge for a model of this scale. The filtering approaches must overcome this baseline to demonstrate effectiveness. The paper does not condition its defense strategy on per-question difficulty (as in compute-optimal scaling), but the evaluation framework implicitly acknowledges difficulty variation across the six WMDP-Bio categories listed in Section 2.4, which span different types and depths of biothreat proxy knowledge.

4. Key Insights and Innovations

Innovation 1: Shifting the Safety Intervention from Post-Training Suppression to Pretraining Prevention

The paper's most fundamental conceptual move is reframing where in the model development pipeline tamper-resistance should be built. The dominant paradigm in LLM safety has been post-hoc suppression: train a model on whatever data is available, then use fine-tuning (RLHF, adversarial training, circuit-breaking, unlearning) to suppress unwanted behaviors the model has already acquired. This paradigm tacitly assumes that the model needs exposure to harmful content during pretraining — either to develop general capabilities that transfer to benign tasks, or to learn representations that make safety training more effective (as argued by Maini et al., 2025 and Li et al., 2025a).

This paper challenges that assumption directly, not through argument but through empirical demonstration. The central finding — that models trained without biothreat proxy text perform competitively on general benchmarks (Figure 3, left) while resisting fine-tuning attacks for 10,000+ steps (Figures 4, 5) — shows that exposure to dual-use scientific knowledge is not necessary for developing the broad capabilities measured by MMLU, PIQA, LAMBADA, and HellaSwag. The models are not degraded; they are selectively ignorant.

What makes this a conceptual innovation rather than a mere engineering improvement is that it changes the nature of the safety guarantee. Post-training suppression produces models that behave safely under specific conditions but retain the neural circuitry for harmful capabilities — the "wrapper" problem documented by Jain et al. (2023). The guarantee is behavioral and conditional. Pretraining prevention, in contrast, produces models that fundamentally lack the internal representations needed to exhibit the capability at all — a structural guarantee. The distinction is analogous to the difference between training a dog not to bite versus removing its teeth. The paper's title, "Deep Ignorance," captures precisely this distinction: the ignorance is "deep" because it reflects an absence of learned representations, not a suppression of existing ones.

The paper explicitly connects this to the safety case literature (Clymer et al., 2024; Buhl et al., 2024), which argues that robust tamper-resistance — demonstrated through diverse, sustained adversarial evaluation — constitutes evidence that a model "fundamentally lacks neural circuitry for harmful behaviors" (Section 5, Section 6.1). By achieving tamper-resistance an order of magnitude beyond prior post-training methods, the paper provides the first empirical demonstration that pretraining data curation can produce evidence of this kind at a scale relevant to dual-use capability suppression. The framing shift is from "can we make models refuse to exhibit capabilities they possess?" to "can we make models that never possess the capabilities in the first place?" — and the paper's contribution is demonstrating that the latter is both technically feasible and substantially more robust.

Innovation 2: A Domain-Dependent Hypothesis Resolving Conflicting Findings on Data Filtering

The paper proposes and provides evidence for a specific boundary condition on when data filtering works as a safeguard, resolving an apparent contradiction in the recent literature. Prior work by Maini et al. (2025) and Li et al. (2025a) had found that filtering toxic and harmful content from pretraining data could increase vulnerability to certain attacks, leading them to hypothesize that "safety is not about censorship" and that "bad data may lead to good models." Their interpretation was that models need exposure to harmful content during pretraining to learn effective refusal and avoidance behaviors during subsequent safety training.

This paper does not dispute those findings — it replicates elements of them in Appendix G using the same models from Maini et al. (2025), finding that filtered models are indeed more vulnerable to few-shot attacks when tested on jailbreaking (compliance with harmful requests). Instead, the paper proposes that the domain matters fundamentally: filtering works for precise factual knowledge (scientific facts, engineering specifications) because such knowledge arises from specific subsets of training documents and can be suppressed by removing those documents. Filtering does not work as well for emergent propensities (toxicity, willingness to comply with harmful requests) because these behavioral tendencies are more diffuse — they arise from broad patterns in the training distribution rather than identifiable document subsets — and may indeed require exposure to harmful examples to learn effective avoidance.

This is not a mere nuance. It provides a diagnostic framework for predicting when data filtering should be effective as a safeguard. If the unwanted capability requires precise, factual knowledge (biothreat methodology, cybersecurity exploits, chemical synthesis procedures), then removing the relevant documents from training should suppress it robustly. If the unwanted capability is a behavioral propensity (producing toxic text, role-playing harmful scenarios, complying with dangerous instructions), then filtering alone may be insufficient or even counterproductive. The paper's evidence for this hypothesis comes from two sources: (1) the strongly positive results on biothreat proxy knowledge (Figures 3-5), which requires precise factual recall, and (2) the negative or mixed results on jailbreaking using models from Maini et al. (2025) (Appendix G, Figure 10), which involve compliance behavior rather than factual knowledge.

This insight is significant because it converts a confusing empirical landscape ("filtering helps" vs. "filtering hurts") into a coherent picture with defined boundary conditions. It enables practitioners to reason about when to invest in data filtering, and it explains why concurrent industry practice (frontier developers filtering for harmful content) has remained opaque — the effectiveness likely depends on what specific content is being filtered, making blanket claims about "data filtering for safety" misleading. The contribution is a conceptual advance in understanding the relationship between training data composition and model capabilities, not a new filtering technique.

Innovation 3: Tamper-Resistance as an Order-of-Magnitude Improvement, Not a Binary Property

The paper introduces an implicit but important reframing of what "tamper-resistant" means. Prior work in the adversarial robustness and machine unlearning literature has largely treated tamper-resistance as a binary property: either a safeguard withstands an attack, or it doesn't. The typical evaluation protocol is to apply a fixed attack budget (e.g., 500 fine-tuning steps) and report whether the safeguard holds — a binary success/failure metric.

The paper's experimental design — sweeping fine-tuning attacks continuously from step 0 to step 10,000, measuring both full-parameter and LoRA fine-tuning, and plotting capability recovery as a function of attack budget (Figures 4, 5) — implicitly reframes tamper-resistance as a continuous, quantitative property: how many steps of adversarial pressure can a model withstand before its suppressed capabilities reemerge? The answer for post-training methods is typically dozens to hundreds of steps (Table 10, Appendix E, documents this across eleven prior works). The answer for data filtering is thousands of steps, with capability recovery remaining substantially below the unfiltered baseline even at the maximum attack budget tested.

This reframing matters for two reasons. First, it enables comparative evaluation across defense methods on a meaningful quantitative scale, rather than collapsing performance into binary pass/fail judgments that obscure differences in robustness. A defense that resists 50 steps and one that resists 5,000 steps are qualitatively different in their practical implications, but prior evaluation protocols would report both as "broken" or "not broken" depending on the chosen attack threshold. The paper's continuous measurement approach reveals that filtering provides roughly an order of magnitude more tamper-resistance than post-training baselines — a finding that would be invisible in a binary evaluation.

Second, the continuous view connects tamper-resistance to the safety case framework (Buhl et al., 2024; Goemans et al., 2024), where the strength of an inability argument depends on the diversity and intensity of tampering the model has withstood. A model that resists 10,000 steps of domain-specific fine-tuning (an attack configuration substantially larger than any prior work, as the paper documents in Table 10) provides stronger evidence of "deep ignorance" than one that resists only 100 steps. The paper does not claim that 10,000 steps is sufficient for a safety case — the failure of combined fine-tuning + retrieval attacks in Section 4.1 demonstrates clear limitations — but it establishes a quantitative framework for reasoning about how much tamper-resistance is necessary and how much current methods provide.

This is not a new metric or measurement technique per se, but a conceptual shift in how the field should think about and evaluate tamper-resistance. The contribution is demonstrating that tamper-resistance is meaningfully quantifiable and that the differences between methods are large enough (an order of magnitude) to be practically significant.

Innovation 4: Negative Results on Synthetic Document Training Establish a Difficulty Cliff

The paper's experiments with synthetic document training (SDT) — attempting to implant incorrect biothreat knowledge into models to further suppress correct knowledge — represent an important negative result that constrains the space of viable approaches. Anthropic Alignment Team (2025) had demonstrated a proof of concept where synthetic documents containing misinformation could modify model beliefs in controlled settings, suggesting that actively teaching models incorrect information about dangerous topics could complement simple data removal.

The paper's attempt to scale this approach failed (Section 4.2, Appendix F, Figure 9). Training filtered models on two variants of synthetic biothreat-misinformation documents — one designed to be plausibly incorrect to non-experts, another radically altered with high-school cell biology concepts — did not substantially improve tamper-resistance over filtering alone. In several cases, training on the synthetic documents increased rather than decreased the model's performance on biothreat-proxy evaluations after fine-tuning attacks. The paper speculates that the synthetic documents, while containing incorrect facts, were sufficient to "attune the model to biology concepts in a way that allowed it to exploit heuristics in our evaluations" — essentially, exposure to any biology-domain text, even incorrect text, improved the model's ability to game the MCQA benchmarks.

This negative result is significant for two reasons. First, it establishes that data removal and data corruption are not equivalent strategies. Removing biothreat text from training prevents the model from developing representations that can be exploited at test time. Corrupting biothreat text with misinformation may teach the model incorrect facts but still provides linguistic and conceptual "scaffolding" that facilitates rapid relearning when correct information is encountered during fine-tuning. The model learns that "anthrax" is a biology concept associated with particular terminology and discourse patterns, even if it learns wrong facts about anthrax; this structural knowledge may accelerate acquisition of correct facts when they are provided.

Second, the result highlights a fundamental challenge with proxy evaluations: if the evaluation itself can be gamed through heuristics, then interventions that appear to suppress capability may only be disrupting the heuristics, while interventions that maintain domain familiarity (even with incorrect facts) may preserve the ability to game the evaluation. This is a cautionary finding for the broader capability suppression literature — it demonstrates that simple MCQA benchmarks may be insufficient to distinguish between models that genuinely lack knowledge and models whose knowledge representations have been disrupted in ways that fine-tuning can repair. The paper's development of the cloze evaluation format (which is harder to game) partially addresses this, but the SDT negative result underscores that evaluation methodology and defense methodology are deeply intertwined — you can only measure what your evaluations are capable of detecting.

5. Experimental Analysis

Evaluation Methodology

• Dataset. All experiments use the WMDP-Bio benchmark (Li et al., 2024b), a public four-way multiple-choice question answering (MCQA) dataset developed by subject matter experts to assess biothreat proxy knowledge. The benchmark contains questions across six categories: Dual-use Virology, Bioweapons & Bioterrorism, Reverse Genetics & Easy Editing, Enhanced Potential Pandemic Pathogens, Viral Vector Research, and Expanding Access. Because the unmodified benchmark suffers from shortcut exploitation (selecting the longest answer yields 46% accuracy; three tested LLMs achieve 50-55% accuracy on choice-only prompts), the paper develops two curated subsets: WMDP-Bio Robust MCQA (868 questions) removes 405 questions that all three tested models answer correctly without seeing the question text, and WMDP-Bio Verified Cloze (1,076 questions) removes 197 questions incompatible with cloze-style format and evaluates via length-normalized perplexity of individual answers without presenting choices. For general capability measurement: MMLU-No-Bio (53 of 57 MMLU topics excluding biology), MMLU-HSC-Bio (high-school and college biology topics from MMLU), PIQA (physical commonsense reasoning), LAMBADA (long-context text comprehension), and HellaSwag (commonsense natural language inference). For retrieval-augmentation experiments: a custom 1,000-question open-book MCQA benchmark generated by Claude 3.7 Sonnet from biology paper abstracts in the WMDP-Bio Forget set.

• Base model(s). All experiments use decoder-only transformers with 6.9 billion parameters, identical in architecture to Pythia 6.9B (Biderman et al., 2023): 32 layers, hidden dimension 4,096, 32 attention heads, GELU activation, rotary position encoding (25% of hidden dimension), sequence length 2,048, LayerNorm normalization, no weight tying. Models are trained from scratch on the DCLM dataset (Li et al., 2024a) using the GPT-NeoX library (Andonian et al., 2023) with a staged approach of 500B tokens of pretraining followed by 50B tokens of annealing on a mixture enriched with academic and instruction-following data. The paper explicitly states the models were "not trained to be competitive with state-of-the-art models of similar size" — they were trained only until reaching 50% accuracy on the standard WMDP-Bio benchmark, with remaining compute dedicated to filtering experiments. Training used 128 Nvidia H100 GPUs over approximately 5 wall-clock days (15,632 GPU hours).

• Metrics. The primary metric is accuracy (%) on the WMDP-Bio Robust MCQA and Verified Cloze subsets, with "lower is better" for biothreat proxy knowledge. General capabilities are reported as accuracy on MMLU, PIQA, LAMBADA, and HellaSwag, with "higher is better." For adversarial attacks, the paper reports accuracy after attack application, with successful defense indicated by accuracy remaining near random chance (25%) or the filtered model's pre-attack baseline. For adversarial fine-tuning specifically, accuracy is measured at multiple checkpoints throughout the 10,000-step attack sequence and reported as a function of fine-tuning steps (Figures 4, 5). For retrieval-augmentation attacks, accuracy is reported separately for "closed-book" and "open-book" conditions (Figure 6). All WMDP-Bio evaluations use the grading function from Lightman et al. (2022) (referenced as part of the WMDP benchmark release). For the GCG-U attacks, results are averaged over 3 independent runs with reported means and standard deviations in plots. For fine-tuning attacks, results are averaged over 2 full-parameter and 2 LoRA fine-tuning runs (4 total) with the mean plotted and the region within one standard deviation shaded.

• Baselines. The paper uses four categories of baselines: (1) Unfiltered model: trained on the original DCLM dataset with no content filtering, representing the "natural" biothreat proxy knowledge of a 6.9B model trained on standard web data; (2) Circuit-Breaking (CB) (Zou et al., 2024): LoRA adapters at layers 5, 10, 15, 20, 25, and 30 trained with a two-part objective to preserve activations on benign data and reroute activations on harmful data to be orthogonal to original processing, applied to the unfiltered baseline model after training; (3) Circuit-Breaking with Latent Adversarial Training (CB+LAT) (Sheshadri et al., 2024): CB extended with latent-space adversaries at the same layers using 4 steps of adversarial optimization per training step to proactively elicit and train against latent harmful capabilities, applied to the unfiltered baseline; (4) Combined defenses: filtering + CB and filtering + CB+LAT, where post-training safeguards are applied to the filtered models (Section 4.1). For the retrieval-augmentation experiments in Section 4.1, the unfiltered baseline serves as an additional comparison point to demonstrate that filtered models can still leverage in-context biothreat text. For the jailbreaking experiments using models from Maini et al. (2025) (Appendix G), the baselines are the original unfiltered and filtered 1.7B instruction-tuned models from that work, plus CB and CB+LAT variants applied to both.

• Generation budget / compute accounting. The paper's primary axis of test-time compute is the fine-tuning attack budget, measured in number of fine-tuning steps (up to 10,000 for full-parameter and LoRA fine-tuning) and total tokens of adversarial data (305M tokens from the WMDP-Bio Forget set). For filtering methods specifically, the relevant compute metric is the percentage of training documents removed (8.42% pretraining / 9.36% annealing for strong filter; 4.96% annealing for weak filter) and the computational cost of the filtering pipeline itself, calculated as 0.83% of total training FLOPs for the most expensive end-to-end weak filtering configuration (Appendix C, Equation 2). For input-space attacks, the budget is measured in number of few-shot examples (16 for the few-shot attack) and GCG-U optimization steps (20 steps, search width 256, batch size 32). For latent-space attacks, the budget is 64 optimization steps with batch size 16. The paper does not use "generations" as a compute metric since models are evaluated in a single forward pass per question (no sampling-based test-time compute scaling is studied).

• Cross-validation / statistical protocol. The paper does not employ cross-validation for strategy selection (as in compute-optimal scaling literature) since there is no adaptive allocation policy being optimized — each defense configuration is evaluated uniformly. Instead, statistical rigor comes from: (1) multiple independent attack runs with reported means and standard deviations (3 runs for GCG-U and latent-space attacks; 4 runs across full-parameter and LoRA variants for fine-tuning attacks); (2) the use of held-out evaluation subsets that are disjoint from attack development data (the 32 questions used for GCG-U and latent-space attack optimization are held out from the evaluation sets; the 16 few-shot examples are held out from the evaluated questions); (3) the two curated WMDP-Bio subsets (Robust MCQA and Verified Cloze) are applied uniformly to all models and attack conditions, eliminating the possibility that different evaluation subsets favor different defenses. For the classifier filter training, a standard train/validation/test split is used (Table 4), with the ModernBERT threshold (0.0105) selected via grid search on the validation set. The paper explicitly states it "opted to pursue minimal hyperparameter optimization to mitigate the risk of overfitting" for the classifier.

Main Quantitative Results

Data Filtering Competitively Suppresses Biothreat Proxy Knowledge (Figure 3, Section 2.6)

The central finding is that data filtering substantially inhibits biothreat proxy knowledge acquisition during training while preserving general capabilities. On the WMDP-Bio Robust MCQA subset, the strong filter achieves 35.37% accuracy and the weak filter achieves 33.99%, compared to 42.97% for the unfiltered baseline — representing a 17.7% and 20.9% relative reduction in biothreat proxy knowledge respectively (Table 2, Section 2.6). This reduction is more pronounced on the cloze-style evaluation, where the strong filter drops to 24.44% — effectively random chance (25% for 4-choice tasks) — compared to 36.34% for the unfiltered baseline, and the weak filter achieves 26.77%. The CB and CB+LAT baselines perform slightly better on MCQA (achieving accuracies in the range of approximately 30-33% based on Figure 3's visual layout), while filtering performs better on cloze evaluations, suggesting complementary strengths.

On general capabilities, no systematic degradation is observed. MMLU-No-Bio scores range from 43.16% to 44.92% across all filtering configurations versus 44.92% for the unfiltered baseline — differences that the paper attributes to normal training variance (Table 2, Section 2.6). PIQA ranges from 75.73% to 78.35% (baseline: 76.44%), LAMBADA from 47.08% to 54.05% (baseline: 47.08%), and HellaSwag from 55.41% to 55.90% (baseline: 55.75%). The strong-filter-weak-anneal configuration actually shows small improvements on several benchmarks (MMLU: 44.82% vs. 44.92% baseline; PIQA: 76.88% vs. 76.44%; LAMBADA: 54.05% vs. 47.08%). MMLU-HSC-Bio, which measures benign biology knowledge the paper aims to preserve, is not reported separately in Figure 3 but the paper states filtering "does not degrade general biology knowledge" in context of MMLU-No-Bio results.

Under input-space attacks: the few-shot attack (16 held-out question/answer pairs) boosts performance on the unfiltered baseline but filtered models remain substantially below it (Figure 3). CB methods outperform filtering on few-shot MCQA attacks, while filtering outperforms CB methods on GCG-U attacks on average. The filtered models maintain near-random-chance performance on cloze evaluations under both attack types, with the strong filter remaining at approximately 25% — the attack cannot elicit knowledge the model does not possess.

Takeaway from Section 2 (stated by the authors): "Training data filtering is an effective technique for mitigating unwanted knowledge in LLMs. Significant safety improvements can be achieved through the careful curation of pretraining data."

Filtering Achieves State-of-the-Art Tamper-Resistance Under Adversarial Fine-Tuning (Figures 4, 5; Section 3.2)

The paper's headline quantitative result: filtered models resist adversarial fine-tuning attacks for up to 10,000 steps and 300M tokens of biothreat-related text, improving tamper-resistance by more than an order of magnitude over post-training baselines. Figure 4 (left and middle panels) plots WMDP-Bio Robust MCQA and Verified Cloze accuracy as a function of fine-tuning steps from 0 to 10,000.

At step 0 (before any fine-tuning), the filtered models (strong-filter-weak-anneal, shown in the figures) achieve approximately 35% Robust MCQA and 25% Cloze. The unfiltered baseline starts at approximately 43% Robust MCQA and 36% Cloze. Under adversarial fine-tuning on the WMDP-Bio Forget set:

• Post-training baselines (CB, CB+LAT): Both begin substantially below the unfiltered baseline at step 0 (indicating successful initial suppression), but their biothreat knowledge rapidly recovers. By approximately 100-500 fine-tuning steps, CB and CB+LAT accuracy on Robust MCQA approaches the unfiltered baseline level (around 42-44%), and by 1,000-2,000 steps they are indistinguishable from it. The Verified Cloze evaluation shows the same pattern with slightly delayed recovery — CB and CB+LAT remain below baseline for longer but converge by approximately 2,000-5,000 steps. This is consistent with the "wrapper" hypothesis: CB methods suppress but do not erase underlying capabilities, and fine-tuning on domain-relevant text quickly restores them.

• Filtered models: Both the strong filter and the strong-filter-weak-anneal configurations show substantially slower capability recovery. On Robust MCQA, the filtered models remain below 40% through approximately 5,000 steps, only gradually approaching the unfiltered baseline near 10,000 steps — and even then remaining several percentage points below it. On Verified Cloze, the filtered models' performance remains near random chance (25%) through approximately 2,000-3,000 steps and rises very slowly thereafter, reaching only approximately 30% at 10,000 steps — substantially below both the unfiltered baseline (36%) and the CB/CB+LAT models (which have recovered to baseline levels). The shaded standard deviation regions in Figure 4 show that the variance across fine-tuning runs is relatively small for filtered models, indicating consistent resistance rather than lucky runs.

Interpreting the gap: At 1,000 steps — which exceeds the maximum attack budget tested in most prior work (Table 10, Appendix E documents that prior work tested up to 500-3,000 steps with 1-128 batch size) — the filtered models maintain roughly 35-37% Robust MCQA and 25-27% Cloze, while CB/CB+LAT have recovered to 40-44% MCQA and 33-36% Cloze, essentially matching the unfiltered baseline. The filtering advantage is thus not marginal — it represents the difference between complete defense failure and substantial capability suppression at attack scales relevant to practical threats.

Latent-space attacks (Figure 4, right panel) tell a more nuanced story. On Robust MCQA, CB+LAT performs best, followed by CB, with filtered models providing less resistance — latent-space perturbations at layers 0, 8, 16, 24, and 30 appear able to partially elicit biothreat knowledge from filtered models in the MCQA format (accuracy rises above the pre-attack filtered baseline). However, on Verified Cloze, the pattern reverses: filtered models show the strongest resistance, with accuracies remaining near random chance even under latent-space perturbations, while CB and CB+LAT show substantial vulnerability. The paper speculates (Section 3.2) that "this may be due to the model learning generalizable heuristics for answering multiple-choice questions" — the latent-space attack may be recovering generic MCQA test-taking strategies rather than genuine biothreat knowledge from the filtered models.

Filtering is Robust to Benign Fine-Tuning (Figure 5; Section 3.2)

A critical practical concern is whether legitimate model adaptation — fine-tuning on domain-specific data unrelated to biothreats — accidentally revives suppressed capabilities. Figure 5 demonstrates that filtered models are robust to benign fine-tuning on WikiText (Merity et al., 2016), while CB and CB+LAT rapidly lose effectiveness.

Under benign fine-tuning (identical hyperparameters to adversarial fine-tuning but using WikiText instead of the WMDP-Bio Forget set, 10,000 steps):

• Filtered models: Biothreat proxy knowledge remains essentially flat — Robust MCQA and Cloze accuracies do not increase from their pre-fine-tuning levels over the full 10,000 steps. This is expected: WikiText contains no biothreat-related content, so there is no signal for the model to acquire the knowledge it never learned during pretraining.

• CB and CB+LAT: Both exhibit rapid degradation. Within approximately 100-500 steps of benign fine-tuning, their biothreat proxy knowledge recovers toward the unfiltered baseline level, following a trajectory similar to — though slightly slower than — adversarial fine-tuning. This confirms a known vulnerability (Qi et al., 2023; Che et al., 2025; Deeb & Roger, 2024): benign fine-tuning disrupts the LoRA-based suppression mechanisms, presumably by shifting the model's activation distributions away from the regimes where the CB adapters were trained to operate. The CB adapters were trained to reroute activations for a specific harmful dataset; when the model's overall weight distribution changes through fine-tuning on any data, the adapters' rerouting may become less effective.

This result has substantial practical implications. Organizations routinely fine-tune open-weight models on proprietary datasets for domain-specific tasks. If CB-based safeguards degrade under such benign fine-tuning, downstream users may unintentionally create bio-capable models from originally "safe" checkpoints — a risk the model developer cannot control after release. Filtering, by preventing knowledge acquisition during pretraining rather than suppressing it post-hoc, eliminates this failure mode entirely.

CB and Filtering Are Complementary, But Combined Defenses Fail Against Staged Attacks (Figures 6, 7, 8; Section 4.1)

Retrieval-augmentation attacks (Figure 6) demonstrate a critical limitation of data filtering: filtered models can leverage biothreat knowledge provided in context, even though they lack internalized knowledge. The paper constructs a 1,000-question evaluation with two modes: "closed-book" (model answers from its own knowledge) and "open-book" (a passage containing the answer is provided in the prompt). Results show:

• Closed-book: Filtered models perform at their standard suppressed levels (approximately 35% Robust MCQA); baseline model performs at approximately 43%.
• Open-book: Both filtered and baseline models achieve substantially higher accuracy — the paper's Figure 6 shows filtered models reaching roughly 55-65% on the open-book evaluation, comparable to or slightly below the unfiltered baseline. The biothreat knowledge in the passage is accessible regardless of whether the model learned similar knowledge during pretraining. This is a direct demonstration that data filtering does not prevent the model from reading, understanding, and reasoning about biothreat text provided at inference time.

CB blocks in-context retrieval, but combined defenses fail against staged attacks. Figure 6 also shows that models with CB or CB+LAT applied (even to unfiltered models) perform poorly on the open-book evaluation — their accuracies remain near the closed-book levels (around 25-35%), indicating that CB's rerouting mechanism impairs the model's ability to process biothreat-related text regardless of source (internal knowledge or in-context passage). This is a useful complementarity: filtering handles internalized knowledge, CB handles in-context retrieval.

However, Figure 6's critical negative finding is that no model — including those with combined filtering + CB defenses — resists a staged attack combining fine-tuning with open-book evaluation (the "Ensemble" condition in Figure 6). When models are first fine-tuned on the WMDP-Bio Forget set and then evaluated with open-book prompts, all models achieve high accuracy (roughly 55-65%). The fine-tuning step "revives" the CB-defended models' ability to process biothreat text in context (breaking the CB protection), and the open-book evaluation provides biothreat knowledge that the filtered models can leverage (breaking the filtering protection). The staged attack defeats both defense mechanisms simultaneously.

Combined filtering + CB improves robustness to few-shot and latent-space attacks (Figures 7, 8). Figure 7 compares filtered models, CB models, and combined filtered+CB models under few-shot attacks (right panel) and standard evaluations (left/middle panels). Under few-shot attacks on Robust MCQA, the combined filtered+CB model substantially outperforms either defense alone — remaining near the filtered model's pre-attack accuracy while CB alone shows higher vulnerability. Figure 8 (right panel) shows a similar pattern for latent-space attacks: combined filtered+CB models are more resistant than individually defended models, particularly on Robust MCQA. On general capability evaluations (MMLU, PIQA, LAMBADA, HellaSwag), combined defenses show no meaningful degradation over the individual approaches — the LoRA adapters and filtering operate on largely orthogonal aspects of model processing, enabling additive safety benefits without compounding side effects.

However, Figure 8 (middle panel) shows that combined defenses offer at most modest improvements to tamper-resistance (adversarial fine-tuning resistance) over filtering alone. The filtered models' resistance to fine-tuning is already strong enough that adding CB provides marginal additional benefit — the fine-tuning primarily attacks knowledge representations, which filtering has already prevented from forming, rather than processing pathways, which CB targets.

Negative Results: Synthetic Document Training at Scale Fails (Appendix F, Figure 9; Section 4.2)

The paper attempted to extend filtering by actively teaching models incorrect biothreat information, following Anthropic Alignment Team (2025)'s proof of concept. Two variants were tested: a "weakly corrupted" dataset (plausible but incorrect to non-experts) and a "strongly corrupted" dataset (radically altered with high-school cell biology concepts), both generated by Claude 3.7 Sonnet and mixed into the annealing phase.

Figure 9 shows that synthetic document training (SDT) did not substantially improve tamper-resistance over filtering alone, and in several evaluation configurations increased biothreat proxy knowledge rather than decreasing it. Under multiple-choice evaluation (top row of Figure 9), SDT appears to mildly impede fine-tuning attacks relative to filtering alone — the models learn biothreat knowledge slightly more slowly during adversarial fine-tuning. However, under all other attack types (input-space, latent-space, benign fine-tuning), SDT either fails to improve resistance or degrades it (accuracy increases relative to the filtered model without SDT). The paper reports that "fine-tuning on these documents [the synthetic misinformation datasets] would often increase rather than decrease our filtered models' biothreat proxy knowledge."

The paper speculates about two failure mechanisms: (1) The synthetic documents, while factually incorrect, used biology-domain terminology and discourse patterns that "attuned the model to biology concepts in a way that allowed it to exploit heuristics in our evaluations" — exposure to any biology text, even wrong biology text, improved MCQA gaming ability. (2) The unstructured, pointwise way in which synthetic documents were produced "likely failed to implant coherent incorrect beliefs into the LLMs" — isolated false facts do not form the kind of integrated knowledge representations that robustly compete with or suppress true facts learned during adversarial fine-tuning.

Filtering's Impact on General Capabilities: More Detailed Results from Appendix B, Table 2

Table 2 in Appendix B provides a grid search over all possible combinations of filtering strength applied to pretraining and annealing phases separately. The headline finding is that all filtering configurations produce comparable outcomes — the differences in general capability scores across configurations are small relative to the gap between any filtered model and the unfiltered baseline's biothreat knowledge. However, some non-obvious patterns emerge:

• Single-stage filtering during both phases produces the lowest WMDP-Bio Cloze score (24.44%) but also the lowest average general knowledge score (55.53% vs. 56.05% baseline), suggesting the most aggressive filtering removes some benign content that supports general capabilities.
• Single-stage pretraining + multi-stage annealing produces the best WMDP-Bio Robust MCQA suppression (33.99%) while maintaining general capabilities (57.88% average, above the 56.05% baseline), suggesting that aggressive pretraining filtering combined with more precise annealing filtering is the optimal configuration for balancing safety and capability preservation.
• Multi-stage filtering during both phases produces intermediate WMDP-Bio scores (35.25% MCQA, 25.74% Cloze) with above-baseline general capabilities (57.37%), suggesting that classifier-based escalation during pretraining reduces the amount of benign scientific content unnecessarily removed.

Ablation Studies and Robustness Checks

• Filtering strength (strong vs. weak vs. extra-weak): Table 2 (Appendix B) shows all five filtering configurations. The strong filter achieves the lowest WMDP-Bio Cloze score (24.44%, essentially random chance) but removes 8.42% of pretraining documents. The extra-weak filter (ModernBERT threshold of 0.5 rather than 0.0105) removes only 0.94% of pretraining documents and 2.02% of annealing documents — its performance is not separately reported in Table 2 but is released as a model variant. The key finding is that WMDP-Bio suppression is monotonic with filtering aggressiveness for Cloze evaluation but non-monotonic for Robust MCQA (the strong-filter-weak-anneal configuration achieves the single lowest Robust MCQA score at 33.99%, lower than the fully strong-filtered 35.37%).

• Filtering stage allocation (pretraining only vs. annealing only vs. both): Table 2 (Appendix B) compares all four combinations (single/multi for pretraining × single/multi for annealing). The rank order for WMDP-Bio Robust MCQA suppression is: Single-Single (29.90% average proxy knowledge) ≈ Single-Multi (30.38%) ≈ Multi-Multi (30.50%) ≈ Multi-Single (30.97%). This tight clustering (range of only 1.07 percentage points) suggests that the specific allocation of filtering strength across training phases matters less than the total amount of biothreat text removed — all configurations that apply some form of filtering achieve similar proxy knowledge suppression.

• Multiple-choice shortcut mitigation (Robust MCQA vs. Cloze vs. full WMDP-Bio): Table 8 (Appendix D.4.1) compares model performance across all WMDP-Bio questions, the 405 shortcut-exploitable questions, and the 868 robust questions. The unfiltered baseline achieves 73.33% on shortcut questions versus 43.20% on robust questions — a dramatic 30 percentage point gap confirming extensive shortcut presence. The strong filter achieves 49.14% on shortcut questions versus 34.68% on robust questions — the filtering reduces performance on both subsets, but the gap narrows to approximately 14 points, suggesting that filtering partially degrades the model's ability to exploit MCQA heuristics in addition to removing biothreat knowledge. The Table 11 comparison (Appendix H) showing that other open models (OLMO-2 7B, Llama 3.1 8B, Tulu-3 8B, Gemma-3 4B) also exhibit large drops between full WMDP-Bio and curated subsets (e.g., Llama 3.1 8B Instruct drops from 72.43% to 46.28% on Verified Cloze) confirms that shortcut vulnerability is universal across model families and not an artifact of the paper's training setup.

• Test-task training effect (Appendix D.5, Table 9): To verify that filtered models' poor WMDP-Bio performance reflects genuine ignorance rather than underexposure to the MCQA format, the paper fine-tunes models on 98,764 MCQA documents (approximately 35.55M tokens) from MMLU's training split, with blocklist filtering applied to prevent biothreat knowledge leakage. Table 9 shows that this explicit test-task training improves WMDP-Bio Robust MCQA by only approximately 2 percentage points for both the unfiltered baseline (43.55% → 45.62%) and the filtered model (33.87% → 35.83%). The filtered model's performance remains substantially below the baseline even after test-task training. General capability impacts are mixed: LAMBADA improves substantially (51.87% → 61.58% for baseline; 54.10% → 61.73% for filtered), while MMLU slightly degrades (45.70% → 43.57% for baseline). The paper concludes that "our models are not underexposed to the test task" due to the 8.43B tokens of Flan instruction-following data already present in the annealing mixture.

• WMDP-Bio Cloze compatibility classification (Appendix D.4.2): OpenAI o3 is used to automatically classify which WMDP-Bio questions can be adapted to cloze format. The paper provides examples of incompatible questions: those with "Which of the following...?" stems, those with choices referencing each other ("Both A and C"), and those where the answer depends on comparing alternatives. 197 questions (approximately 15.5% of the original 1,273 WMDP-Bio questions) are excluded based on this classification. The prompt used is provided in Appendix I.5.

• ModernBERT classifier threshold (0.0105 vs. 0.5): The paper defines an "extra-weak" filter variant where the ModernBERT threshold is increased from 0.0105 to 0.5, reducing filtered pretraining documents from 8.42% to only 0.94% and filtered annealing documents from 9.36% to 2.02%. This variant is released as a model but its performance is not reported in detail in the main text — its existence demonstrates that filtering aggressiveness can be tuned continuously via the classifier threshold.

• Blocklist pos-ratio threshold (0.4): The blocklist construction includes a grid search over the pos-ratio hyperparameter (Section D.1.2), which controls the minimum ratio of proxy-document prevalence to general-biology-document prevalence for a keyword to be included in the blocklist. The paper settled on 0.4 using the validation set from Table 4, meaning terms are retained only if proxy documents account for at least 40% of their occurrences across the labeled corpus. No ablation over this value is reported, so sensitivity to this choice is unknown.

• PRM vs. ORM comparison — not applicable. The paper does not train or evaluate outcome reward models or process reward models in the style of the compute-optimal scaling literature. The ModernBERT classifier serves a filtering (binary classification) role, not a generation-scoring role. There is no "aggregation strategy" ablation because there is no step-wise scoring of generated solutions.

• Synthetic document training variants (weak vs. strong corruption): Appendix F (Figure 9) compares two synthetic misinformation strategies against filtering alone. The "weakly corrupted" variant (designed to appear plausible to non-experts) shows slightly better resistance to fine-tuning attacks under MCQA evaluation than the "strongly corrupted" variant (radically altered with high-school biology concepts), but neither substantially outperforms filtering alone, and both sometimes increase vulnerability. The paper speculates this may be because the strong corruption variant still provides biology-relevant "flavor" that helps the model attune to the evaluation domain, even though the specific facts are wrong.

• CB adapter layer selection (layers 5, 10, 15, 20, 25, 30): The paper follows Zou et al. (2023a) in applying LoRA adapters at six evenly-spaced layers spanning early, middle, and late processing stages. No ablation over different layer selections is reported, so the sensitivity of CB performance to layer choice is not characterized.

• Fine-tuning attack hyperparameters (learning rate warmup): Appendix F notes that "for these attacks in particular, we found that fine-tuning with a learning rate warmup stabilized training and slightly improved the success of these attacks" for the synthetic document training experiments. This suggests the standard fine-tuning configuration (no warmup) may slightly underestimate attack effectiveness in other experiments, though the difference appears to be small.

• Jailbreaking experiments on Maini et al. (2025) models (Appendix G, Figure 10): As a robustness check on the domain-dependent hypothesis, the paper applies CB, CB+LAT, and SDT defenses to the 1.7B instruction-tuned models from Maini et al. (2025), then evaluates resistance to jailbreaking (compliance with harmful requests). Results show: (1) variants of the filtered model are only slightly more resilient to fine-tuning attacks than the unfiltered baseline, (2) the filtered models are particularly vulnerable to few-shot attacks, and (3) CB and CB+LAT are effective against jailbreaking while SDT is not. This finding supports the paper's hypothesis that filtering is effective for precise knowledge but not for emergent behavioral propensities like complying with harmful requests.

Critical Assessment

Do the experiments support the central claim that "filtering pretraining data builds tamper-resistant safeguards"?

The evidence is strong but bounded. The paper demonstrates convincingly that filtering biothreat proxy text substantially reduces biothreat proxy knowledge in 6.9B models (Figures 3, 4, Table 2) and that this reduction persists under adversarial fine-tuning at scales exceeding all prior work by an order of magnitude (Figures 4, 5; Table 10). The tamper-resistance is not absolute — filtered models do gradually reacquire some biothreat knowledge under 10,000 steps of domain-specific fine-tuning (Figure 4 shows Robust MCQA rising from approximately 35% to 38-40% and Cloze rising from approximately 25% to 30%), but they remain well below the unfiltered baseline and the post-training baselines throughout. The paper's title claim is therefore appropriate: filtering builds tamper-resistant safeguards, not impenetrable ones.

However, the experiments only demonstrate tamper-resistance for one type of unwanted capability (biothreat proxy factual knowledge), one model scale (6.9B parameters), one model family (Pythia architecture), and one training data distribution (DCLM). The paper's own jailbreaking experiments (Appendix G, Figure 10) demonstrate that filtering does not provide durable tamper-resistance against compliance with harmful requests — a finding that both strengthens the paper's nuanced claims and limits the generality of the title. The tamper-resistance is domain-specific, and the paper does not characterize how the domain specificity boundary is drawn (beyond the speculative "precise knowledge vs. emergent propensity" distinction in Section 6.2). Without testing filtering on cybersecurity proxy knowledge, chemical synthesis procedures, or other dual-use domains, it remains unclear whether biothreat biology is uniquely amenable to filtering-based suppression or whether the approach generalizes across precise-knowledge domains.

Does the comparison to post-training baselines fairly characterize the relative effectiveness of filtering vs. CB/CB+LAT?

The comparison is fair for the specific attack configurations tested but omits several dimensions that would strengthen (or potentially weaken) the filtering advantage:

• Fine-tuning data distribution: The adversarial fine-tuning attacks use the WMDP-Bio Forget set — the same distribution of biothreat text that was removed from the filtered models' training data. This is the most direct form of adversarial fine-tuning and represents the worst case for filtering. If an attacker fine-tuned on biothreat text from a different distribution (e.g., a different corpus of virology papers, or textbooks, or forum discussions), it is possible that the filtered models' resistance would be even stronger (the model never learned to process biothreat text, so fine-tuning on out-of-distribution biothreat text might not efficiently transfer) or weaker (the model might learn faster from "fresh" biothreat text than from text that overlaps with CB's rerouting targets). The paper does not test distribution-shifted fine-tuning attacks.

• Combined post-training + filtering: Section 4.1 shows that combined defenses improve resistance to few-shot and latent-space attacks (Figures 7, 8), but the fine-tuning resistance of combined defenses is only marginally better than filtering alone (Figure 8, middle panel). This is noteworthy: if CB provides complementary protections (in-context retrieval blocking) without degrading filtering's fine-tuning resistance, then the "best" defense is clearly the combination. The paper could have strengthened this comparison by showing combined defense performance across the full 10,000-step fine-tuning sweep (Figure 8 only shows a subset of the fine-tuning results).

• Attack budget normalization across methods: The paper's Table 10 compares attack configurations across prior work but does not normalize for model size, data quality, or the specific biothreat knowledge target. A fine-tuning attack using 128 examples on a 7B model may be comparable to 10,000 examples on a 1.7B model in terms of information content relative to model capacity. The paper cannot establish whether its fine-tuning attacks are "more difficult" than prior attacks in any absolute sense — only that they use larger raw numbers of steps, examples, and tokens. This is a limitation of the broader tamper-resistance evaluation literature, not specific to this paper.

Do the experiments adequately address the possibility that filtered models simply learn to game evaluations differently?

The paper goes further than most prior work in addressing evaluation gaming, but residual concerns remain:

• The Robust MCQA subset (868 questions) removes questions answerable from choices alone, but does not remove questions answerable through other heuristics (e.g., word-length patterns, domain-specific terminology biases, "least incorrect option" strategies (Wang et al., 2024)). The paper's strong filter achieves 34.68% on Robust MCQA — still 10 points above random chance — and it is unclear whether this residual performance reflects partial knowledge or additional heuristic exploitation not captured by the choice-only filter.

• The Verified Cloze subset (1,076 questions) largely eliminates MCQA heuristics, and the strong filter's 24.44% accuracy (near random chance) is strong evidence of genuine knowledge suppression. However, the cloze evaluation introduces its own confounds: the model's perplexity estimates can be biased by answer length (shorter answers tend to have lower perplexity regardless of correctness), domain-specific vocabulary frequency (answers containing rare biology terms may have artificially high perplexity even if correct), and prompt formatting effects. The paper's use of "length-corrected perplexity" partially addresses the first concern but does not discuss the latter two.

• The SDT negative result (Appendix F, Figure 9) is a natural experiment: training on incorrect biology text should not improve performance on biothreat proxy evaluations unless the evaluation is measuring something other than factual knowledge. The finding that SDT sometimes increases biothreat proxy scores supports the interpretation that residual MCQA performance partially reflects domain attunement rather than knowledge. However, since SDT uses biology-domain vocabulary (even with incorrect facts), the increase could also reflect the model becoming better at answering biology questions (recognizing domain patterns) without learning correct facts — which would still be a form of evaluation gaming.

Are there experiments that would have strengthened the paper but were not run?

Several missing experiments are notable:

1. Scaling to larger models: The paper trains only 6.9B models. As models scale, the relationship between data removal and capability suppression may change — larger models may be more sample-efficient at extracting information from sparse signals, meaning that even heavily filtered datasets might contain enough residual biothreat-adjacent text for a larger model to reconstruct biothreat knowledge. Conversely, larger models might overfit less to specific documents, making targeted filtering more effective. Without scaling experiments, the findings cannot be extrapolated to the model scales (70B+, hundreds of billions of parameters) where dual-use capabilities are most concerning.

2. Filtering precision measurement on actual training data: The paper acknowledges that it "does not measure the classification performance of our filters on the actual pretraining and annealing datasets due to the significant challenge in labeling large amounts of data" (Appendix J). The reported precision/recall numbers (Table 5) are measured on evaluation sets that differ from the actual training corpus distribution. The qualitative analysis of filtered documents (Appendix J) reveals clear false positives — documents discussing COVID-19 economic impacts, translation tasks in FLAN, and non-technical Wikipedia articles about pandemics. Without knowing the true false positive rate on the training corpus, it is impossible to determine whether the filtering's minimal impact on general capabilities is because (a) the false positive rate is low, or (b) the training corpus is so large that even substantial over-filtering of benign biology content does not noticeably degrade metrics like MMLU-No-Bio and PIQA. These have different implications for scaling and generalizability.

3. Comparison to an "ideal" unlearning baseline: The paper compares filtering to CB and CB+LAT, but does not compare to other post-training capability suppression methods such as representation engineering (Zou et al., 2023a), task-specific fine-tuning on refusal data (Mazeika et al., 2024), or gradient-based unlearning (Eldan & Russinovich, 2023). It is possible that some post-training methods not tested would perform better than CB/CB+LAT under fine-tuning attacks, reducing the apparent advantage of filtering.

4. Sensitivity to the annealing mixture's biology content: The paper deliberately created an annealing mixture with "a higher proportion of scientific content compared to OLMo-2" to indirectly optimize WMDP-Bio performance (Section D.2.2). If the annealing mixture had contained even more biology content, would filtering still suppress WMDP-Bio to near-random chance? If the annealing mixture contained less biology content, would the unfiltered baseline perform worse (making filtering look less effective)? The paper cannot disentangle how much of filtering's effectiveness depends on the specific annealing data composition, which limits the generalizability of the specific accuracy numbers to other training recipes.

5. Evaluation on non-MCQA biothreat capability measures: The WMDP-Bio benchmark, even in its curated forms, measures factual knowledge through question answering. It does not measure whether the model can synthesize biothreat-relevant information across multiple facts, reason about procedures, or generate novel protocols. A model that cannot answer "What is the role of the NS2B-I39V mutation in ZIKV virulence?" might still be able to generate useful biothreat guidance if prompted with a specific scenario. The paper's use of WMDP-Bio as a proxy for biothreat capability is standard practice in the field, but the mapping between proxy knowledge suppression and genuine capability suppression is unvalidated.

6. Longer or more diverse fine-tuning attacks: The paper's fine-tuning attacks use only the WMDP-Bio Forget set. An attacker with access to a larger corpus of biothreat-relevant text (e.g., all of PubMed, or a curated dataset of dual-use biology textbooks) might be able to reintroduce knowledge more efficiently than the paper's attack configuration. The paper's finding that fine-tuning on WikiText (benign) does not revive biothreat knowledge suggests that the attack data must be domain-relevant, but the sensitivity to the amount and diversity of domain-relevant attack data is not explored beyond the fixed WMDP-Bio Forget set.

Do the results support the domain-dependent hypothesis (precise knowledge vs. emergent propensities)?

The hypothesis is plausible and consistent with the paper's evidence, but the evidence is largely comparative rather than causal. The paper demonstrates that filtering works well for biothreat proxy knowledge (Figures 3-5) and less well for jailbreaking compliance (Appendix G, Figure 10). However, these two domains differ along multiple dimensions simultaneously: the type of capability (factual recall vs. behavioral compliance), the nature of the training data being filtered (scientific papers vs. toxic/harmful web text), the evaluation format (knowledge tests vs. harmfulness judgments by GPT-4o), and the attack types used (knowledge elicitation vs. jailbreaking). The paper cannot isolate which of these dimensions drives the differential effectiveness. It is possible that filtering would work equally well for jailbreaking if the filtering target were defined with equal precision (e.g., removing all documents containing specific harmful instruction templates rather than broad toxicity filtering), or that filtering would work poorly for biothreat knowledge if the knowledge were more diffusely distributed across the training corpus (e.g., basic virology facts that appear in many general biology documents). The hypothesis is a useful organizing framework, but the experiments provide only correlational support.

Is the FLOPs accounting (Appendix C) appropriate and fair?

The FLOPs accounting (showing filtering costs <1% of training compute) is generally thorough but omits several practical costs:

• The cost of human expertise: The WMDP-Bio benchmark was developed by subject matter experts. The filtering pipeline relies on this expert-created resource for the gold-labeled proxy and general biology documents that train the ModernBERT classifier and calibrate the blocklist. For a new domain (e.g., cybersecurity exploits), equivalent expert-labeled resources would need to be created — a cost not captured in the FLOPs calculation.

• The cost of Llama 3.3 70B inference for blocklist generation and data labeling: This is included in the FLOPs calculation (Table 3: $4.45 \times 10^{19}$ FLOPs for distillation and $1.33 \times 10^{20}$ FLOPs for synthetic data generation). However, these operations require access to a 70B-parameter model and substantial GPU resources, which may not be available to all practitioners.

• The cost of filtering false positives in terms of data efficiency: The strong filter removes 8.42% of pretraining documents. If a non-trivial fraction of these are false positives (the paper acknowledges this is likely but unquantified), then the model is trained on less data than the unfiltered baseline, which could affect capabilities in ways not captured by the evaluated benchmarks. The paper's finding of no degradation on MMLU-No-Bio, PIQA, LAMBADA, and HellaSwag is reassuring but not exhaustive — there may be capabilities not measured by these benchmarks that are degraded.

• Amortization across multiple training runs: The filtering pipeline's upfront costs (blocklist generation, classifier training) are largely one-time costs that can be amortized across many model training runs. The per-training-run filtering inference cost is $6.92 \times 10^{19} + 7.77 \times 10^{18} = 7.70 \times 10^{19}$ FLOPs for end-to-end weak filtering (Table 3), or 0.25% of training FLOPs. This is the more relevant number for practitioners who would adopt an existing filtering pipeline, and it is substantially lower than the 0.83% total cost including upfront development.

Overall assessment: The paper provides strong evidence that pretraining data filtering can suppress biothreat proxy knowledge in 6.9B models with minimal capability degradation and substantially greater tamper-resistance under fine-tuning attacks than existing post-training methods. The experiments are thorough within their defined scope, and the paper is transparent about limitations that bound the generality of the findings. The primary scientific uncertainty is not whether the reported effects are real, but whether they scale to larger models, generalize to other dual-use knowledge domains, and translate from proxy knowledge suppression on multiple-choice benchmarks to genuine capability suppression in real-world use cases. The paper makes progress on an important practical problem and provides a foundation for future work to address these open questions, while carefully avoiding overclaiming about the completeness or generality of the solution.

6. Limitations and Trade-offs

The Difficulty Estimation Cost Remains Unaccounted For and Likely Dominates the Filtering Overhead

The paper's headline result — that filtering costs less than 1% of total training FLOPs — captures only the inference cost of the filtering pipeline once it has been constructed. The upfront cost of building the filtering infrastructure is substantially larger and is only partially accounted for.

The filtering pipeline depends on several expensive prerequisite steps. First, the blocklist is generated by prompting Llama 3.3 70B (a 70B-parameter model requiring significant GPU resources) to extract keywords from all 15,231 gold-standard proxy documents, then a second round of refinement to confirm each keyword's relevance. The FLOPs for this are reported as $4.45 \times 10^{19}$ for distillation and $1.33 \times 10^{20}$ for synthetic data generation (Table 3, Appendix C). Second, the ModernBERT classifier is trained on a labeled dataset of 198,184 documents — but labeling those documents itself requires Llama 3.3 70B inference on approximately 75,000 DCLM documents. Third, the entire pipeline assumes access to expert-curated resources: the WMDP-Bio Forget and bio Retain corpora (24,453 and 66,360 expert-labeled papers respectively), developed by subject matter experts at significant human cost.

More critically, the paper acknowledges (Section 3.2, implicitly in the difficulty estimation discussion) that evaluating whether filtering worked requires the same kind of proxy benchmark that filtering was designed to defeat. In other words, to know that your filter has successfully removed biothreat knowledge from a training corpus, you must already have a labeled dataset of biothreat documents — the very resource that is scarce and expensive to produce for new domains.

Consequence: The FLOPs cost of approximately 0.83% of training compute is misleading as a measure of the total cost of adopting this approach. For a practitioner facing a new dual-use domain (e.g., cybersecurity exploits, chemical synthesis procedures), the primary cost would not be the filtering inference FLOPs but the human-expert and large-model inference cost of developing the labeled datasets, blocklists, and classifier training data necessary to build the filtering pipeline in the first place. The paper provides no methodology for estimating or reducing these upfront costs, and the reported <1% figure only applies to the marginal cost of running an already-built pipeline on a new training run.

Evidence in the paper: Table 3 breaks down the filtering FLOPs by component, but the human-expert cost of creating the WMDP benchmark (which provides the gold labels) is not quantified. The paper's qualitative acknowledgment that "the vast majority of filtered documents come from Semantic Scholar (Pes2o)" and that "our filters likely have high false-positive rates" (Appendix J) suggests that the current pipeline is specifically tuned to the distribution of scientific papers, and adapting it to other domains would require substantial re-engineering. Section 6.5 acknowledges future work on "establishing scaling trends for data filtering" but does not discuss the cost of developing filtering infrastructure for new domains.

Mitigation status: Not addressed. The paper focuses on demonstrating that filtering is technically feasible and effective rather than cheap to set up. The authors are transparent that they "relied on the WMDP-Bio Forget corpus" — an existing expert-curated resource — and do not claim that building filtering infrastructure for arbitrary domains would be similarly inexpensive.

---

The Approach Only Works for Precise Factual Knowledge, Not for Behavioral Propensities

The paper proposes (Section 6.2) a domain-dependent hypothesis: that filtering is effective for suppressing precise knowledge (scientific facts, engineering procedures) but not for suppressing emergent propensities (toxicity, willingness to comply with harmful requests, alignment with harmful values). This hypothesis is central to understanding when the method applies, yet the paper provides only indirect and partially conflicting evidence for it.

The hypothesis itself is plausible. To generate a correct answer to a factual question like "What is the role of the NS2B-I39V mutation in ZIKV virulence?", the model must have encountered that specific information during training. Filtering the documents that contain it should therefore prevent the model from learning it. In contrast, producing toxic text or complying with a harmful request does not require any specific factual knowledge — it is a general behavioral tendency that may arise from diffuse patterns in the training distribution, including patterns in how people talk about sensitive topics even when the specific harmful information is absent.

However, the paper's evidence for this distinction is largely correlational. The positive results on biothreat proxy knowledge (Figures 3-5, Section 3) and the negative or mixed results on jailbreaking using models from Maini et al. (2025) (Appendix G, Figure 10) differ along multiple dimensions simultaneously: the type of capability (knowledge vs. behavior), the nature of the filtered data (scientific papers vs. broadly toxic web text), the evaluation format (knowledge tests vs. harmfulness judgments by GPT-4o), and the attack types (knowledge elicitation vs. jailbreaking). The paper cannot isolate which of these dimensions drives the differential effectiveness.

Consequence: A practitioner deciding whether to use data filtering for a new safety concern cannot reliably predict whether it will work based on the paper's findings. If the unwanted capability is a form of knowledge (e.g., how to synthesize a dangerous compound), the paper suggests filtering will be effective. If the unwanted capability is a form of behavior (e.g., providing detailed instructions when asked about a dangerous topic, even if the model lacks precise knowledge), the paper's own evidence suggests filtering may be insufficient or even counterproductive — the models from Maini et al. (2025) trained on filtered data were more vulnerable to few-shot jailbreaking attacks than unfiltered baselines (Appendix G, Figure 10). The boundary between "knowledge" and "propensity" is not sharp, and many real-world safety concerns involve both (e.g., a model that knows facts about explosives and is willing to share them when prompted adversarially).

Evidence in the paper: The jailbreaking experiments in Appendix G (Figure 10) show that filtered models from Maini et al. (2025) exhibit only slightly improved resistance to fine-tuning attacks and substantially increased vulnerability to few-shot attacks compared to unfiltered baselines. The paper's own synthetic document training experiments (Appendix F, Figure 9) show that training filtered models on incorrect biology text sometimes increases their biothreat proxy knowledge scores — a finding consistent with the idea that exposure to any domain-relevant text, even incorrect text, can improve performance on domain evaluations. Both results undermine the straightforward narrative that "removing data removes capability" and suggest that the relationship between training data composition and model behavior is more complex than simple data removal can address.

Mitigation status: The paper carefully scopes its claims: the title specifies "builds tamper-resistant safeguards" rather than "solves open-weight safety," and Section 6.2 explicitly discusses the domain-dependent hypothesis and its limitations. The authors characterize the hypothesis as speculative ("we speculate that this hypothesis only applies to…"). The paper does not propose a method for determining a priori whether a given safety concern falls into the "knowledge" or "propensity" category, leaving this as a judgment call for practitioners.

---

Single Benchmark, Single Model Scale, Single Model Family

All of the paper's positive results — the demonstration that filtering suppresses biothreat proxy knowledge and resists fine-tuning attacks — are obtained on a single benchmark (WMDP-Bio and its curated subsets), with a single model scale (6.9B parameters), a single architecture (Pythia), trained on a single data distribution (DCLM), using a single filtering target (biothreat biology). The paper does provide comparative results on models from Maini et al. (2025) at the 1.7B scale (Appendix G), but these experiments concern jailbreaking (a different safety target) and produce negative or mixed results for filtering.

The paper acknowledges this limitation explicitly in Section 6.3: "We only study unimodal 6.9B parameter language models without instruction fine-tuning. Due to logistical and financial considerations, we only trained these models on 550B tokens until they surpassed 50% performance on the standard WMDP-Bio eval. They do not have competitive capabilities with other open LLMs of similar sizes. Additionally, we also only experiment in the context of biothreat proxy knowledge."

Consequence: The paper cannot distinguish between findings that are specific to biothreat biology (which may be particularly amenable to filtering because the relevant knowledge appears predominantly in a concentrated set of scientific papers) and findings that would generalize to other dual-use domains (cybersecurity, chemical synthesis, etc.). It also cannot determine whether the relationship between filtering and tamper-resistance scales with model size. As models grow larger, they may become more sample-efficient at extracting information from sparse signals — a 70B or 405B parameter model trained on the same filtered dataset might be able to reconstruct biothreat knowledge from residual, unflagged documents that a 6.9B model cannot. Conversely, larger models might overfit less to specific documents, making targeted filtering more effective. Without scaling experiments, neither hypothesis can be evaluated.

The specific training configuration also limits generalizability. The models were trained on only 550B tokens — substantially less than state-of-the-art models of similar size (Llama 3.1 8B was trained on approximately 15T tokens). The models achieve 50% on standard WMDP-Bio, while comparable models achieve 60-72% (Table 11, Appendix H). It is possible that continuing training on filtered data would eventually allow the model to acquire biothreat knowledge through more diffuse learning across many marginally-relevant documents — the filtered models might simply be slower to learn biothreat knowledge rather than permanently incapable of learning it.

The paper's own comparison with other open-weight models (Table 11, Appendix H) reveals that all models exhibit substantial drops between full WMDP-Bio and the curated subsets (e.g., Llama 3.1 8B Instruct drops from 72.43% to 46.28% on Verified Cloze), but the filtering models' performance is the lowest. This is consistent with filtering being effective but does not establish that filtering would achieve comparable suppression in a model with more training data, better hyperparameters, or a different architecture.

Mitigation status: The paper is transparent about this limitation in Section 6.3 and explicitly calls for "training larger, more capable, and/or multimodal model organisms for data filtering" in Section 6.5. The release of the model suite is positioned as enabling future research on these questions. The authors do not claim their specific accuracy numbers will transfer to other scales or domains. However, the paper's title and abstract claims about "tamper-resistant safeguards" and "state-of-the-art tamper-resistance" are presented without qualification about scale or domain specificity, which could mislead readers who do not carefully examine the experimental constraints.

---

The Protection Is Fully Reversible Through Combined Fine-Tuning and In-Context Retrieval

Section 4.1 demonstrates that no defense tested — including the combination of filtering with Circuit-Breaking — withstands a staged attack that first fine-tunes the model on biothreat text, then provides biothreat knowledge in context at inference time. This is the paper's most important negative result, and it reveals a fundamental limitation: filtering prevents the model from internalizing biothreat knowledge, but does nothing to prevent the model from using such knowledge when it is provided in the prompt.

The mechanism is straightforward. Filtering removes biothreat text from pretraining, so the model's weights do not encode biothreat facts. However, the model's general language understanding and reasoning capabilities remain intact — it can still read, comprehend, and reason about text presented in its context window. When an attacker provides a passage containing biothreat information (e.g., a paper abstract) and asks a question about it, the filtered model performs comparably to the unfiltered baseline (Figure 6, open-book condition). When the attacker first fine-tunes the model on biothreat text to break any Circuit-Breaking protections, then provides biothreat text in context, the combined attack succeeds against all defenses.

Consequence: In practice, any deployment where users can provide text to the model (e.g., through document upload, web search integration, retrieval-augmented generation, or simply long prompts) is vulnerable to this attack. This covers virtually all real-world LLM applications. The filtering defense only protects against scenarios where the user interacts with the model through short, uninformative prompts and the model must rely entirely on its internal knowledge — a narrow threat model that excludes the most common usage patterns for LLMs (chat interfaces with document context, coding assistants with file access, research tools with paper upload).

The paper frames this finding as motivation for defense-in-depth (Section 4.1), arguing that filtering and Circuit-Breaking are complementary (filtering blocks internalized knowledge, CB blocks in-context retrieval). But the staged attack defeats both simultaneously: the fine-tuning step strips the CB protection (the LoRA adapters become ineffective when the model's weights are substantially modified), and the in-context passage provides the biothreat knowledge that filtering prevented the model from internalizing. The defense-in-depth strategy relies on both layers holding simultaneously, and the attack breaks them sequentially — first CB through fine-tuning, then filtering through in-context retrieval.

Evidence in the paper: Figure 6 (Section 4.1) shows the ensemble attack results. The filtered models achieve high accuracy (approximately 55-65%) on the open-book evaluation after fine-tuning, comparable to the unfiltered baseline. The combined filtered+CB models, which showed strong resistance to either attack individually, similarly fail under the staged attack. The paper states this finding clearly: "no models resist an ensemble fine-tuning + open-book attack" (Figure 6 caption). This is not a minor caveat — it demonstrates that the core defense mechanism (knowledge prevention through data removal) is fundamentally circumventable by any attacker who can provide relevant text at inference time.

Mitigation status: The paper does not attempt to solve this problem and does not propose a defense against staged fine-tuning + retrieval attacks. This is appropriate for a research paper that aims to characterize the strengths and weaknesses of an approach rather than claim to have solved open-weight safety. However, the implication is substantial: data filtering alone cannot be a sufficient safeguard for any model deployed in a context where users control the input text, which is effectively all deployment contexts. The paper's framing of filtering as "a promising layer of defense" within a broader strategy is accurate, but the staged attack result suggests that the other layers in that strategy do not yet exist.

---

The Filtered Models Are Not Competitively Capable, Undermining Claims About Practical Deployability

The paper states that its models were "not trained to be competitive with state-of-the-art models of similar size" and were trained "only to the point of reaching 50% accuracy on the public WMDP-Bio benchmark" (Section 2.2). Table 11 (Appendix H) shows that other 7-8B parameter open models achieve 60-72% on the standard WMDP-Bio eval, compared to the paper's unfiltered baseline at 52.79%.

This is not necessarily a problem for the paper's scientific claims — the experimental design does not require state-of-the-art general capabilities to test whether filtering suppresses specific knowledge. However, it substantially weakens the practical conclusions that a practitioner might draw. The paper demonstrates that a relatively weak model can be made "bio-ignorant" through filtering. But the practical concern motivating the work — as articulated in the introduction's quotes about Gemini, Claude, and ChatGPT reaching concerning capability thresholds — is about highly capable models that possess dangerous knowledge alongside powerful general reasoning. The paper does not demonstrate that the same filtering approach would work on a model with strong general capabilities trained on trillions of tokens, where the relationship between filtered data and capability acquisition may differ.

Consequence: A practitioner reading this paper might conclude that filtering is a viable strategy for their own model development, but the paper provides no evidence that the approach scales to the capability regimes where dual-use risks are most concerning. A model that barely achieves 50% on the benchmark used to measure the capability being suppressed (WMDP-Bio) is in a fundamentally different regime from a model that achieves 70%+ on the same benchmark while also possessing strong reasoning, coding, and instruction-following abilities. In the latter regime, the model's general intelligence might enable it to infer biothreat knowledge from sparse or indirect signals in the training data — signals that would be insufficient for a weaker model but that a stronger model's reasoning capabilities could exploit.

The paper's FLOPs-matched comparison framework (if one existed) would need to compare a filtered, highly-capable model against an unfiltered, highly-capable model to establish that filtering does not disproportionately impact the capabilities that make the model useful. The current experiments compare filtered and unfiltered versions of a deliberately capability-limited model, which provides evidence about the mechanism of knowledge suppression but not about the tradeoff between capability and safety that practitioners face.

Evidence in the paper: Section 2.2 explicitly acknowledges the capability limitation: "Our models are not trained to be competitive with state-of-the-art models of similar size. We only trained models to the point of reaching 50% accuracy on the public WMDP-Bio benchmark and dedicated our remaining compute budget to filtering experiments." Table 11 (Appendix H) quantifies the gap: OLMO-2 7B Instruct achieves 66.38% on standard WMDP-Bio vs. 52.79% for the paper's unfiltered baseline, and 45.35% vs. 36.34% on Verified Cloze. The 14-16 percentage point gap on standard WMDP-Bio is substantial and indicates that the paper's models have meaningfully different capability profiles than comparable open-weight models.

Mitigation status: The paper acknowledges this limitation in Section 6.3 and Section 6.5, calling for future work on "training larger, more capable, and/or multimodal model organisms for data filtering." The model suite release is explicitly intended to enable such research. The authors are transparent that their models are "model organisms" for studying filtering mechanisms rather than deployment-ready systems. However, the paper's abstract and introduction do not qualify their claims with this capability limitation, which could lead readers to overestimate the readiness of the approach for practical deployment.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper shifts the center of gravity of LLM safety research from post-training suppression toward pretraining prevention — a conceptual reframing with concrete methodological consequences. The dominant paradigm for the past several years has treated safety as an add-on: train the most capable model possible on whatever data is available, then apply fine-tuning, RLHF, circuit-breaking, or unlearning techniques to suppress unwanted behaviors the model has already acquired. This paper demonstrates that this paradigm has a hard ceiling — the "wrapper" problem described by Jain et al. (2023) — and that moving the safety intervention earlier in the pipeline can produce qualitatively more robust guarantees.

The scale of the improvement is what makes this more than an incremental advance. The paper shows that filtering resists more than an order of magnitude more adversarial fine-tuning than existing post-training techniques: 10,000 steps versus the dozens to hundreds that prior work tested (Table 10, Appendix E). This is not a marginal gain from better hyperparameters — it reflects a fundamentally different safety mechanism. Post-training methods suppress existing neural representations; filtering prevents those representations from forming. The difference manifests empirically as a roughly 10-20× improvement in the number of fine-tuning steps a model can withstand before unwanted capabilities reemerge (Figures 4, 5).

The paper also resolves a genuine tension in the recent literature. Maini et al. (2025) and Li et al. (2025a) found that filtering toxic content sometimes made models less robust to attacks, leading them to argue that "safety is not about censorship." This paper does not dispute those findings — it replicates elements of them (Appendix G) — but provides a domain-dependent boundary condition that explains the apparent contradiction. Filtering works for precise factual knowledge (biothreat biology, likely cybersecurity exploits, chemical synthesis procedures) because such knowledge arises from identifiable document subsets. It works poorly for emergent behavioral propensities (toxicity, willingness to comply with harmful requests) because these arise from diffuse patterns in the training distribution and may require exposure to negative examples for the model to learn avoidance. This distinction converts a confusing empirical landscape into a coherent framework for predicting when data filtering will be effective.

For the research community, this shifts priorities along several axes:

• Making verifier and unlearning research less central for fact-based capability suppression. The paper demonstrates that for suppressing precise dual-use knowledge, training a better process reward model or developing more sophisticated unlearning algorithms is attacking the wrong problem — the capability is already encoded in the weights, and suppression will always be shallower than prevention. Resources previously directed at improving post-training tamper-resistance for factual knowledge should arguably shift toward improving data filtering pipelines and understanding their scaling properties.

• Elevating data curation from a preprocessing detail to a first-class safety intervention. Data filtering has long been practiced by frontier model developers (the paper documents that Gemma 3, GPT-OSS, Llama 4, and Claude Opus 4 all mention filtering efforts), but the scientific community lacked open, reproducible models and evaluations to study it systematically. This paper provides that foundation — released models, documented filtering decisions, and rigorous evaluation protocols — establishing data curation as a legitimate and necessary topic for safety research rather than an opaque industry practice.

• Making "inability arguments" empirically grounded. The safety case literature (Buhl et al., 2024; Goemans et al., 2024) has argued that a model's robustness to diverse tampering attacks would constitute evidence that it "fundamentally lacks neural circuitry for harmful behaviors." But prior to this work, no method had demonstrated tamper-resistance at a scale that would make such an argument credible for dual-use capabilities. The paper's demonstration of 10,000-step resistance — under attack configurations larger than any prior work — provides the first empirical foundation for inability-based safety cases grounded in data curation rather than (easily stripped) post-training suppression.

• Exposing the in-context retrieval vulnerability as a fundamental limitation that requires new solutions. The paper's most important negative result — that no defense withstands combined fine-tuning + in-context retrieval attacks (Figure 6, Section 4.1) — clarifies the boundary of what data filtering can achieve. It cannot prevent a model from using harmful knowledge provided at inference time. This redirects attention toward the problem of controlling how models process and reason about their input context, which is a fundamentally different challenge than controlling what knowledge they internalize during training. The paper's demonstration that Circuit-Breaking partially addresses this (by impairing the model's ability to process biothreat-related text regardless of source) points toward representation-level interventions as a complementary defense layer, but the staged attack result shows current methods are insufficient.

Follow-Up Research This Work Enables

Scaling filtering to larger models and measuring whether capability suppression holds at frontier scales. The paper trains only 6.9B-parameter models on 550B tokens. The central practical question is whether the same approach works at the 70B–405B scale trained on 10-15T tokens, where dual-use capabilities become genuinely concerning. A strong follow-up would train filtered and unfiltered models at multiple scales (e.g., 1B, 7B, 30B, 70B parameters) on the same filtered corpus, measure WMDP-Bio suppression as a function of model scale, and test whether larger models can reconstruct biothreat knowledge from sparse residual signals that smaller models miss. The hypothesis to test is whether filtering effectiveness decays with model scale (because larger models are more sample-efficient at extracting information from limited data) or remains constant or improves (because larger models overfit less to specific documents). The paper provides the filtering pipeline and evaluation framework; a scaling study would directly inform whether the approach is viable for frontier models or is limited to smaller-scale research settings.

Constructing inability-based safety cases using filtered models as the demonstration vehicle. The paper argues (Section 5, Section 6.1) that robustness to diverse tampering attacks constitutes evidence of "deep ignorance" — that filtered models fundamentally lack the neural circuitry for biothreat knowledge rather than merely suppressing it. A rigorous follow-up would combine the filtering approach with interpretability methods to mechanistically verify this claim. Specifically: (1) train sparse autoencoders or probes on the filtered and unfiltered models to determine whether biothreat-relevant features are absent from the filtered model's representations or merely harder to extract; (2) apply activation patching and causal tracing to demonstrate that the filtered model's responses to biothreat questions rely on generic language heuristics rather than factual retrieval circuits; (3) test whether fine-tuning the filtered model on completely unrelated domains (e.g., code generation, literature analysis) sometimes revives biothreat knowledge, which would indicate latent representations rather than true absence. The paper's released model suite (filtered and unfiltered versions of the same architecture trained on the same data distribution except for the removed documents) provides an ideal testbed for these analyses, since differences between models can be causally attributed to the removed data rather than confounded by architecture or training recipe differences.

Characterizing the filtering knowledge–propensity boundary with controlled experiments. The paper's domain-dependent hypothesis (Section 6.2) — that filtering works for precise knowledge but not for emergent propensities — is plausible but supported only by cross-domain comparisons (biothreat facts vs. jailbreaking compliance). A targeted follow-up would test this hypothesis within a single domain where "knowledge" and "propensity" can be independently manipulated. For example: train models on a corpus of chemistry knowledge with two separate filtering targets — (1) removal of documents containing specific synthesis procedures for dangerous compounds (precise knowledge) and (2) removal of documents containing toxic or violent language when discussing chemistry (propensity). Then evaluate whether the filtered models show differential tamper-resistance for procedure-recall questions versus toxic-language-generation prompts under the same fine-tuning attacks used in this paper. If the hypothesis is correct, filtering should suppress procedure recall robustly but show the Maini et al. (2025) pattern for toxicity — possibly even increasing vulnerability to certain attacks. This would directly validate or falsify the paper's proposed boundary condition.

Developing cheap, generalizable filtering infrastructure that does not depend on domain-specific expert labels. The paper's filtering pipeline relies heavily on the WMDP-Bio benchmark — an expert-created resource with manually labeled proxy and retain documents. For a new dual-use domain (e.g., cybersecurity, chemical synthesis, autonomous systems), equivalent resources may not exist. A practical follow-up would develop methods for bootstrapping filtering pipelines from small amounts of domain expertise: given a short list of 50-100 target concepts or capabilities, can an LLM generate a reasonable blocklist and training data for a ModernBERT classifier without thousands of expert-labeled documents? The paper's blocklist generation approach (Llama 3.3 70B extracting keywords from proxy documents) provides a template, but it depended on having the labeled proxy documents as input. A strong follow-up would test whether the pipeline can be "primed" with only capability descriptions (e.g., "documents describing how to synthesize compound X") rather than example documents, measure the resulting precision/recall on a held-out test set, and compare the downstream tamper-resistance of models trained on the bootstrapped-filtered corpus versus models trained on an expert-filtered corpus. This would determine whether the approach can scale to new domains without incurring prohibitive expert labeling costs — the primary barrier to practical adoption identified in Section 7 (this section, above).

Stress-testing filtering against attack strategies specifically designed to exploit its failure modes. The paper demonstrates that filtered models resist domain-specific fine-tuning far longer than post-training baselines, but the attacks are straightforward: fine-tune on the WMDP-Bio Forget set and measure capability recovery. A determined adversary would design attacks targeting the specific mechanism of filtering-based defense. Valuable follow-up attacks include: (1) curriculum fine-tuning — start with simple biology concepts the filtered model does know (basic cell biology, genetics) and gradually escalate to biothreat-relevant content, testing whether foundational biology knowledge serves as a scaffold for more efficient acquisition of the suppressed knowledge; (2) multilingual attacks — fine-tune on biothreat text in languages other than English (since the filtering pipeline was developed and evaluated on English text only), testing whether the model can acquire biothreat knowledge through cross-lingual transfer from unfiltered non-English documents; (3) multimodal attacks — if the filtered model were extended to process images, test whether providing diagrams and figures alongside biothreat text accelerates knowledge acquisition, since the filtering pipeline only processes text; (4) adversarial document generation — use an LLM to rewrite biothreat proxy documents to evade the specific blocklist and classifier used in filtering (e.g., replacing all blocklist terms with synonyms, euphemisms, or code words), then fine-tune on these rewritten documents to test whether the filtering pipeline's recall degrades under deliberate evasion. Each of these attack strategies probes a different assumption underlying the filtering defense, and negative results (filtering resists them) would substantially strengthen the safety case, while positive results (filtering fails) would clarify its limitations for practitioners.

Combining filtering with representation-level monitoring for defense-in-depth against staged attacks. The paper's most important failure mode (Section 4.1, Figure 6) is that no defense resists combined fine-tuning + in-context retrieval attacks. A natural follow-up would develop and test a three-layer defense architecture: (1) pretraining data filtering to prevent internalized knowledge acquisition, (2) Circuit-Breaking or similar representation-level interventions to impair in-context processing of biothreat text, and (3) anomaly detection at inference time — a monitor that detects when a model's activations indicate it is processing biothreat-relevant text, regardless of source (internal knowledge or in-context passage). The paper's finding that CB models show reduced performance on open-book biothreat evaluations (Figure 6) demonstrates that representation-level interventions can impair in-context processing, but fine-tuning strips this protection. If a monitoring system could detect fine-tuning (e.g., by measuring the divergence of the model's activations from a reference checkpoint on a set of held-out prompts), it could trigger fallback measures (refuse to answer, route to a safer model, alert administrators) before the adversary exploits the now-unprotected model. A strong follow-up would implement this full pipeline, measure the end-to-end attack success rate under the staged attack from Section 4.1, and characterize the tradeoff between monitor sensitivity (catching fine-tuning early) and false positive rate (blocking legitimate adaptations).

Practical Applications and Downstream Use Cases

Controlled release of open-weight models with domain-specific capability restrictions. The most direct application of this work is for model developers who want to release open-weight models but are concerned about specific dual-use capabilities (biothreat knowledge, cybersecurity exploits, chemical weapon synthesis). The paper provides a concrete, evaluated methodology: (1) identify the domain(s) of concern, (2) construct a filtering pipeline using the multi-stage blocklist + classifier approach, (3) train the model from scratch on the filtered corpus, (4) verify capability suppression using curated evaluations that control for shortcut exploitation (as the paper does with Robust MCQA and Verified Cloze), and (5) document the filtering decisions and residual risks transparently. The paper's demonstration that this approach preserves general capabilities (MMLU-No-Bio, PIQA, LAMBADA, HellaSwag within normal variance of the unfiltered baseline; Figure 3) addresses the primary objection that capability-restricted models would be useless. The 6.9B scale, while below frontier capabilities, is sufficient for many open-weight use cases (research, education, domain-specific fine-tuning), and the filtering adds less than 1% to training FLOPs (Appendix C, Equation 2: 0.83% for end-to-end weak filtering), making it economically viable even for resource-constrained organizations. The paper's release of model checkpoints, training configurations, and evaluation protocols provides a template that other developers can adapt to their specific capability concerns and training pipelines.

Evidence generation for regulatory compliance and safety case documentation. Frontier model developers are increasingly expected to document their safety procedures and provide evidence that their models do not pose unacceptable risks (as indicated by the paper's opening quotes about Google DeepMind, Anthropic, and OpenAI activating safety protocols). The paper's methodology provides a structured approach to generating one specific type of evidence: capability absence through data curation. A developer could adopt the filtering pipeline, train a model, and then document: the filtering methodology (blocklist terms, classifier architecture, training data composition), the evaluation protocol (curated benchmarks, attack regimes), the results (WMDP-Bio or equivalent scores, tamper-resistance under fine-tuning attacks up to a specified number of steps), and the limitations (in-context retrieval vulnerability, domain-specificity). This evidence package would support an inability-based safety argument: "we have high confidence the model lacks capability X because (a) documents containing X-relevant information were removed from training, (b) the model performs near random chance on curated evaluations of X, and (c) the model resists fine-tuning attacks designed to reintroduce X for up to N steps." The paper's Table 10 (Appendix E), comparing attack configurations across prior work, provides a basis for calibrating what "sufficient" tamper-resistance testing looks like. A developer could adopt the paper's attack scale (10,000 steps, 300M tokens, 80,000 unique examples) as a reference point for their own testing, arguing (with appropriate caveats) that if their model resists attacks at this scale, it constitutes meaningful evidence of capability absence.

Selective data release for research on safety mechanisms. The paper's release of 11 filtered and unfiltered models (Table 1, Appendix A) — trained on the same architecture, same data distribution, and same hyperparameters, differing only in which documents were filtered — provides a uniquely clean testbed for studying how training data composition affects learned representations. This has applications beyond biothreat safety. Researchers studying machine unlearning can use these models to test whether unlearning algorithms genuinely erase knowledge (by applying them to the unfiltered baseline and measuring whether they can reproduce the filtered model's near-random-chance WMDP-Bio performance) or merely suppress it (if fine-tuning on the WMDP-Bio Forget set revives knowledge faster from the unlearned model than from the filtered model). Researchers studying knowledge localization can compare filtered and unfiltered models to identify which model components (attention heads, MLP layers, specific neurons) encode biothreat-relevant information — these components should be present in the unfiltered model and absent or differently configured in the filtered model. Researchers developing better filtering methods can use the paper's pipeline as a baseline and test whether their methods produce models with equivalent or better tamper-resistance while removing fewer documents (higher precision). The key contribution is that the models are released, documented, and paired with rigorous evaluation protocols, lowering the barrier to entry for safety research that previously required expensive pretraining runs from scratch.