Uncertainty-Aware Gradient Signal-to-Noise Data Selection for Instruction Tuning

🎯 Pitch

A tiny GPT-2 proxy, fitted with multiple LoRA heads, reveals that the most valuable instruction-tuning examples are not the hardest or cleanest, but those where gradient drops rapidly while ensemble agreement remains high—a signal they call G-SNR. Training on just 5–15% of data ranked this way matches full-dataset performance and consistently beats existing filters, even accelerating convergence.

1. Executive Summary

This paper proposes GRADFILTERING, an uncertainty-aware data selection framework for instruction tuning that uses a small GPT-2 proxy with a LoRA ensemble to compute a Gradient Signal-to-Noise Ratio (G-SNR) utility — a score combining relative gradient drop between early and late training with late-epoch ensemble variance — and selects subsets based on that ranking. Evaluated on Alpaca and Alpaca-GPT4 with LLaMA-2-7B/13B under both LoRA and full-parameter fine-tuning, models trained on GRADFILTERING-selected 5–15% subsets match or outperform random splits and the Superfiltering baseline in 19 out of 24 LLM-as-a-judge pairwise comparisons, with human evaluation confirming the same preference trends. The selected subsets also converge faster and reach lower training loss than competitive filters under identical compute budgets, establishing that uncertainty-aware gradient statistics provide an effective signal for curating large instruction-tuning corpora without requiring task-specific validation sets or external proxy scores.

2. Context and Motivation

The Core Problem: Instruction Datasets Are Too Large for Full Fine-Tuning

The fundamental problem this paper addresses is deceptively simple: modern instruction-tuning datasets contain hundreds of thousands of examples, but only a small fraction of them meaningfully contribute to a model's instruction-following ability. The practical consequence is that full-data fine-tuning is expensive and often wasteful — many examples are redundant, noisy, or trivially easy, consuming GPU-hours and memory bandwidth without improving the final model's capability. If we could identify which subset of training examples actually matters, we could fine-tune on a fraction of the data (5–15%) and achieve comparable or better performance while dramatically reducing cost.

This is not merely an efficiency concern — it has structural implications for how instruction tuning is done. Instruction datasets are typically constructed through automated pipelines: self-instruction (Wang et al., 2023), distillation from stronger models (Peng et al., 2023; Xu et al., 2023), or scraping and filtering web sources (Taori et al., 2023). These pipelines produce data at scale but with limited quality control. The resulting corpora contain:

Redundant examples: Many instruction–response pairs are near-duplicates or teach the same skill repeatedly, contributing diminishing returns.
Noisy examples: Automated generation produces ill-formed instructions, inconsistent responses, or content that contradicts the desired model behavior.
Trivial examples: Instructions that are too easy (e.g., simple factual queries the base model already handles) provide minimal learning signal but still consume compute.

The paper's central thesis — reflected in its appeal to "uncertainty-aware" selection — is that these different categories of examples exhibit distinct training dynamics signatures that can be detected via their gradients. Redundant examples produce small, consistent gradients; noisy examples produce large but highly variable gradients across ensemble members; valuable examples produce large gradients that decay steadily as training progresses, indicating they genuinely drive learning. A data selection method that can distinguish these signatures would enable practitioners to curate instruction datasets programmatically, without manual inspection or task-specific heuristics.

Why This Problem Matters Practically

The significance of efficient data selection for instruction tuning extends beyond the narrow goal of reducing training cost. Several practical considerations make this a pressing problem:

The cost scaling of instruction tuning is steep and non-negotiable in many settings. Full-parameter fine-tuning of a 7B-parameter model on 52,000 instruction–response pairs (the scale of Alpaca) requires substantial compute — and this is at the small end of modern practice. Models like LLaMA-2-13B, LLaMA-2-70B, or their instruction-tuned variants would require proportionally more. For organizations fine-tuning models regularly (e.g., adapting to new domains, incorporating user feedback, or maintaining versioned deployments), the cumulative cost of full-data training is significant. Reducing the effective dataset size by 85–95% without degrading quality translates directly to faster iteration cycles, lower GPU costs, and reduced energy consumption.

Data quality directly impacts safety and alignment. Instruction tuning is the primary mechanism for aligning LLMs with human preferences — teaching them to be helpful, harmless, and honest (Ouyang et al., 2022). If the training data contains noisy or contradictory examples, the aligned model may inherit those inconsistencies. A principled data selection method that identifies high-quality, high-signal examples could produce better-aligned models than training on the full, unfiltered dataset. This flips the typical narrative: data selection is not just about doing more with less, but potentially about doing better with less, by excluding harmful training signals.

The instruction-tuning ecosystem lacks a general, task-agnostic selection criterion. Different downstream applications require different instruction-following behaviors: a coding assistant needs different training data than a creative writing tool or a customer support agent. Current data selection methods either require a predefined validation set representative of the target task (tying selection to specific downstream objectives) or rely on static proxy scores (perplexity, IFD) that do not adapt to the training dynamics of the specific model being fine-tuned. A method that works across tasks and models without per-task calibration — which is what GRADFILTERING claims to provide via its "objective-agnostic" design — would be a general-purpose tool that simplifies the deployment of instruction-tuned models across diverse applications.

Where Prior Approaches Fall Short

The paper situates itself against two broad families of prior work, each with distinct limitations that motivate the GRADFILTERING design.

Classical Influence Functions: Theoretically Grounded but Impractical for LLMs

The most principled approaches to data valuation come from the influence function literature. Influence functions (Koh and Liang, 2017) estimate the effect of upweighting or removing a specific training example on the model's loss at a target point. The key idea is to compute:

$\mathcal{I}_{\text{up,loss}}(z_i, z_{\text{test}}) = -\nabla_\theta L(z_{\text{test}}, \hat{\theta})^\top H_{\hat{\theta}}^{-1} \nabla_\theta L(z_i, \hat{\theta})$

where $H_{\hat{\theta}}$ is the Hessian of the training loss at the converged parameters $\hat{\theta}$ , and $\nabla_\theta L(z_i, \hat{\theta})$ is the per-example gradient of training example $z_i$ . This formulation characterizes how a small perturbation to example $z_i$ would change the loss on test example $z_{\text{test}}$ , providing a theoretically grounded utility score.

However, this approach has three well-documented failure modes that make it unsuitable for modern LLM fine-tuning:

Computational intractability: The Hessian $H_{\hat{\theta}}$ for a model with billions of parameters is a matrix with quadrillions of entries — computing, inverting, or even storing it is infeasible. Approximations exist (e.g., conjugate gradient methods, LiSSA), but they introduce variance and require careful hyperparameter tuning.
Brittleness for deep networks: Influence functions rely on the assumption that the loss is approximately quadratic around the optimum, which holds locally for convex models but breaks down for deep networks with highly non-convex loss landscapes. Small parameter changes can lead to dramatically different influence estimates, making the method unreliable in practice for large-scale fine-tuning.
Dependence on a target task: Influence functions require specifying a particular test example or a validation set to compute influence against. If the downstream task changes or is unknown at training time, the computed influences lose relevance. This makes them inherently task-specific, which conflicts with the goal of general-purpose instruction tuning where the model should perform well on a broad distribution of instructions, not a predefined validation set.

First-order approximations such as TracIn (Pruthi et al., 2020) simplify the computation by accumulating per-example gradient dot products with test gradients along the optimization trajectory, avoiding the Hessian but retaining the dependence on a specific test set. They still do not scale cleanly to modern LLM fine-tuning pipelines due to the cost of storing and comparing gradients across thousands of training examples and optimization steps.

LESS: Sophisticated but Task-Dependent and Expensive

LESS (Xia et al., 2024) represents the most recent attempt to bring gradient-based data selection to instruction tuning. It introduces an optimizer-aware influence formulation that explicitly accounts for the Adam update rule (rather than treating gradient descent as a continuous flow, as standard influence functions do), and builds a gradient datastore using low-rank (LoRA-based) representations of per-example gradients. Data selection then becomes a nearest-neighbor search in this gradient store: given a small set of target validation examples, LESS retrieves training examples whose gradients are most similar, under the intuition that these examples will reduce the validation loss most effectively.

LESS achieves strong empirical performance, demonstrating that gradient-based similarity can identify relevant instruction-tuning data for specific downstream tasks. However, the paper identifies three structural limitations that motivate a different approach:

Computational and storage requirements: LESS must compute and store per-example gradient features for every training example using a strong base model (e.g., LLaMA-2-7B or larger). This gradient datastore is substantial — for 50,000+ examples with LoRA gradient features, the storage and nearest-neighbor search infrastructure is non-trivial.
Task dependency: LESS tailors selection to a particular downstream task by using that task's validation examples as queries. If you change the target task, you need to recompute selection — or at minimum re-query the datastore with new validation gradients. This makes LESS a targeted data selection method rather than a general-purpose curation tool. For organizations deploying instruction-tuned models across many tasks (or open-ended instruction following where no single validation set captures the desired behavior), this task-dependence is a practical limitation.
No explicit uncertainty modeling: LESS uses gradient similarity as a deterministic signal but does not model the confidence or stability of that signal. Two training examples with similar gradient directions might have very different noise characteristics — one might be a clean, informative example while the other is a spurious pattern that happens to align with the validation gradient by chance. Without uncertainty quantification, LESS cannot distinguish these cases.

GRADFILTERING is explicitly positioned as addressing these gaps: it uses a small proxy model (GPT-2) rather than the strong model itself to compute gradient features, making it cheaper; it is objective-agnostic because it computes utilities purely from training dynamics without reference to a validation set; and it explicitly models epistemic uncertainty via ensemble disagreement to down-weight noisy gradient signals.

Superfiltering (Li et al., 2024) is the paper's primary baseline and represents an alternative philosophy: weak-to-strong data selection using a small proxy model. The core idea is that small models (GPT-2) can identify high-quality instruction-tuning data for large models (LLaMA-2) because their judgments of data quality — measured via Instruction-Following Difficulty (IFD) — correlate well with the large model's eventual performance. The IFD score is computed as:

$\text{IFD}(x, y) = \frac{\text{PPL}(y | \text{prompt})}{\text{PPL}(y)}$

where $\text{PPL}(y | \text{prompt})$ is the perplexity of generating the response $y$ given the instruction prompt (using the proxy model), and $\text{PPL}(y)$ is the unconditional perplexity of the response alone. The ratio measures how much the instruction helps predict the response — higher values indicate that the instruction provides useful context, suggesting a well-formed instruction–response pair; lower values suggest the instruction is irrelevant or the response is generic. Superfiltering then selects the top-k examples by IFD score and fine-tunes the large model on this subset.

Superfiltering is effective and efficient — the proxy model is small and the IFD computation requires only forward passes. It often matches or exceeds full-data fine-tuning with small subsets (e.g., 10% of Alpaca), and the paper's own Table 1 confirms that Superfiltering is a strong baseline (e.g., PWS of 1.10 at 10% on Alpaca LLaMA2-7B LoRA).

However, the paper argues that Superfiltering has three fundamental limitations that GRADFILTERING addresses:

Static, per-example scoring: Each example receives a single IFD score from the pre-trained proxy model, independent of all other examples. This ignores interactions between training samples — an important consideration because gradient-based optimization accumulates parameter updates across minibatches. An example that looks informative in isolation might be redundant when combined with other examples teaching the same skill, while an example that looks only moderately useful might provide critical diversity that prevents overfitting. Static scoring cannot capture these complementarities.
No training dynamics: IFD scores are computed before any fine-tuning occurs, using the proxy model's pre-trained weights. They provide no signal about how the model learns from each example during training. An example might have a high IFD score because it contains novel vocabulary that challenges the proxy model, but if that vocabulary is irrelevant to instruction-following (e.g., domain-specific jargon that the large model already knows), the example may contribute little to the fine-tuning objective. Conversely, examples that initially look easy might be exactly the ones where the model learns to refine subtle aspects of instruction-following — a signal that only training dynamics can reveal.
No uncertainty quantification: IFD is a point estimate with no associated confidence measure. The proxy model might be poorly calibrated on certain types of instructions (long-form reasoning, multi-turn conversations, instructions requiring world knowledge), producing IFD scores that are systematically unreliable for those examples. Without an uncertainty signal, Superfiltering cannot distinguish between high-IFD examples where the proxy's judgment is trustworthy and high-IFD examples where the proxy is out of its depth.

GRADFILTERING is designed to overcome these limitations by (a) using gradient-based statistics that capture how training examples interact through shared parameter updates, (b) tracking gradient magnitudes across training epochs to measure learning progress, and (c) quantifying uncertainty via ensemble disagreement to down-weight noisy gradient signals. The key conceptual advance is the Gradient Signal-to-Noise Ratio (G-SNR), which combines a gradient-drop term (signal) with a late-epoch variance term (noise) into a single utility score.

The Missing Piece: Epistemic Uncertainty in Data Valuation

While data valuation and ensemble methods have existed independently, the paper identifies a critical gap: epistemic uncertainty has not been leveraged for gradient-based data selection in instruction tuning. The paper argues that this is a conceptual oversight because instruction-tuning datasets are precisely the kind of setting where uncertainty should matter most:

Noise is endemic: Automated instruction generation pipelines produce inconsistent quality. An example that looks informative to one model initialization might be perceived as noise by another. Uncertainty helps disambiguate: if multiple independent model trajectories agree that an example is valuable, we can be more confident in selecting it; if they disagree wildly, the example is likely noisy or ambiguous.
Distribution shift is common: Instruction-tuning datasets often contain examples from diverse sources (web scraping, synthetic generation, human annotation) with different stylistic and qualitative properties. Some subsets may be in-distribution for the proxy model while others are out-of-distribution, and the proxy's gradient signals will be more reliable for the former. Uncertainty provides a principled way to down-weight OOD examples whose gradient statistics are unreliable.
Training dynamics reveal quality: Not all learning is equally useful. An example that induces large gradient updates early in training might be teaching the model something it already knows (high loss due to poor initialization, not informativeness). An example whose gradient norm decays steadily from early to late epochs is likely driving genuine learning — the model extracts useful signal and the residual loss decreases. Epistemic uncertainty, measured as ensemble disagreement at late epochs, helps distinguish cases where the gradient drop reflects learning (ensemble members converge in their assessment) from cases where it reflects noise (ensemble members continue to disagree).

The paper draws on established work in Bayesian deep learning — deep ensembles (Lakshminarayanan et al., 2017) and Monte Carlo dropout (Gal and Ghahramani, 2016) — but notes that full ensembles of LLMs are prohibitively expensive. The key innovation is adapting LoRA ensembles (Mühlematter et al., 2024) for this purpose: rather than training multiple full copies of the model, train multiple independent LoRA adapters on a shared frozen backbone. This recovers much of the uncertainty quantification benefit of deep ensembles at a fraction of the parameter count (a few million parameters per adapter rather than billions per full model). The paper's t-SNE visualizations (Figure 1) provide qualitative evidence that these LoRA ensemble members indeed follow non-degenerate, diverse trajectories in "gradient-profile space" — they are not merely noisy copies but genuinely reflect different plausible parameter updates around the frozen backbone.

How This Paper Positions Itself

GRADFILTERING occupies a specific niche in the data selection landscape that the paper defines along three axes:

1. Gradient-based vs. score-based. Gradient-based methods (influence functions, TracIn, LESS) use parameter-space information — how training examples pull the model's weights — to assess utility. Score-based methods (Superfiltering, perplexity filtering, reward model scoring) use output-space information — the proxy model's likelihood or a quality score — to assess utility. GRADFILTERING is firmly in the gradient-based camp but distinguishes itself from prior gradient-based work by not requiring a target validation set. Instead, it extracts the utility signal from the training dynamics themselves: early-to-late gradient drop and ensemble disagreement are properties of how the model learns from each example during training, not properties of how well the example aligns with a downstream task.

2. Targeted vs. objective-agnostic. Targeted methods (LESS, influence functions) select data to optimize performance on a specific validation set. Objective-agnostic methods (Superfiltering, random sampling) select data based on properties of the training data alone, without reference to downstream tasks. GRADFILTERING is explicitly objective-agnostic: G-SNR is computed purely from per-example gradients under the training objective (next-token prediction loss), with no dependence on task-specific rewards, preference labels, or validation examples. The paper argues this is a practical advantage because it means GRADFILTERING can curate a single dataset that works well across a broad distribution of instructions, rather than requiring per-task re-selection.

3. Static vs. dynamic. Static methods (Superfiltering) assign each example a single score before training begins. Dynamic methods (LESS, influence functions, GRADFILTERING) use information that emerges during training — gradient similarities, optimization trajectories, or ensemble disagreement. GRADFILTERING is dynamic in a specific, lightweight way: it runs a short training process (2 epochs) on a small proxy model (GPT-2) to collect gradient statistics at two snapshots, then computes G-SNR from these snapshots. This is more expensive than a pure forward-pass method like IFD but substantially cheaper than full training-trajectory methods that track gradients at every optimization step, and cheaper than LESS which requires computing gradients of the full target model.

The paper also draws an implicit analogy to dataset cartography (Swayamdipta et al., 2020), a method from the NLP interpretability literature that visualizes training examples in a 2D space defined by confidence (mean model probability of the true label across epochs) and variability (standard deviation of that probability across epochs). Examples that are high-confidence and low-variability are "easy-to-learn" and can be pruned; low-confidence and low-variability are "hard-to-learn" and should be retained; low-confidence and high-variability are "ambiguous" and often correspond to labeling errors. GRADFILTERING can be viewed as a gradient-based analogue of this idea: instead of per-example loss statistics, it tracks per-example gradient norm statistics, where the gradient drop captures a form of "learnability" (G-SNR's first factor) and the ensemble variance captures "ambiguity" (G-SNR's second factor). This connection is not made explicit in the paper but provides useful context for the G-SNR formulation: it maps the well-validated intuition that training dynamics reveal data quality into the gradient domain, where it can be computed once from a small proxy and then transferred to the target model.

The Practical Significance of "Objective-Agnostic" Selection

One claim the paper makes repeatedly — in the abstract, Section 6, and the conclusion — is that GRADFILTERING is "objective-agnostic." This is more than a rhetorical point; it addresses a real deployment challenge in instruction tuning. Consider a practical scenario: a company wants to fine-tune LLaMA-2-7B to handle customer support queries for multiple product lines. Each product line has different vocabulary, different failure modes, and different evaluation criteria (e.g., billing accuracy for one, technical troubleshooting for another). If the company uses LESS, they would need to construct a validation set for each product line and run selection separately — or construct an aggregate validation set that risks over-representing some product lines and under-representing others. If they use Superfiltering, the IFD scores might systematically undervalue examples with product-specific terminology (which the GPT-2 proxy finds confusing) even though those examples are critical for domain adaptation.

GRADFILTERING's objective-agnosticism sidesteps these issues: it ranks training examples based on how consistently they drive learning in parameter space, regardless of what is being learned. A customer support example about billing reconciliation and one about password reset procedures would both be evaluated by how their gradients evolve during the proxy model's fine-tuning — both would score highly if they produce large, consistent gradient drops, regardless of their content. The paper doesn't evaluate this multi-task scenario directly, but the principle is that G-SNR should transfer across tasks because it measures how teachable an example is, not what it teaches.

This connects to a deeper idea: G-SNR may be capturing fundamental properties of instruction–response pairs that make them effective training signals, independent of the specific target task. These might include:

Clear causal relationship between instruction and response: The instruction genuinely constrains the appropriate response, so the model learns a meaningful mapping rather than memorizing surface patterns.
Appropriate difficulty: The example is challenging enough to produce a non-trivial gradient but not so hard that the model cannot improve — the "zone of proximal development" for the proxy model.
Consistency: The instruction–response pair is internally coherent and unambiguous, so different model initializations agree on what should be learned from it.

If these properties correlate with downstream instruction-following quality — and the paper's empirical results suggest they do — then G-SNR provides a principled, general-purpose filter that could be applied to any instruction dataset without task-specific calibration.

3. Technical Approach

3.1 Reader Orientation

GRADFILTERING is a data selection pipeline that runs a short, cheap fine-tuning process on a small GPT-2 proxy model with multiple independent LoRA adapters, collects per-example gradient statistics across ensemble members and training epochs, and then aggregates those statistics into a single Gradient Signal-to-Noise Ratio (G-SNR) utility score for each training example — the top-scoring examples are then used to fine-tune the target large language model. The system solves the problem of identifying which instruction–response pairs actually drive useful learning without requiring a task-specific validation set, external reward models, or expensive gradient computations on the target model itself, by exploiting the insight that examples which produce large, consistent gradient updates that decay steadily over training are likely to be the most informative for instruction following.

3.2 Big-Picture Architecture (Diagram in Words)

The GRADFILTERING pipeline has four sequential phases, illustrated in Figure 2:

Phase 1 — LoRA-Ensemble Training: A frozen GPT-2 backbone is augmented with $M$ independent LoRA adapters (each initialized differently), and this ensemble is fine-tuned on the full instruction dataset $\mathcal{D}$ for a small number of epochs ( $T=2$ ). Each ensemble member $m$ produces a separate parameter trajectory $\Delta\theta^{(m)}_e$ at each epoch $e$ .
Phase 2 — Dynamic Gradient Collection: During the Phase 1 training process, for every training example $i$ , every ensemble member $m$ , and every epoch $e$ , the system computes and stores the per-example LoRA gradient $g^{(m,e)}_i$ — the gradient of the token-level training loss on example $i$ with respect to the LoRA adapter parameters of member $m$ at epoch $e$ . This produces a collection of gradient profiles $\{g^{(m,e)}_i\}_{m=1}^M$ at each epoch for each example.
Phase 3 — Uncertainty-Aware Utility Scoring: The collected gradient profiles are aggregated into a single scalar utility $u_i$ for each training example via the Gradient Signal-to-Noise Ratio (G-SNR) formula. This combines two terms: (a) a relative gradient drop $\Delta G_i / G^{(s)}_i$ measuring how much the ensemble-averaged gradient norm decreases from early epoch $s$ to late epoch $t$ , normalized by the initial magnitude to remove scale effects; and (b) an uncertainty penalty $1 / V^{(t)}_i$ that down-weights examples whose gradient norms have high variance across ensemble members at the late epoch, indicating persistent disagreement and potential noise. Examples are ranked by $u_i$ , and the top- $\alpha$ fraction are selected.
Phase 4 — Targeted Fine-Tuning: The selected subset $\mathcal{D}' \subset \mathcal{D}$ (e.g., the top 5%, 10%, or 15% of examples by G-SNR score) is used to fine-tune the target large language model (e.g., LLaMA-2-7B or LLaMA-2-13B) via either LoRA-based adaptation or full-parameter fine-tuning, using the standard instruction-tuning objective with no modifications to the training procedure.

Information flows linearly: full dataset → proxy ensemble training with gradient collection → G-SNR scoring and ranking → subset selection → target model training. The proxy model (GPT-2) and the target model (LLaMA-2) are completely decoupled — the proxy is only used for data valuation; the target model sees only the selected subset and never interacts with the proxy during its own fine-tuning.

3.3 Roadmap for the Deep Dive

First, the LoRA ensemble architecture — how multiple adapters are attached to a frozen GPT-2 backbone, how they are trained, and why this specific setup produces meaningful epistemic uncertainty signals. This is the foundation that makes all subsequent gradient statistics interpretable.
Second, the per-example gradient collection protocol — what exactly is computed, stored, and at which epochs, including the definition of per-example LoRA gradients and the justification for the early/late snapshot scheme ( $s=1$ , $t=2$ ).
Third, the ensemble-averaged gradient magnitude $G^{(e)}_i$ and the ensemble gradient variance $V^{(e)}_i$ — how these two summary statistics are derived from the raw per-member gradients, and what each captures about training dynamics.
Fourth, the G-SNR utility formula — the complete equation combining gradient drop and uncertainty penalty, with a detailed walkthrough of each factor, why the ratio form is chosen over the raw drop, why variance is in the denominator, and what the $\epsilon$ constants protect against.
Fifth, the selection protocol and design choices — how utilities are used to select subsets, why top- $\alpha$ ranking rather than thresholding, and the practical hyperparameter settings (ensemble size $M=5$ , epochs $T=2$ , LoRA rank $r=8$ , etc.).

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a methods paper that proposes a specific data scoring formula (G-SNR) and a pipeline for computing it efficiently. The core idea is that training dynamics — specifically, how per-example gradient norms change between early and late epochs, and how much ensemble members disagree about those norms at late epochs — provide a sufficient signal to rank the utility of instruction-tuning examples without any task-specific validation data.

The LoRA Ensemble Architecture

GRADFILTERING's ability to quantify uncertainty in gradient signals depends on having multiple independent trajectories through parameter space during fine-tuning. Training multiple full copies of even a small model like GPT-2 would be expensive (each copy has ~124M parameters for the smallest GPT-2 variant), so the paper adopts a LoRA ensemble: multiple independent low-rank adapters attached to a single shared, frozen backbone.

How the ensemble is constructed. Let $\theta_0$ denote the frozen parameters of the GPT-2 backbone. For each ensemble member $m \in \{1, \ldots, M\}$ , the paper attaches a separate set of LoRA adapters $\Delta\theta^{(m)}$ to the query, key, and value projection matrices of each self-attention layer, following the standard LoRA formulation (Hu et al., 2022). Each adapter is a low-rank decomposition $A^{(m)} B^{(m)}$ of rank $r$ , initialized independently for each member. The full parameter vector for member $m$ at epoch $e$ is:

$\theta^{(m)}_e = \theta_0 + \Delta\theta^{(m)}_e$

where $\theta_0$ is the frozen pretrained GPT-2 weights, and $\Delta\theta^{(m)}_e$ are the LoRA adapter parameters for member $m$ after $e$ epochs of fine-tuning. All members share the same $\theta_0$ but have their own $\Delta\theta^{(m)}_e$ that follow independent trajectories because they are initialized differently (different random seeds) and see different data orderings.

What it computes: each $\theta^{(m)}_e$ is a full set of model parameters representing one member's adapted state at epoch $e$ , but only the LoRA portion $\Delta\theta^{(m)}_e$ (a few million parameters per member) is trainable and differs across members. The frozen backbone $\theta_0$ (124M parameters) is shared and computed once per forward pass.

Why this form: full deep ensembles would require training $M$ complete copies of the model, multiplying memory and compute by $M$ . LoRA ensembles share the backbone computation across members during the forward pass (the self-attention keys, queries, and values for the shared backbone are computed once and reused), and only the adapter-specific computations ( $A^{(m)} B^{(m)}$ projections) are repeated per member. The paper's vectorized implementation computes all members' outputs in a single forward pass by stacking the adapter computations, making the ensemble roughly $M$ times more expensive in adapter parameters but substantially less than $M$ times more expensive in total FLOPs compared to a single LoRA model. The paper cites Mühlematter et al. (2024) and Balabanov and Linander (2024) for the theoretical justification: independently initialized LoRA adapters can be interpreted as approximate samples from a posterior distribution over low-rank updates around the pretrained backbone, providing a principled basis for epistemic uncertainty quantification.

Training objective. The ensemble is trained by minimizing the average per-member loss on each batch, not the loss of an averaged prediction:

$\mathcal{L}_{\text{ens}} = \frac{1}{M} \sum_{m=1}^{M} \mathcal{L}\left(f_{\theta^{(m)}_e}(x_i), y_i\right)$

where $f_{\theta^{(m)}_e}(x_i)$ is the output logits of ensemble member $m$ on input $x_i$ , and $\mathcal{L}$ is the standard next-token prediction cross-entropy loss. The gradient of this objective is backpropagated with respect to each adapter's parameters independently.

Why this form and not the alternative: if the system instead optimized $\mathcal{L}\left(\frac{1}{M}\sum_m f_{\theta^{(m)}_e}(x_i), y_i\right)$ — the loss of the ensemble's average prediction — the optimization would explicitly encourage all members to correct the same ensemble-level errors, driving their predictions toward consensus and suppressing the very epistemic disagreement that the gradient-variance statistics are designed to capture. By optimizing the average of per-member losses, each member is trained independently to fit the data from its own initialization, allowing them to develop distinct "opinions" about how to update the backbone. The paper is explicit about this design choice in Section 3.1: "Crucially, we do not optimize the loss of an averaged prediction ... which would explicitly encourage all members to correct the same ensemble error and thus suppress the epistemic disagreement that our gradient-variance statistics are intended to capture."

Practical hyperparameters. The paper sets $M=5$ (ensemble size), LoRA rank $r=8$ , and LoRA alpha $\alpha=16$ (the scaling factor for the LoRA update). Training uses the Adam optimizer with learning rate $5 \times 10^{-5}$ . The backbone is GPT-2 — the paper doesn't specify which GPT-2 size, but standard GPT-2 variants range from 124M (GPT-2 small) to 1.5B (GPT-2 XL) parameters; given the emphasis on efficiency and the proxy being "small," GPT-2 small (124M) is the most likely choice, though the exact variant is not explicitly stated.

Epistemic Justification: Why a LoRA Ensemble Captures Uncertainty

The paper provides both theoretical and empirical justification for why the LoRA ensemble's gradient statistics are meaningful proxies for epistemic uncertainty. This justification is necessary because G-SNR's effectiveness depends on the assumption that ensemble disagreement signals genuine ambiguity about an example's utility, not just random noise from initialization.

Theoretical justification via local linearity. The paper appeals to two structural properties of LoRA fine-tuning (Section 3.1, "Gradient-based view of LoRA ensembles"):

First, empirical studies (the paper cites Li et al., 2025) suggest that LoRA fine-tuning typically keeps the adapted model close to the pretrained backbone in parameter space — the low-rank constraint and small learning rates mean the adapters explore a compact region around $\theta_0$ . In this locally linear regime, a first-order Taylor expansion around $\theta_0$ provides an accurate linearization of the model's outputs and losses as a function of the adapter parameters. Under this linearization, the per-example gradient $g^{(m,e)}_i$ (the gradient of the loss on example $i$ with respect to the adapter parameters of member $m$ at epoch $e$ ) is the direction of steepest loss change for that example under that member's current parameterization. Its norm $\|g^{(m,e)}_i\|_2$ reflects how strongly example $(x_i, y_i)$ would push the parameters to change if used for an update at that point.

Second, independently initialized LoRA adapters can be viewed as approximate samples from a posterior distribution over low-rank updates around the frozen backbone (the paper cites Mühlematter et al., 2024 and Balabanov and Linander, 2024 for this interpretation). Under this Bayesian view, the collection $\{g^{(m,e)}_i\}_{m=1}^{M}$ across ensemble members at epoch $e$ approximates samples from a distribution over update directions induced by example $i$ at that stage of training. The ensemble mean $G^{(e)}_i$ estimates the expected gradient magnitude; the ensemble variance $V^{(e)}_i$ quantifies epistemic uncertainty about how the model should adapt to this specific example.

Empirical justification via t-SNE trajectories (Figure 1). The paper provides qualitative evidence that the LoRA ensemble members follow meaningfully diverse trajectories in what it calls "gradient-profile space." The procedure for generating Figure 1 works as follows:

For each ensemble member $m$ and each epoch $e$ , construct a high-dimensional feature vector $\mathbf{g}_{m,e} \in \mathbb{R}^N$ whose $i$ -th coordinate encodes the relative gradient drop on example $i$ between epoch 1 and epoch $e$ . This is a global descriptor of how the member's gradient pattern has shifted for all training examples simultaneously, not a per-example trajectory. The paper then applies PCA to reduce dimensionality from $N$ (52,000 — the number of training examples) to $d=25$ for denoising, then runs t-SNE on the PCA embeddings to get 2D points $\mathbf{z}_{m,e} \in \mathbb{R}^2$ . Each member's points across epochs are connected into a polyline $\mathbf{z}_{m,1} \to \mathbf{z}_{m,2} \to \cdots \to \mathbf{z}_{m,T}$ .

The empirical patterns visible in Figure 1 are:

Early divergence: All members start from a similar region at epoch 1 (their initial gradient profiles are similar because they are all initialized near $\theta_0$ ), but their trajectories quickly diverge into distinct regions of the embedding space. This indicates that different initializations lead to meaningfully different update patterns — the ensemble is not a degenerate set of noisy copies.
Stable, non-collapsing trajectories: After the initial divergence, each member follows a stable trajectory that does not collapse toward a common attractor. This suggests that the members settle into distinct parameter-space neighborhoods rather than converging to a single mode, providing genuine diversity for uncertainty estimation.
Clustering at later epochs: Later-epoch points (epoch 5 and beyond) are visibly more tightly clustered than early-epoch points (especially epoch 1), particularly for the Alpaca dataset (Figure 1a). This indicates that ensemble disagreement is gradually reduced as training proceeds — members converge in their assessment of which examples are informative, even if they follow different paths to get there. This is exactly the behavior that makes late-epoch variance a useful noise detector: examples where disagreement persists at late epochs are genuinely ambiguous.

The paper explicitly states that these patterns "provide qualitative evidence that (i) LoRA adapters induce a rich, structured distribution over update directions around the frozen backbone, and (ii) the mean and variance of per-example gradients across members capture non-degenerate epistemic uncertainty, thereby justifying our use of these statistics for data valuation."

Practical Choices for the Proxy Training Protocol

The paper makes several specific design choices about how the proxy ensemble is trained, each motivated by empirical observation or practical constraints:

Choice of proxy model: GPT-2 rather than the target LLaMA-2. The proxy model must be small enough that training an ensemble of $M=5$ members for 2 epochs is substantially cheaper than directly computing gradient features on the target model (as LESS does). GPT-2 (likely GPT-2 small, 124M parameters) with LoRA adapters of rank 8 adds only a few million trainable parameters per member, making the entire ensemble far cheaper than even a single forward pass through the target LLaMA-2-7B model for gradient computation. The paper also relies on the finding from Superfiltering (Li et al., 2024) that GPT-2's judgments of data quality correlate well with LLaMA-2's, providing a basis for the weak-to-strong transfer.

Choice of training epochs: $T=2$ rather than longer training. The paper observes in Figure 1 that "the trajectories of the LoRA ensemble members change substantially between epochs 1 and 2, and then remain relatively stable afterwards." This means the most informative gradient dynamics — the early-to-late transition that captures how much the model learns from each example — are concentrated in the first two epochs. Training longer would increase computational cost without providing substantially more information for the G-SNR calculation. The paper explicitly states: "Motivated by this observation, we set the terminal epoch to be $T=2$ when computing gradient statistics, which further reduces computational cost while preserving the essential training-dynamics signal."

Choice of early and late snapshots: $s=1$ and $t=2$ . The gradient-drop term in G-SNR uses epoch 1 as the early reference and epoch 2 as the late reference (the paper refers to these as $s$ and $t$ but doesn't make the assignment explicit in the main text — it is implicit from the $T=2$ setting). Epoch 1 represents the initial state where the model has seen each example exactly once under its own initialization; epoch 2 represents the state after a second pass, where learning progress has occurred for genuinely informative examples but not for noise. Using epoch 0 (pre-training) is not considered, likely because the pretrained GPT-2 has not been exposed to the instruction format and would produce uninformatively high gradients for nearly all examples due to format mismatch rather than content difficulty.

Ensemble size $M=5$ . This is a pragmatic choice balancing diversity and cost. Five members provide enough independent trajectories to estimate mean and variance with reasonable stability, while keeping the ensemble training cost manageable. The paper does not ablate ensemble size (which would be a natural ablation study), so the sensitivity to this choice is unknown.

Per-Example Gradient Collection

The core data that GRADFILTERING collects is the set of per-example LoRA gradients for every training example, every ensemble member, and every training epoch. This is the raw material from which all subsequent utility statistics are derived.

What is collected. For each example $i$ (instruction–response pair $(x_i, y_i)$ ), each ensemble member $m \in \{1, \ldots, M\}$ , and each epoch $e \in \{1, \ldots, T\}$ , the system computes:

$g^{(m,e)}_i = \nabla_{\Delta\theta^{(m)}_e} \, \mathcal{L}\left(f_{\theta^{(m)}_e}(x_i), y_i\right)$

where $\mathcal{L}$ is the token-level training loss (standard next-token prediction cross-entropy), $f_{\theta^{(m)}_e}(x_i)$ is the member $m$ 's output logits for input $x_i$ , and the gradient is taken only with respect to the LoRA adapter parameters $\Delta\theta^{(m)}_e$ , not the frozen backbone $\theta_0$ . The paper explicitly notes that $g^{(m,e)}_i$ is the gradient of the loss evaluated at the member's current parameters $\theta^{(m)}_e$ , not the gradient that was actually used to update the model (which would be averaged over a minibatch).

What it captures. $g^{(m,e)}_i$ measures how strongly example $i$ would push the adapter parameters of member $m$ to change if it were used alone for an update at epoch $e$ . A large gradient norm means the example is challenging for that member at that stage — the model's current prediction is far from the target, and correcting it requires a substantial parameter update. A small gradient norm means the model already handles this example well — the loss is low and the parameters are near a local optimum for this example.

Why collect gradients and not just losses. Loss values alone conflate two sources of difficulty: an example might have high loss because it is genuinely informative (teaching the model something new) or because it is noisy/ill-formed (impossible to predict well). Gradient norms provide richer information because they reflect not just prediction error but also how that error interacts with the model's current parameterization — the gradient direction indicates how the model would need to change to improve, and examples that demand large, consistent parameter changes across ensemble members are more likely to be genuinely informative than examples that demand large but inconsistent updates.

Why collect per-example gradients rather than per-minibatch gradients. Standard training only computes gradients averaged over minibatches (for the actual parameter update), but per-example gradients provide example-level resolution necessary for selection. The paper doesn't detail the implementation, but computing per-example gradients typically requires either (a) running each example through the model individually (no minibatch), which is slow but exact, or (b) using gradient accumulation techniques that save per-example gradients before averaging. Given the small GPT-2 proxy, option (a) is feasible and is likely what the paper implements.

Storage requirements. For a dataset of 52,000 examples, $M=5$ members, and $T=2$ epochs, the system stores $52,000 \times 5 \times 2 = 520,000$ per-example gradient vectors. However, the paper's utility formulas only use gradient norms ( $\|g^{(m,e)}_i\|_2$ ), not the full gradient vectors, for the G-SNR computation. This means only scalars need to be stored long-term — the full gradients are computed, their norms are extracted and stored, and the gradients themselves can be discarded. This is a critical practical detail: storing 520,000 full LoRA gradient vectors (each with potentially millions of entries) would be prohibitively expensive; storing 520,000 scalars is trivial. The paper doesn't make this explicit, but equations 3–5 only reference gradient norms, confirming that the full vectors are not needed after norm extraction.

Why gradients with respect to only LoRA parameters and not the full model. This is a key efficiency consideration. The LoRA adapter parameters $\Delta\theta$ are a small fraction of the total parameters (for rank-8 LoRA on GPT-2 small, the adapter might have ~0.3M parameters vs. 124M frozen backbone parameters). Computing and storing gradients with respect to only these few parameters is much cheaper than full-model gradients. Additionally, because LoRA adapters are the only part of the model that changes during fine-tuning, their gradients capture the entirety of the training signal — the frozen backbone's parameters don't move, so their gradients (while theoretically computable) don't provide additional information about learning dynamics. The paper explicitly states the gradients are taken "only with respect to the LoRA adapters $\Delta\theta^{(m)}_e$ " (Equation 2).

Ensemble-Averaged Gradient Magnitude $G^{(e)}_i$

From the per-member per-example gradient norms, the paper computes a single ensemble-averaged gradient magnitude for each example at each epoch:

$G^{(e)}_i = \frac{1}{M} \sum_{m=1}^{M} \left\| g^{(m,e)}_i \right\|_2$

where $M$ is the number of ensemble members ( $M=5$ ), $g^{(m,e)}_i$ is the per-example gradient of member $m$ at epoch $e$ for example $i$ , and $\|\cdot\|_2$ is the Euclidean (L2) norm of the gradient vector.

What it computes: for a given example $i$ and epoch $e$ , the system takes the raw gradient vector for each ensemble member, computes its L2 norm (a single non-negative scalar representing the magnitude of the gradient), and averages these scalars across the $M$ members. The result is a single scalar $G^{(e)}_i$ that represents the expected strength of the training signal that example $i$ produces at epoch $e$ , averaged over the ensemble's uncertainty about the correct parameterization.

Why average over ensemble members: individual members may have idiosyncratic gradient responses due to their specific initialization and training trajectory. A single member might find an example trivial (small gradient norm) while another finds it challenging (large gradient norm). Averaging smooths out these member-specific variations and provides an estimate of the example's typical difficulty across plausible model states. This is the "signal" that the gradient-drop computation uses as its input.

Physical interpretation: $G^{(e)}_i$ can be understood as measuring how much the average ensemble member's parameters would change if optimized on example $i$ alone at epoch $e$ . A high value means the example is currently challenging for the ensemble — the loss is high and the gradient is steep. A low value means the example is currently well-learned — the loss is low and the gradient is shallow.

Why L2 norm and not another norm: the L2 norm is the standard measure of gradient magnitude in optimization theory, corresponding to the Euclidean length of the steepest descent direction. It is rotationally invariant (unlike L1) and smoothly differentiable (unlike L∞), making it a natural choice for summarizing the "size" of a gradient update. The paper does not justify this choice explicitly, but it is standard practice in gradient-based optimization analysis.

Ensemble Disagreement via Gradient Variance $V^{(e)}_i$

The paper quantifies epistemic uncertainty through the variance of gradient norms across ensemble members at each epoch:

$V^{(e)}_i = \frac{1}{M} \sum_{m=1}^{M} \left\| g^{(m,e)}_i \right\|_2^2 - \left( \frac{1}{M} \sum_{m=1}^{M} \left\| g^{(m,e)}_i \right\|_2 \right)^2$

where the first term is the mean squared gradient norm (average of $\|g^{(m,e)}_i\|_2^2$ across members), and the second term is the square of the mean gradient norm $(G^{(e)}_i)^2$ . This is the standard definition of variance: the expected squared deviation from the mean.

What it computes: $V^{(e)}_i$ measures how much the ensemble members disagree about the gradient magnitude of example $i$ at epoch $e$ . If all members produce similar gradient norms, $V^{(e)}_i$ is close to zero — the example's gradient signal is consistent across the ensemble. If some members produce large gradients and others produce small ones, $V^{(e)}_i$ is large — there is substantial uncertainty about how strongly the model should update for this example.

Physical interpretation of high variance: a high $V^{(e)}_i$ at late epochs indicates that, even after seeing the full training data twice, different ensemble members still disagree about how challenging this example is. This could arise from several sources:

Label noise or ambiguity: the instruction–response pair is inconsistent or ill-formed, so different model initializations interpret it differently — some try hard to fit it (large gradient), others learn to ignore it (small gradient).
Out-of-distribution character: the example uses vocabulary or patterns that the GPT-2 proxy handles unreliably, so its gradient response is sensitive to the specific adapter initialization.
Genuine difficulty with multiple valid approaches: the example teaches a skill that can be learned in different ways (e.g., different reasoning chains could lead to the same correct answer), and ensemble members have converged to different strategies.

Physical interpretation of low variance: a low $V^{(e)}_i$ indicates consensus — all ensemble members agree about how much this example matters. If the mean gradient $G^{(e)}_i$ is also small (as expected at late epochs for well-learned examples), this consensus means the example has been stably acquired. If $G^{(e)}_i$ is large and variance is low (unusual at late epochs), it means the example remains challenging for all members in the same way — possibly an inherently difficult but clean example.

Why variance of norms rather than variance of gradient directions: the paper's G-SNR formula only uses the magnitude of disagreement (norm variance), not the direction of disagreement (cosine similarity of gradient vectors). This is a simplification that reduces computation and storage — only scalar norms need to be stored rather than full gradient vectors for variance computation. However, it also means that two ensemble members could have identical gradient norms but orthogonal gradient directions (they want to update the model in completely different ways), and $V^{(e)}_i$ would be zero despite genuine epistemic disagreement about how to learn from the example. The paper acknowledges this limitation in Section 7 (Limitations): "it operates on gradient norms and their variance, ignoring gradient direction; examples that are locally uninformative in norm but crucial for aligning with rare or long-horizon behaviors may be under-valued." This is a deliberate trade-off: full gradient-vector variance would be more expressive but substantially more expensive to compute and store.

Why variance rather than standard deviation: variance ( $\sigma^2$ ) and standard deviation ( $\sigma$ ) are monotonic transformations of each other and would produce identical rankings. The paper uses variance because it appears naturally in the derivation (mean squared minus squared mean) and avoids the square root operation.

The G-SNR Utility Formula

This is the core algorithmic contribution of the paper. The G-SNR utility combines the gradient dynamics statistics into a single scalar score per example:

$u^{\text{G-SNR}}_i = \frac{G^{(s)}_i - G^{(t)}_i}{G^{(s)}_i + \epsilon} \cdot \frac{1}{V^{(t)}_i + \epsilon}$

where $s$ is the early epoch (epoch 1), $t$ is the late epoch (epoch 2, given $T=2$ ), $G^{(s)}_i$ is the ensemble-averaged gradient norm at the early epoch, $G^{(t)}_i$ is the ensemble-averaged gradient norm at the late epoch, $V^{(t)}_i$ is the ensemble gradient variance at the late epoch, and $\epsilon$ is a small positive constant for numerical stability (preventing division by zero).

What it computes — operational walkthrough:

For each training example $i$ :

Compute the raw gradient drop: $\Delta_i = G^{(1)}_i - G^{(2)}_i$ . This is the absolute decrease in ensemble-averaged gradient norm from epoch 1 to epoch 2. A large positive value means the example induced strong updates initially (high $G^{(1)}_i$ ) but became much easier after one epoch of training (low $G^{(2)}_i$ ) — this is the signature of a genuinely informative example that the model successfully learned from. A small or negative value means the gradient didn't decrease (the example remained challenging or became more challenging) — this could indicate noise, an intrinsically hard example beyond the proxy's capability, or an example that interacts poorly with other training data.
Normalize by initial magnitude: Divide the raw drop by $G^{(1)}_i + \epsilon$ to get the relative gradient drop. This normalization is critical: without it, examples with large absolute gradients (e.g., long responses with many tokens, or examples with rare vocabulary) would dominate the ranking simply because their raw gradient norms are larger, not because they are more informative. The relative drop measures what fraction of the initial learning signal has been resolved — an example whose gradient drops from 10.0 to 2.0 (80% reduction) and one whose gradient drops from 1.0 to 0.2 (also 80% reduction) would have the same relative drop, even though their absolute drops differ by a factor of 10.
Penalize by late-epoch variance: Multiply by the reciprocal of $V^{(2)}_i + \epsilon$ . This down-weights examples where ensemble members disagree about the gradient magnitude at the late epoch. High variance means the gradient drop signal (step 2) is unreliable — different members would compute different drops because their late-epoch gradient norms differ. The reciprocal $1/V^{(2)}_i$ maps high variance to low utility and low variance to high utility, operating as a confidence weight on the gradient drop signal.

The final score $u^{\text{G-SNR}}_i$ is high for examples that (a) showed a large relative reduction in gradient norm and (b) have low ensemble disagreement about that reduction. It is low for examples that showed little reduction, had high disagreement, or both.

Why this form — the three factors analyzed individually:

Factor 1: $\Delta G_i = G^{(s)}_i - G^{(t)}_i$ (raw gradient drop). This measures the absolute learning progress on example $i$ . The intuition is that informative examples follow a characteristic trajectory: they start with high loss/high gradient because the initial model (trained from scratch on the adapters) hasn't learned the instruction–response mapping yet, but after one epoch of training they become substantially easier because the model has extracted useful signal. Noise, in contrast, may have high initial gradient (hard to predict) but the gradient doesn't decrease because it's genuinely unpredictable — no amount of training on other examples helps the model fit this noisy example. Redundant examples may have low gradients throughout because they're similar to many other examples and the model learns them quickly from those others. Trivial examples may have low initial gradients and remain low.

The paper acknowledges that the raw drop $\Delta G_i$ can be negative — if $G^{(2)}_i > G^{(1)}_i$ , meaning the example became more challenging after one epoch. This can happen if the model initially memorized surface patterns that broke down on the second pass, or if the example interacts negatively with other training data. Negative-drop examples are pushed to the bottom of the ranking, which the paper argues is desirable: "Empirically, downweighting highly negative utilities is desirable, as they often correlate with examples whose gradients do not exhibit consistent improvement over training."

Factor 2: $1 / (G^{(s)}_i + \epsilon)$ (scale normalization). Without normalization, the utility would be $\Delta G_i \cdot 1/(V^{(t)}_i + \epsilon)$ . This would systematically favor examples with large absolute gradient norms — long responses, examples with rare tokens, or any example that happens to produce large loss values for reasons unrelated to informativeness. The paper's ablation (Table 2, Alternative #1) confirms that using raw gradient drop ( $G_1 - G_T$ ) alone performs substantially worse than the normalized version across most settings. For instance, on Alpaca-GPT4 with LLaMA2-7B (LoRA, 5%), the raw-drop utility loses 0.44 in pairwise winning score relative to the full G-SNR (Table 2, Utility Alternative #1, LoRA, 5% column).

The normalization converts the absolute drop into a fractional improvement: an example whose gradient drops from 10.0 to 2.0 has a relative drop of $(10.0 - 2.0) / 10.0 = 0.80$ , while one that drops from 1.0 to 0.2 has the same relative drop. This makes the utility comparable across examples of different intrinsic difficulty or length, focusing on the efficiency of learning rather than the magnitude of the initial challenge.

Factor 3: $1 / (V^{(t)}_i + \epsilon)$ (uncertainty penalty). This factor embodies the "uncertainty-aware" aspect of the method. If the late-epoch variance $V^{(2)}_i$ is high, it means the ensemble hasn't reached consensus on how much this example matters — some members found it informative while others didn't, and this disagreement persists even after two full passes through the data. Such examples are unreliable as training data: they might be noisy, out-of-distribution for the proxy, or ambiguous in ways that make the gradient-drop signal untrustworthy.

The reciprocal mapping $1/V^{(t)}_i$ means that as variance increases, utility decreases hyperbolically — small increases in variance for already-low-variance examples produce small utility changes, but the same absolute increase in variance for moderate-variance examples produces larger utility drops. This has the effect of creating a soft threshold: examples with very low variance are treated as high-confidence and their utility is dominated by the gradient-drop term; examples with moderate-to-high variance are progressively penalized out of the top ranks.

The paper's ablation (Table 2, Alternative #3) tests using only the variance-normalized drop without the scale normalization — $(G_1 - G_T) / (V_T + \epsilon)$ — and finds it also underperforms the full G-SNR, typically by smaller margins than the raw drop but still consistently negative. For instance, on Alpaca LLaMA2-13B (Full, 10%), Alternative #3 loses 0.10 in PWS relative to G-SNR (Table 2). This confirms that both normalization factors — scale normalization by $G^{(s)}_i$ and uncertainty normalization by $V^{(t)}_i$ — contribute independently to the utility's effectiveness.

The role of $\epsilon$ . The paper includes small constants $\epsilon$ in the denominator of both factors "for numerical stability." The primary concern is division by zero: if $G^{(s)}_i = 0$ (an example that produces zero initial gradient — theoretically possible if the model already assigns zero loss to it, though unlikely in practice), the first factor would be undefined without $\epsilon$ . Similarly, if $V^{(t)}_i = 0$ (perfect agreement across ensemble members), the second factor would be undefined. The paper does not specify the value of $\epsilon$ , but typical choices for such stability constants in deep learning are $10^{-8}$ or $10^{-6}$ . The choice matters mainly for corner cases; for typical examples with non-zero gradients and non-zero variance, $\epsilon$ is negligible.

The signal-to-noise analogy. The paper explicitly names G-SNR as a "Gradient Signal-to-Noise Ratio" and states that it "behaves like a signal-to-noise ratio: it prefers large, consistent gradient drops (signal) while suppressing examples with high ensemble disagreement (noise)." This analogy is apt but not literal — in classical signal processing, SNR is $P_{\text{signal}} / P_{\text{noise}}$ , a ratio of powers. G-SNR is a product of two ratios: a normalized gradient drop (signal fraction) times an inverse variance (inverse noise). The result is high when signal is high AND noise is low, which captures the SNR intuition without being formally a single ratio.

Why not a simpler form? The paper's ablation (Table 2) systematically tests three simpler alternatives:

Alternative #1 (raw drop): $u_i = G^{(1)}_i - G^{(2)}_i$ . This is the simplest gradient-based utility but performs worst because it ignores both scale normalization (favoring long/hard examples) and uncertainty (not down-weighting noisy examples).
Alternative #2 (relative drop, no uncertainty): $u_i = (G^{(1)}_i - G^{(2)}_i) / (G^{(1)}_i + \epsilon)$ . This normalizes for scale but still treats all gradient drops as equally reliable. It performs better than Alternative #1 but worse than full G-SNR, as shown in Table 2.
Alternative #3 (variance-normalized raw drop): $u_i = (G^{(1)}_i - G^{(2)}_i) / (V^{(2)}_i + \epsilon)$ . This incorporates uncertainty but doesn't normalize for initial scale, so it still favors large-gradient examples.

The consistent underperformance of all three alternatives — shown quantitatively in the negative deltas across Table 2 — provides empirical validation that both the relative normalization and the uncertainty penalty are necessary for robust performance across datasets and model sizes. The full multiplicative form $(\text{relative drop}) \times (\text{inverse variance})$ ensures that an example must score well on both dimensions to receive a high utility.

Negative utilities and the ranking protocol. Because $G^{(1)}_i - G^{(2)}_i$ can be negative (the gradient magnitude increases rather than decreases), G-SNR can produce negative utility scores. The paper notes: "In practice, this simply pushes such examples toward the bottom of the ranking; under our top- $\alpha$ selection rule, only the relative ordering matters." This is important because it means the method does not need to distinguish between "slightly positive" and "slightly negative" examples — both will be far from the top of the ranking and not selected. The cutoff is determined by the selection ratio $\alpha$ , not by a zero threshold on the utility score.

Selection Protocol: From Utilities to Subsets

Once G-SNR utilities are computed for all $N$ training examples, the selection procedure is straightforward:

Ranking: Sort all training examples in descending order by $u^{\text{G-SNR}}_i$ . Examples with the highest G-SNR scores are at the top; examples with negative or near-zero scores are at the bottom.
Top- $\alpha$ selection: Select the top $\alpha \cdot N$ examples as the training subset $\mathcal{D}'$ , where $\alpha$ is the desired selection ratio (0.05, 0.10, or 0.15 in the paper's experiments). For Alpaca and Alpaca-GPT4 with $N=52{,}000$ , this corresponds to 2,600, 5,200, or 7,800 examples respectively.
Target model fine-tuning: Fine-tune the target LLM (LLaMA-2-7B or LLaMA-2-13B) on $\mathcal{D}'$ using the standard instruction-tuning protocol — next-token prediction loss on the response tokens given the instruction prompt. The paper tests both LoRA-based adaptation (low-rank adapters, same architecture as the proxy but on the larger backbone) and full-parameter fine-tuning, keeping all training hyperparameters (learning rate, batch size, number of epochs) identical across data selection strategies for fair comparison.
Cross-strategy comparison: To evaluate GRADFILTERING against baselines, the paper applies the same top- $\alpha$ selection protocol to random sampling (select $\alpha \cdot N$ examples uniformly at random) and Superfiltering (select the top $\alpha \cdot N$ examples by IFD score). All three strategies produce subsets of identical size, trained with identical hyperparameters, to isolate the effect of data selection.

Why top- $\alpha$ rather than threshold-based selection. A natural alternative would be to set a threshold on $u^{\text{G-SNR}}_i$ (e.g., select all examples with $u_i > \tau$ ) and let the subset size vary depending on how many examples exceed the threshold. The paper uses top- $\alpha$ instead because (a) it guarantees a fixed, predictable subset size (important for fair comparison with baselines that also select fixed-size subsets), (b) it avoids the need to calibrate a threshold value that might vary across datasets and proxy models, and (c) it automatically adapts to the distribution of G-SNR scores — the top $\alpha$ fraction always captures the best available examples regardless of the absolute score range.

The objective-agnostic property. The paper emphasizes that GRADFILTERING is "objective-agnostic" because the G-SNR computation uses only the training loss gradients — there is no validation set, no task-specific reward model, no preference labels, and no external quality score. This distinguishes it from LESS (which requires target validation examples for gradient similarity search) and from reward-model-based filtering methods. The practical implication is that a single GRADFILTERING run produces a single ranked dataset that can be used to fine-tune models for any downstream instruction-following task — the curation is done once, and the selected subset is expected to transfer across tasks.

This property relies on the assumption that "being informative for the training objective" (which is what G-SNR measures) correlates with "producing good instruction-following behavior" (which is what we care about). The paper's empirical results in Table 1 support this assumption — models trained on G-SNR-selected subsets outperform random baselines and often match Superfiltering across diverse evaluation prompts from WizardLM and Vicuna datasets. However, the assumption could break down if the training objective (next-token prediction) misaligns with desired downstream behavior in specific ways — for instance, if an example teaches factually incorrect information that is easy for the model to learn (high gradient drop, low variance) but harmful for deployment.

Target Model Fine-Tuning Configuration

The paper keeps the target model training protocol fixed across all data selection strategies to ensure fair comparison. Key hyperparameters:

Models: LLaMA-2-7B and LLaMA-2-13B (Touvron et al., 2023).
Adaptation regimes: LoRA (low-rank adapters on a frozen backbone) and full-parameter fine-tuning. Both are tested for each combination of dataset, model size, and selection ratio.
Optimizer: Adam (Kingma, 2014).
Learning rates: $2 \times 10^{-5}$ for LLaMA-2-7B and $1 \times 10^{-5}$ for LLaMA-2-13B. The paper doesn't specify batch size, number of epochs, or sequence length for the target model training, though these are presumably standardized across strategies within each experiment.
Training objective: Standard next-token prediction cross-entropy loss on the response tokens.

The paper does not modify the target model's training procedure based on which examples were selected — the same hyperparameters are used whether the training data is the full 52,000 examples, a random 10% subset, a Superfiltering-selected 10% subset, or a GRADFILTERING-selected 10% subset. This isolates the effect of data quality from the effect of training hyperparameters: any performance differences must come from which examples were used, not from how they were used.

The Weak-to-Strong Transfer Assumption

GRADFILTERING relies on a critical assumption that is empirically validated but not theoretically guaranteed: that GPT-2's gradient dynamics on the instruction-tuning data correlate with LLaMA-2's eventual instruction-following performance. This is a weak-to-strong transfer assumption, similar in spirit to Superfiltering's finding that GPT-2 IFD scores correlate with LLaMA-2 performance.

The mechanism of transfer is indirect. GRADFILTERING doesn't claim that GPT-2's gradients are similar to LLaMA-2's gradients (they almost certainly are not, given the vast difference in model scale and capability). Rather, it claims that properties of instruction–response pairs that make them effective training signals for the proxy model generalize to the target model. An instruction that is clear, well-formed, appropriately difficult, and internally consistent should be learnable for both GPT-2 and LLaMA-2, even if the absolute difficulty and learning rate differ. Conversely, an instruction that is noisy, ambiguous, or contradictory should be problematic for both, even if the specific failure modes differ.

This assumption is supported by the empirical results in Table 1 but is not formally analyzed. The paper does not, for example, compute the correlation between GPT-2 G-SNR scores and LLaMA-2 per-example loss reduction, which would directly test the transfer assumption. Instead, it relies on downstream evaluation — LLaMA-2 models fine-tuned on G-SNR-selected data perform well — as evidence that the transfer works in practice.

Summary of Key Design Choices

LoRA ensemble on GPT-2 proxy: provides multiple diverse gradient trajectories for uncertainty estimation at low computational cost, leveraging the shared frozen backbone for efficiency.
Gradient norms only: discarding gradient directions simplifies storage and computation to scalar operations, at the cost of potentially missing informative directional disagreement.
Two-epoch training: sufficient to capture the early-to-late gradient transition while minimizing proxy training cost, based on empirical observation that trajectories stabilize after epoch 2.
Relative gradient drop normalized by initial magnitude: removes scale bias so that short and long responses, easy and hard examples, are compared on the basis of learning efficiency rather than absolute difficulty.
Late-epoch variance as uncertainty penalty in the denominator: ensures that only examples where the ensemble reaches consensus on their informativeness receive high utility scores, implementing the "uncertainty-aware" design principle.
Top- $\alpha$ selection: simple, deterministic, size-controlled subset construction that doesn't require threshold tuning and adapts automatically to score distributions.
Fixed target model training protocol: isolates the effect of data selection from confounds like learning rate, batch size, or training duration.

4. Key Insights and Innovations

Innovation 1: Reframing Data Valuation as a Gradient Signal-to-Noise Problem

The dominant paradigm for data selection in instruction tuning — represented by Superfiltering (Li et al., 2024) — treats data quality as an intrinsic, static property of each example: compute a score (perplexity, IFD, reward model rating), rank examples, select the top-k. This framing implicitly assumes that data utility is separable — that example A's value is independent of whether example B is also in the training set, and that a proxy model's snapshot judgment suffices to identify good examples. GRADFILTERING rejects both assumptions.

The paper's core conceptual move is to reframe data valuation as a gradient signal-to-noise ratio (G-SNR) problem, where an example's utility is not a static property but an emergent property of training dynamics — revealed by how gradient magnitudes evolve across ensemble members and epochs. The name itself signals the reframing: this is not "data quality scoring" but "gradient signal processing." The "signal" is the reliable, consensus-driven reduction in gradient norm between early and late training; the "noise" is the persistent ensemble disagreement about an example's importance. An example is valuable if and only if it produces a large, consistent, and decaying gradient signal — all three properties must hold simultaneously.

What makes this reframing distinctive is that it internalizes optimization dynamics into the valuation criterion itself. Prior gradient-based methods like LESS (Xia et al., 2024) use gradients as features for similarity search against a target validation set — the gradients are a medium for matching, not the object of analysis. GRADFILTERING inverts this: the gradients ARE the object of analysis. The question is not "which training examples resemble my validation examples in gradient space?" but "which training examples exhibit gradient patterns characteristic of genuine, stable learning?" The answer emerges from within the training process, not from external validation targets.

This reframing has a subtle but important consequence: it makes data valuation objective-agnostic without being content-blind. Superfiltering's IFD score is also objective-agnostic (no validation set needed), but it is content-blind in the sense that it can't distinguish between an example that is hard because it's informative and one that's hard because it's noisy — both produce low IFD. G-SNR can make this distinction because it sees the dynamics: both noisy and informative examples may start with high gradients, but informative examples show gradient decay as learning progresses while noisy examples do not. The dynamics reveal content quality without requiring content understanding.

Evidence: The ablation in Table 2 directly tests whether the G-SNR framing is necessary or whether simpler gradient-based utilities suffice. Using raw gradient drop (Alternative #1) — which captures "how much the gradient decreased" without the signal-to-noise framing — consistently underperforms the full G-SNR, with drops as large as −0.44 PWS (Alpaca-GPT4, LLaMA2-7B, LoRA, 5%). This confirms that the SNR framing (jointly optimizing for high signal AND low noise) is not a cosmetic label but captures a genuinely distinct and more predictive property of training examples.

Significance: This is a reframing-level contribution, not merely a new scoring formula. It suggests that data valuation for instruction tuning should be understood as a filtering problem in the time and ensemble dimensions, not just a ranking problem in the content dimension. The implication for future work is that improvements may come not from better proxy models or more sophisticated content analysis, but from richer characterizations of training dynamics — multi-scale gradient statistics, spectral analysis of the gradient covariance across ensemble members, or even learned dynamics-to-utility mappings.

Innovation 2: LoRA Ensembles as Lightweight Epistemic Uncertainty Probes for Data Selection

Ensembles for uncertainty quantification are a well-established idea in Bayesian deep learning (Lakshminarayanan et al., 2017), and LoRA ensembles specifically have been explored for prediction calibration (Mühlematter et al., 2024; Balabanov and Linander, 2024). But prior to this work, ensembles had not been used for data valuation or selection — their uncertainty estimates were applied to model predictions (how confident is the model in its answer?) not to training data utility (how confident are we that this example is worth training on?). GRADFILTERING repurposes the LoRA ensemble from an inference-time calibration tool into a data-time valuation tool.

The key insight is that epistemic uncertainty about model parameters — captured via ensemble disagreement — can serve as a proxy for data quality. If five independently initialized LoRA adapters, trained on the same data, disagree about how strongly a given example should update the model, that example is likely noisy, ambiguous, or out-of-distribution. If they agree, the example's training signal is reliable. This transforms ensemble variance from a measure of "how uncertain is the model about this input?" into a measure of "how trustworthy is the training signal from this example?"

What makes this distinctive as an innovation is the direction of the uncertainty transfer. Prior work on weak-to-strong data selection (Superfiltering) transfers point estimates from a small model to a large model — the small model's IFD score is used as a proxy for the large model's eventual performance. GRADFILTERING transfers distributional statistics — the small model's ensemble disagreement identifies examples that the large model would also find unreliable. This is a fundamentally different type of transfer: it's not "small model says example A is good, therefore large model will benefit from A," but rather "small model ensemble cannot reach consensus on example B, therefore B is unreliable as training data regardless of model scale."

The t-SNE visualizations in Figure 1 provide the empirical foundation for this transfer claim. They demonstrate qualitatively that the LoRA ensemble members (a) diverge into distinct regions of gradient-profile space (confirming non-degenerate diversity), (b) follow stable individual trajectories (confirming structured rather than random variation), and (c) cluster more tightly at later epochs (confirming that disagreement reduces for genuinely learnable examples while persisting for noise). Without this evidence, the ensemble variance could simply be initialization noise — random fluctuation with no epistemic meaning. Figure 1 argues it is structured epistemic uncertainty.

Evidence: Table 2, Alternatives #2 vs. full G-SNR. The relative gradient drop without uncertainty penalty (Alternative #2) consistently underperforms the full G-SNR that includes the $1/V^{(t)}_i$ term. For example, on Alpaca LLaMA2-13B (Full, 10%), Alternative #2 loses 0.13 PWS while the full G-SNR gains 0.10 over baseline (from Table 1). This confirms that the ensemble variance contributes independent information beyond the gradient drop — removing it degrades performance.

Significance: This is a novel application of an existing technique (LoRA ensembles) rather than a new technique per se, but the application unlocks a capability — gradient-based data valuation with uncertainty quantification at small-model cost — that was previously unavailable. It bridges two largely disconnected literatures: Bayesian uncertainty quantification (which focused on prediction reliability) and data selection for LLMs (which focused on content heuristics or gradient matching). The bridge enables future work on more sophisticated uncertainty-aware data valuation, such as using the full gradient covariance structure (not just norm variance) or adaptive ensemble sizes based on per-example uncertainty.

Innovation 3: The Complementary Roles of Scale Normalization and Uncertainty Normalization in G-SNR

The G-SNR formula $u_i = \frac{G^{(s)}_i - G^{(t)}_i}{G^{(s)}_i + \epsilon} \cdot \frac{1}{V^{(t)}_i + \epsilon}$ might appear as a simple engineering combination of two intuitive terms. But the paper's systematic ablation (Table 2) demonstrates something more fundamental: the two normalizations address orthogonal failure modes of gradient-based data selection, and neither alone suffices.

Scale normalization (dividing by $G^{(s)}_i$ ) addresses the problem that raw gradient magnitude is confounded by example length and difficulty. Without it, long responses or examples with rare vocabulary dominate the ranking because they produce larger absolute loss values — not because they teach instruction-following more effectively. This is a known issue in gradient-based methods, and the fix (relative rather than absolute gradient drop) is natural. What's distinctive is that the paper proves it empirically: Alternative #1 (raw drop) performs dramatically worse than Alternative #2 (relative drop), confirming that scale normalization is not merely cosmetic but essential.

Uncertainty normalization (dividing by $V^{(t)}_i$ ) addresses a different problem: gradient drop magnitude can be unreliable when ensemble members disagree. An example might show a large gradient drop for three ensemble members but a small (or negative) drop for two others. The ensemble-averaged drop would be moderate, but the example is actually high-variance — its utility estimate is uncertain. Without uncertainty normalization, such examples could rank deceptively high. Alternative #3 (variance-normalized raw drop) tests whether uncertainty normalization alone suffices and finds it does not — it still underperforms the full G-SNR because it lacks scale normalization.

The core insight is that these two normalizations are necessary and sufficient together — each addresses an independent failure mode, and their product (the full G-SNR) outperforms all partial forms. This is not obvious a priori: one might expect the two normalizations to be partially redundant (if high-variance examples also tend to have large initial gradients, normalizing by either would suffice) or to interact destructively (the product of two noisy estimates being noisier than either alone). Table 2 demonstrates that neither concern materializes — the product is consistently more robust than any single normalization.

This finding is significant beyond the specific G-SNR formula because it provides design principles for future gradient-based data valuation methods. The principle is: a good gradient utility should (a) be scale-invariant so that example length and intrinsic difficulty don't dominate, and (b) incorporate a measure of statistical reliability so that noisy gradient signals are down-weighted. Any method that violates either principle is likely to underperform one that respects both.

Evidence: Table 2 is the primary evidence. Across 24 experimental configurations (2 datasets × 2 model sizes × 3 selection ratios × 2 adaptation regimes), all three alternatives produce predominantly negative deltas relative to the full G-SNR. The consistency of this pattern — not just that G-SNR wins overall, but that each alternative loses across most individual settings — supports the claim that both normalizations are independently necessary.

Significance: This is an empirical discovery about the structure of gradient-based utility functions, rather than a theoretical contribution. It establishes a de facto standard for what a well-designed gradient utility should look like: multiplicative combination of a normalized signal term and a reliability term. Future work that proposes alternative formulations (e.g., using gradient direction instead of magnitude, or using continuous-time gradient flow rather than discrete snapshots) should preserve this two-factor structure unless they can demonstrate that their alternative addresses both failure modes through a different mechanism.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper uses two standard instruction-tuning corpora: Alpaca (Taori et al., 2023) and Alpaca-GPT4 (Peng et al., 2023), each containing 52,000 instruction–response pairs. Both are constructed through automated pipelines — Alpaca via self-instruction from GPT-3.5, Alpaca-GPT4 via distillation from GPT-4 — making them representative of the noisy, redundant instruction datasets that motivate the data selection problem. For evaluation, the paper follows common practice and uses the WizardLM evaluation set (Xu et al., 2023) with 218 instructions and the Vicuna evaluation set (Zheng et al., 2023) with 80 instructions. Both consist of open-ended instruction-following prompts designed for pairwise preference judgments rather than single-answer grading. The paper does not specify a separate validation set for hyperparameter tuning or strategy selection — the G-SNR computation uses only the training data, and the target model is fine-tuned on the selected subset with fixed hyperparameters.

Base model(s). The target models are from the LLaMA-2 family (Touvron et al., 2023): LLaMA-2-7B and LLaMA-2-13B. The choice of two model sizes tests whether G-SNR's effectiveness transfers across model scales. The proxy model used for G-SNR computation is GPT-2 — the paper does not specify which GPT-2 variant, but given the emphasis on the proxy being "small" (~124M parameters for GPT-2 small vs. 7B/13B for the target models), GPT-2 small is the most likely choice. The paper also considers two adaptation regimes for the target model: LoRA-based fine-tuning (low-rank adapters on a frozen backbone, matching the proxy's architecture) and full-parameter fine-tuning, testing whether G-SNR-selected data transfers across different optimization procedures.

Metrics. The primary metric is the Pairwise Winning Score (PWS), adopted from Li et al. (2024). For a given model fine-tuned on a selected subset, PWS is computed as:

$\text{PWS} = 1 + \frac{\text{Num(Win)} - \text{Num(Lose)}}{\text{Num(All)}}$

where the model is compared pairwise against the full-data baseline (the same target model fine-tuned on 100% of the training data) on each evaluation prompt. Win/Lose/Tie judgments are produced by LLM-as-a-judge evaluators, using the prompt template from Vicuna (Zheng et al., 2023) detailed in Appendix A.1. Two judge models are used: GPT-5.1 (proprietary) and Qwen3-235B-Instruct (open-source), with scores averaged over both judges. The open-source judge is included "to facilitate replication in case proprietary APIs become unavailable in the future." A PWS > 1.0 means the subset-trained model wins more often than the full-data model; PWS = 1.0 means parity; PWS < 1.0 means the full-data model is preferred. The paper also includes a small human evaluation (Section 5.3) with five annotators on 100 prompts sampled from WizardLM and Vicuna, validating that LLM-judge preferences align with human judgments.

Baselines. The paper compares against two data selection strategies:

Random Split: Select $\alpha \cdot N$ examples uniformly at random from the training set. This is the simplest baseline — if a data selection method cannot beat random, it is actively harmful. Random subsets "are frequently below the full-data baseline (PWS < 1), especially at smaller ratios" (Section 5.2), confirming that naive subsampling degrades performance.
Superfiltering (Li et al., 2024): The strong weak-to-strong baseline that computes Instruction-Following Difficulty (IFD) scores using a GPT-2 proxy and selects the top- $\alpha$ examples by IFD. The paper uses this as the primary competitive baseline because it also employs a small proxy model for efficient filtering. Superfiltering is described as "a strong baseline and can be favorable in some configurations" (Section 5.2), making it the appropriate target for demonstrating GRADFILTERING's additional value.

The paper explicitly positions LESS (Xia et al., 2024) as related work but does not include it as an experimental baseline, citing its computational expense and task-specific design as reasons for not running head-to-head comparisons.

Generation budget / compute accounting. The paper's primary unit of comparison is the selection ratio $\alpha \in \{0.05, 0.10, 0.15, 1.00\}$ , corresponding to the fraction of the full 52,000-example training set used for target model fine-tuning. This provides a direct measure of data efficiency: at $\alpha = 0.05$ , the model sees only 2,600 examples vs. 52,000 for the full-data baseline. The cost of computing selection scores is not factored into the budget comparison — GRADFILTERING requires training a GPT-2 LoRA ensemble for 2 epochs, which is cheaper than LESS's gradient datastore construction on the target model but more expensive than Superfiltering's forward-pass-only IFD computation. The paper argues (implicitly) that the proxy training cost is amortized over multiple downstream fine-tuning runs — you compute G-SNR once and reuse the selected subset for any number of target model trainings. For convergence analysis (Section 5.5), the comparison uses identical optimization steps — both GRADFILTERING and Superfiltering train on 10% subsets with the same learning rate, batch size, and optimizer, so the step count directly reflects compute.

Cross-validation / statistical protocol. The paper does not employ cross-validation or statistical significance testing for the main results in Table 1. Each PWS value appears to be a single point estimate from one run of the full pipeline: proxy ensemble training → G-SNR computation → subset selection → target model fine-tuning → pairwise evaluation. This is a limitation — without error bars or multiple seeds, it is impossible to assess whether differences between methods (e.g., GRADFILTERING's PWS of 1.16 vs. Superfiltering's 1.05 in the Alpaca LLaMA2-7B Full 5% setting) are statistically reliable or within the range of run-to-run variance from different random initializations of the LoRA adapters, different data orderings, or different evaluation prompts. The human evaluation (Section 5.3) provides some external validation of the LLM-judge trends, with win/tie/lose counts of 44/19/37 (Alpaca) and 49/8/43 (Alpaca-GPT4) for the 10% subset vs. 100% baseline comparison.

Main Quantitative Results

Overall Performance: GRADFILTERING vs. Random and Superfiltering

Headline finding: GRADFILTERING consistently outperforms random subsets and is competitive with or superior to Superfiltering across most dataset, model size, selection ratio, and adaptation regime combinations, with particularly strong advantages in full-parameter fine-tuning settings (Table 1).

Comparison to Random Split (Table 1): Random subsets are systematically below the full-data baseline (PWS < 1.0), especially at smaller selection ratios. For example, on Alpaca with LLaMA2-7B (LoRA), Random Split achieves PWS of 0.97, 0.93, and 0.92 at 5%, 10%, and 15% respectively — meaning the full-data model wins more often in pairwise comparisons. GRADFILTERING, by contrast, achieves PWS of 1.01, 1.07, and 1.06 at the same ratios — consistently above the full-data parity line of 1.0. This gap is even more pronounced in full-parameter fine-tuning: on Alpaca LLaMA2-7B (Full), Random Split achieves 0.96/0.95/0.98 vs. GRADFILTERING's 1.16/1.16/1.12 at 5%/10%/15%. The fact that GRADFILTERING can train on 5% of the data and beat the full-data model (PWS = 1.16 > 1.0), while Random Split on 5% of the data loses to the full-data model (PWS = 0.96 < 1.0), demonstrates that the selection criterion is identifying genuinely more valuable examples, not just avoiding harmful ones.

Comparison to Superfiltering (Table 1): Superfiltering is a strong baseline that often matches or exceeds the full-data model. For instance, on Alpaca LLaMA2-7B (LoRA), Superfiltering achieves 1.02/1.10/1.02 at 5%/10%/15%, and on Alpaca LLaMA2-13B (LoRA) it achieves an impressive 1.18/1.10/1.11. GRADFILTERING is competitive in these LoRA settings — sometimes slightly better (1.19/1.20/1.12 on Alpaca LLaMA2-13B LoRA vs. Superfiltering's 1.18/1.10/1.11), sometimes slightly worse (1.04/1.02/1.15 on Alpaca-GPT4 LLaMA2-7B LoRA vs. Superfiltering's 0.94/1.12/1.13 — note Superfiltering's 0.94 at 5% is anomalously low, which GRADFILTERING's 1.04 substantially improves upon).

The clearest GRADFILTERING advantage emerges in full-parameter fine-tuning, where the method "provides more consistent gains across backbones and regimes, with particularly strong improvements in several 13B settings and in full-parameter fine-tuning where optimization dynamics and sample interactions are more pronounced" (Section 5.2). On Alpaca LLaMA2-7B (Full): GRADFILTERING achieves 1.16/1.16/1.12 vs. Superfiltering's 1.05/1.08/1.11 — a clear advantage at 5% (1.16 vs. 1.05) and 10% (1.16 vs. 1.08). On Alpaca-GPT4 LLaMA2-13B (Full): GRADFILTERING scores 1.08/1.13/1.01 vs. Superfiltering's 1.00/1.12/1.15 — more mixed, with GRADFILTERING better at 5% and 10% but slightly worse at 15%.

The paper reports that GRADFILTERING "matches or outperform[s] Random and Superfiltering in 19/24 LLM-as-a-judge evaluation cases" (from the abstract). Given the 24 configurations (2 datasets × 2 model sizes × 3 selection ratios × 2 adaptation regimes — but the 100% columns in Table 1 are not independent configurations since they use the full data and PWS is trivially 1.0 by definition; the "24 cases" likely refers to the 12 non-100% configurations × 2 comparison targets — Random and Superfiltering — making 24 pairwise comparisons). In 19 of these 24, GRADFILTERING achieves a PWS ≥ the comparison method.

Notable pattern — GRADFILTERING's relative strength grows with difficulty: On the more challenging Alpaca-GPT4 dataset (which contains GPT-4-generated responses that are longer and more complex than Alpaca's GPT-3.5-generated responses), GRADFILTERING shows a consistent advantage over Superfiltering in several settings where Superfiltering struggles. For example, on Alpaca-GPT4 LLaMA2-7B (LoRA, 5%), Superfiltering scores only 0.94 (below the full-data baseline) while GRADFILTERING scores 1.04 (above baseline). Similarly, on Alpaca-GPT4 LLaMA2-7B (LoRA, 10%), Superfiltering's 1.12 is strong but GRADFILTERING's 1.02 is slightly lower; however, on the same dataset with Full fine-tuning at 5%, GRADFILTERING's 1.01 edges Superfiltering's 1.15 — wait, that's an error in my reading. Let me be precise: Alpaca-GPT4 LLaMA2-7B (Full, 5%): Superfiltering = 1.15, GRADFILTERING = 1.01 — Superfiltering wins here. The pattern is not uniformly in GRADFILTERING's favor on Alpaca-GPT4; rather, GRADFILTERING prevents the occasional Superfiltering failure (the 0.94 at 5% LoRA) while remaining competitive in other settings.

Human evaluation alignment (Section 5.3): The human study compares LLaMA2-13B (full fine-tuning) trained on 10% GRADFILTERING-selected data against the 100% baseline, using 100 prompts from WizardLM and Vicuna evaluated by five annotators on helpfulness, relevance, accuracy, and level of detail. The win/tie/lose counts are 44/19/37 for Alpaca and 49/8/43 for Alpaca-GPT4. Converting to a PWS-like metric: for Alpaca, 1 + (44-37)/100 = 1.07; for Alpaca-GPT4, 1 + (49-43)/100 = 1.06. These are consistent with the LLM-judge PWS values in Table 1 for the corresponding configurations (Alpaca LLaMA2-13B Full 10%: GRADFILTERING = 1.10; Alpaca-GPT4 LLaMA2-13B Full 10%: GRADFILTERING = 1.13). The human study confirms that the LLM-judge preferences are not an artifact of the judge model's biases — real human evaluators also prefer the GRADFILTERING-trained model's outputs at roughly the same rate.

Convergence Analysis: GRADFILTERING Converges Faster

Headline finding: GRADFILTERING-selected 10% subsets produce faster convergence and lower training loss than Superfiltering-selected 10% subsets under identical compute budgets, for both full-parameter and LoRA fine-tuning (Figure 3).

Full-parameter fine-tuning (Figure 3a): On LLaMA-2-13B with 10% of Alpaca, the GRADFILTERING curve (ours-10%) starts at a similar initial loss to Superfiltering-10% but drops more steeply and remains lower throughout training. By step 200, the gap is visible; by step 800, GRADFILTERING reaches a training loss of approximately 1.05–1.06 vs. Superfiltering's approximately 1.08–1.09. The paper interprets this as evidence that "our uncertainty-aware, gradient-based criterion better matches the global training dynamics than optimizing a single proxy" (Section 5.5).

LoRA fine-tuning (Figure 3b): The pattern is similar but the gap is smaller. Both curves start around 1.18–1.20 and decrease to approximately 1.12–1.14 by step 800, with GRADFILTERING maintaining a consistent but narrow advantage (roughly 0.01–0.02 lower loss throughout most of training). The smaller gap in LoRA mode is consistent with the paper's claim that "optimization dynamics and sample interactions are more pronounced" in full-parameter fine-tuning — when only a small number of parameters are trained (LoRA), the choice of training examples has less impact on convergence behavior than when all parameters are updated (full fine-tuning).

Why this matters beyond preference scores: Faster convergence at the same subset size means GRADFILTERING is not just selecting examples that lead to better final models, but selecting examples that optimize more efficiently — the gradient signal from these examples is better-aligned with the global loss landscape, allowing the optimizer to make more progress per step. This is a distinct benefit from final performance: even if two selection methods eventually reach the same loss, the one that gets there faster saves compute. The paper explicitly frames this as support for the "global optimization view" that motivates G-SNR: "examples interact through shared parameters, so a good subset should drive stable, efficient descent" (Section 5.5).

Addressing a potential confound: The paper acknowledges that "GRADFILTERING could be suspected to favor 'easy' data" — if the selected subset consists only of trivially simple examples, the model would converge quickly but produce poor-quality responses (low loss on easy examples doesn't translate to good instruction following). The preference results in Table 1 argue against this interpretation: GRADFILTERING-trained models consistently outperform or match full-data baselines on diverse evaluation prompts, indicating improved response quality rather than a trivial bias toward easy examples. The convergence advantage is genuine efficiency, not a shortcut.

Ablation Studies and Robustness Checks

Ablation of G-SNR utility components (Table 2): The paper tests three simplified utility functions against the full G-SNR, reporting the gain or drop in Pairwise Winning Score relative to the G-SNR results in Table 1. Negative values mean the alternative performs worse than G-SNR; positive values mean it performs better.

Utility Alternative #1 (raw gradient drop): $u_i = G^{(1)}_i - G^{(T)}_i$ . This is the simplest possible gradient-based utility — just the absolute decrease in gradient norm. Finding: Consistently the worst performer. On Alpaca-GPT4 LLaMA2-7B (LoRA, 5%), it loses 0.44 PWS relative to G-SNR; on Alpaca-GPT4 LLaMA2-13B (LoRA, 5%), it loses 0.52 PWS. These are the largest drops in the entire table, confirming that "absolute scale alone is not a reliable indicator of data utility" (Section 5.4). The raw drop is confounded by example length and intrinsic difficulty — long responses with many tokens produce larger absolute gradient norms and larger absolute drops, even if the relative learning progress is small. Note a rare positive value: +0.04 on Alpaca LLaMA2-13B (LoRA, 10%), suggesting that in some configurations the raw drop might be adequate, but this is an outlier.
Utility Alternative #2 (relative gradient drop, no uncertainty): $u_i = (G^{(1)}_i - G^{(T)}_i) / (G^{(1)}_i + \epsilon)$ . This normalizes for initial gradient magnitude but ignores ensemble variance. Finding: Better than Alternative #1, but still consistently worse than G-SNR. The drops are typically in the -0.10 to -0.30 range for LoRA settings and -0.10 to -0.35 for Full settings. For instance, on Alpaca-GPT4 LLaMA2-7B (Full, 10%), it loses 0.34 PWS. This demonstrates that scale normalization alone is insufficient — high-confidence, low-variance gradient drops contain more information than high-variance drops of the same magnitude, and ignoring this distinction degrades selection quality. The relative drop gives equal weight to a cleanly-learned example and a noisily-learned one, and the latter act as contaminants in the selected subset.
Utility Alternative #3 (variance-normalized raw drop, no scale normalization): $u_i = (G^{(1)}_i - G^{(T)}_i) / (V^{(T)}_i + \epsilon)$ . This incorporates uncertainty but doesn't normalize for initial gradient scale. Finding: Intermediate performance — better than raw drop, worse than full G-SNR. Drops range from -0.02 to -0.41 across configurations. For example, on Alpaca-GPT4 LLaMA2-7B (Full, 15%), it loses 0.41 PWS. This confirms that uncertainty normalization alone doesn't solve the scale problem — long examples with large absolute gradient drops still dominate the ranking, even if their variance is low.

Key takeaway from the ablation: The three alternatives correspond to removing one or both of the two normalization factors in G-SNR. Alternative #1 removes both (pure drop), Alternative #2 keeps only scale normalization (relative drop), Alternative #3 keeps only uncertainty normalization (variance-normalized drop). The consistent ranking — G-SNR > Alternative #2 ≈ Alternative #3 > Alternative #1 — supports the paper's claim that both normalizations are independently necessary and that their product (the full G-SNR) is the most robust formulation.

Robustness across adaptation regimes: The ablation trends hold for both LoRA and Full fine-tuning, with some variation in magnitude. In LoRA settings, the degradation from removing components is sometimes larger (e.g., Alternative #1 loses 0.44–0.52 on Alpaca-GPT4 LoRA at 5%), suggesting that LoRA's limited parameter capacity makes it more sensitive to data quality — noisy examples have a proportionally larger impact when there are fewer parameters to absorb the noise. In Full settings, the degradation is more moderate but still consistent, confirming that G-SNR's advantages are not specific to one adaptation regime.

Robustness across model sizes and datasets: The ablation patterns are broadly consistent across LLaMA2-7B and LLaMA2-13B, and across Alpaca and Alpaca-GPT4, though the magnitude of differences varies. On Alpaca-GPT4, where responses are longer and more complex (GPT-4 generated), the importance of scale normalization (Alternative #2 vs. #1) appears larger — the raw-drop alternative loses more on Alpaca-GPT4 (-0.44, -0.52) than on Alpaca (-0.04, -0.19 at the same 5% LoRA settings), consistent with the idea that longer responses amplify the scale confound.

Negative utility handling: The paper notes that "G-SNR and its variants can also produce negative utilities" when $G^{(1)}_i - G^{(2)}_i < 0$ (the gradient magnitude increases rather than decreases). Under top- $\alpha$ selection, negative-utility examples are simply pushed to the bottom of the ranking and not selected. The paper argues that "downweighting highly negative utilities is desirable, as they often correlate with examples whose gradients do not exhibit consistent improvement over training" (Section 5.4). This is an implicit claim that G-SNR correctly identifies examples that are actively harmful — they become harder for the model after training, suggesting interference with other examples or inherent noise. The paper does not provide a dedicated analysis of negative-utility examples (e.g., manual inspection or correlation with downstream harm), so this claim is plausible but unverified.

LoRA ensemble trajectory visualization (Figure 1): While not an ablation in the traditional sense, Figure 1 serves as a robustness check on the core assumption that LoRA ensembles produce meaningful epistemic uncertainty. The t-SNE visualizations show that ensemble member trajectories (a) diverge from similar starting points, (b) follow stable, non-collapsing paths, and (c) cluster more tightly at later epochs. If the trajectories had collapsed to a single point (all members identical), the ensemble variance would be pure initialization noise with no epistemic content. If they had diverged chaotically (no structure), the ensemble variance would be uninformative random fluctuation. The observed pattern — structured divergence followed by partial convergence — is consistent with genuine epistemic uncertainty that reduces as training resolves ambiguity about which examples are informative. The paper acknowledges this is "qualitative evidence" and does not quantify the relationship between trajectory diversity and selection quality, but it provides a necessary sanity check.

Ratio of sequential to parallel (not applicable to this paper — I am noting this is not relevant): GRADFILTERING uses only parallel ensemble training (all members train independently on the same data), not sequential revisions. There is no ratio sweep in this paper.

Critical Assessment

The experiments in this paper are carefully designed to isolate the effect of the G-SNR selection criterion, but they also have several structural limitations that constrain the strength of the conclusions that can be drawn. I examine each of the paper's central claims against the reported evidence.

Claim 1 (from the abstract): "GRADFILTERING matches or surpasses random subsets and strong baselines in most LLM-as-a-judge evaluations."

What was tested: Table 1 reports pairwise winning scores for 12 primary configurations (2 datasets × 2 model sizes × 3 selection ratios) across two adaptation regimes (LoRA and Full), compared against Random Split and Superfiltering. The paper states GRADFILTERING wins in 19/24 evaluation cases.

Does the evidence support the claim? Yes, but with important nuance. GRADFILTERING clearly outperforms Random Split in nearly all configurations — this is a low bar and the paper clears it comfortably. Against Superfiltering, the picture is more mixed. In LoRA settings, the two methods are roughly comparable — Superfiltering sometimes has a slight edge, GRADFILTERING sometimes does. The gap is small enough (often 0.01–0.05 PWS) that without confidence intervals, it's unclear whether these differences are statistically meaningful or within run-to-run noise. In Full fine-tuning settings, GRADFILTERING shows a more consistent advantage, particularly at smaller selection ratios (5%, 10%) on Alpaca. But on Alpaca-GPT4 Full, the advantage is narrower and in some configurations Superfiltering actually wins (e.g., Alpaca-GPT4 LLaMA2-7B Full 5%: Superfiltering 1.15 vs. GRADFILTERING 1.01).

What's missing: The paper does not report any measure of variance — no standard deviations, no confidence intervals, no multiple random seeds. Each number in Table 1 is a single point estimate. Given that the G-SNR computation involves random initialization of LoRA adapters, random data ordering during proxy training, and random sampling during LLM-as-a-judge evaluation, there is non-trivial stochasticity at every stage. A difference of 0.02 PWS could easily be within the noise floor. The human evaluation partially addresses this by providing an independent measurement (the win/tie/lose counts), which align with the LLM-judge trends, but this is a single comparison at one configuration (13B, 10%, Full) rather than a systematic variance analysis.

The "19/24" framing is somewhat generous. Counting "match or outperform" includes cases where GRADFILTERING is equal to the baseline (PWS difference ≈ 0), which doesn't demonstrate superiority. A more conservative count of configurations where GRADFILTERING has a clear lead (say, ≥ 0.03 PWS advantage) would yield fewer wins, particularly against Superfiltering in LoRA settings. The abstract's claim is technically true but masks the closer competition with Superfiltering.

Claim 2 (from the abstract and Section 5.5): "GRADFILTERING-selected subsets converge faster than competitive filters under the same compute budget."

What was tested: Figure 3 shows training loss curves for one configuration (LLaMA-2-13B, 10% Alpaca) comparing GRADFILTERING and Superfiltering, for both Full and LoRA fine-tuning.

Does the evidence support the claim? The evidence is suggestive but extremely limited. Faster convergence is demonstrated for exactly one configuration (one model size, one dataset, one selection ratio). The paper generalizes this to "GRADFILTERING-selected subsets converge faster" in the abstract and Section 5.5, but the experiment only shows this for a single data point. It's plausible that the convergence advantage generalizes — the mechanism (G-SNR selecting examples with cleaner gradient signals) should apply broadly — but the paper does not demonstrate this. A minimal extension would be to show convergence curves for at least one other configuration (e.g., 7B model, or Alpaca-GPT4, or 5% selection ratio) to establish that this is not an idiosyncratic result.

Additionally, the convergence analysis only compares GRADFILTERING to Superfiltering — not to Random Split. It would be informative to see whether random subsets also converge faster than Superfiltering (which would suggest Superfiltering selects "harder" examples that train more slowly but potentially reach higher quality) or slower (which would suggest both non-random methods improve convergence over random, but GRADFILTERING does so more). The absence of a Random Split convergence curve limits the interpretability of the result.

The claim about "lower training loss" requires careful interpretation. Lower training loss at convergence does not necessarily imply better generalization — it could indicate overfitting to a subset that is too easy or too narrow. The preference results in Table 1 partially address this concern (GRADFILTERING-trained models produce preferred responses), but training loss and downstream performance are different metrics. A model with lower training loss but worse evaluation performance would be a negative result; the paper avoids this by showing that GRADFILTERING achieves both lower loss and better or comparable evaluation scores.

Claim 3 (from Section 5.4 and Table 2): "Combining a relative gradient-drop term with an uncertainty-based penalty provides a more stable and informative signal than either component in isolation."

What was tested: Table 2 ablates three alternative utility functions against the full G-SNR across all 24 configurations.

Does the evidence support the claim? Strongly. This is the most robust empirical finding in the paper. The pattern is consistent across nearly all configurations: Alternative #1 (raw drop) is worst, Alternatives #2 and #3 are intermediate, and full G-SNR is best. There are a few exceptions — Alternative #3 shows a +0.01 on Alpaca LLaMA2-7B LoRA 10%, and Alternative #1 shows a +0.04 on Alpaca LLaMA2-13B LoRA 10% — but these are small positive values in a sea of negative ones, and they don't challenge the overall pattern.

What makes this evidence convincing: The ablation is comprehensive — it covers all model sizes, datasets, selection ratios, and adaptation regimes — and the direction of effect is consistent. This is not a cherry-picked result on one configuration. The magnitude of the degradation from removing components is often substantial (0.2–0.5 PWS), suggesting the effects are practically meaningful, not just statistically detectable. For full-parameter fine-tuning, the degradation from removing components is sometimes larger than for LoRA (e.g., Alternative #1 loses 0.31 on Alpaca LLaMA2-7B Full 5% vs. 0.04 on LoRA), consistent with the idea that full fine-tuning — where all parameters change — is more sensitive to the quality of the gradient signal because there's more capacity to go wrong with noisy data.

A minor limitation: The ablation only tests three specific decompositions of the G-SNR formula. It doesn't explore other natural variants, such as using the product of the early and late variance (penalizing examples with high variance at either epoch), using a different norm (L1 instead of L2), using the median instead of the mean for ensemble aggregation, or computing the relative drop between different epoch pairs (1→5, 1→3, 2→5). The space of possible utility functions is large, and the ablation only covers the simplest three. The conclusion that G-SNR is the "most robust" formulation is relative to these three alternatives — there might be better formulations not tested.

Claim 4 (implicit): "G-SNR is objective-agnostic and transfers from GPT-2 to LLaMA-2."

What was tested: G-SNR is computed on GPT-2 gradients; the selected subsets are used to train LLaMA-2 models. The LLaMA-2 models are then evaluated on WizardLM and Vicuna prompts.

Does the evidence support the claim? The transfer clearly works in the configurations tested — if it didn't, GRADFILTERING would perform no better than random. However, the paper provides no direct evidence for the transfer mechanism. It does not, for example:

Report the correlation between GPT-2 G-SNR scores and LLaMA-2 per-example loss reduction (if G-SNR transfers, examples that are informative for GPT-2 should also be informative for LLaMA-2).
Test whether GPT-2 G-SNR scores correlate better with LLaMA-2 performance than GPT-2 IFD scores do (if the gradient-based method captures something beyond what IFD captures, the correlation with downstream performance should be higher).
Test whether using a larger proxy model (e.g., LLaMA-2-7B itself, or GPT-2-medium) improves selection quality (if transfer is imperfect due to capacity mismatch, a stronger proxy should close the gap).
Test whether using the target model itself (LLaMA-2-7B) to compute G-SNR would produce better subsets than using GPT-2 (this would establish the cost of the proxy mismatch).

The weak-to-strong transfer is an empirical claim that the paper treats as validated by the downstream results, but the mechanism remains a black box. This matters because if the transfer breaks down for other model families (e.g., non-LLaMA architectures, much larger scale gaps, different training objectives), there's no diagnostic to detect the failure beyond running full fine-tuning experiments.

Structural limitations of the experimental design:

Single proxy model (GPT-2). All G-SNR computations use the same GPT-2 variant. The sensitivity of results to proxy choice is unknown. A smaller proxy (e.g., GPT-2 distilled) might be cheaper but less reliable; a larger proxy (GPT-2-medium) might be more reliable but more expensive. The paper provides no guidance on how to choose the proxy.
Single training objective (next-token prediction). The "objective-agnostic" claim is supported only for supervised instruction tuning. The paper does not test G-SNR on other objectives where gradient dynamics might behave differently — e.g., preference optimization (DPO, RLHF), multi-task mixtures with varying loss scales, or objectives with auxiliary losses. This is acknowledged as a limitation in Section 6: "we instantiate and evaluate GRADFILTERING only on supervised instruction tuning, but we argue that the same G-SNR principle can naturally extend to other objectives."
Fixed early/late snapshots (epochs 1 and 2). The choice of $s=1$ and $t=2$ is justified by the t-SNE observation that trajectories stabilize after epoch 2, but this is an empirical observation on the specific datasets and proxy model used. If the proxy model converges more slowly on a different dataset (e.g., a larger, more diverse instruction corpus), epochs 1 and 2 might not capture the informative part of the training dynamics — the gradient drop might still be in progress, making the G-SNR estimate noisy or biased. The paper does not test sensitivity to the epoch choice.
No evaluation on standard NLP benchmarks. All evaluation is pairwise preference on open-ended instruction-following prompts. The paper does not test whether GRADFILTERING-selected subsets preserve performance on standard NLP tasks (e.g., MMLU, HellaSwag, GSM8K) that are sometimes degraded by instruction tuning (the "alignment tax"). This matters because a selection method that preserves instruction-following quality while discarding 90% of data might inadvertently discard examples that are important for maintaining broad capabilities.
Fixed selection ratios (5%, 10%, 15%). These specific ratios are tested, but the paper doesn't explore whether there's an optimal ratio or how performance degrades as the ratio decreases below 5%. If 1% of data could achieve 95% of full-data performance, that would be a much stronger result. Conversely, if performance collapses at 2.5%, that would establish a floor.
No comparison to LESS or other gradient-based methods. The paper positions LESS (Xia et al., 2024) as a key related work and discusses its limitations at length in Section 2.1, but does not include it as an experimental baseline. This is a significant omission. The argument that LESS is "too expensive" is reasonable for making GRADFILTERING practical, but a head-to-head comparison on at least one configuration would establish whether G-SNR's simpler, objective-agnostic approach sacrifices performance relative to the more sophisticated, task-specific LESS approach. Without this comparison, we don't know whether GRADFILTERING is competitive with the state-of-the-art or merely with a strong but simpler baseline (Superfiltering).
No test on truly large-scale instruction datasets. Alpaca and Alpaca-GPT4 each contain 52,000 examples — large enough to motivate data selection, but small compared to modern instruction datasets that can contain millions of examples (e.g., OpenOrca, Dolly, self-instruct corpora). The paper's claim that G-SNR helps with "large-scale instruction datasets" (Section 1) is plausible but not tested at scale. At 52,000 examples, even random sampling sometimes works reasonably well; the value of sophisticated data selection might be larger when the dataset is more noisy and redundant, which is more likely at million-example scales.

What experiments would have strengthened the paper:

Variance analysis: Run the full pipeline with 3–5 different random seeds for the LoRA ensemble initialization and report means and standard deviations for key results. This would establish whether the observed differences (e.g., 1.16 vs. 1.05) are reliable.
Correlation analysis: For a held-out set of examples, compute both GPT-2 G-SNR and LLaMA-2 gradient-drop statistics, and report the correlation. This would directly test the transfer assumption.
Proxy model scaling: Run GRADFILTERING with different proxy sizes (GPT-2 small, GPT-2 medium, maybe a small LLaMA) and measure how selection quality scales with proxy capability.
Epoch ablation: Test G-SNR with different snapshot pairs (1→2, 1→3, 1→4, 2→3) to establish sensitivity to the epoch choice.
LESS comparison: Run LESS on one configuration (e.g., Alpaca, LLaMA2-7B, 10%) and compare PWS against GRADFILTERING on the same evaluation prompts, using the same target model training protocol. This would position GRADFILTERING relative to the strongest gradient-based baseline.
Scaling to larger datasets: Test on a 200K+ instruction dataset to see if the advantages over Superfiltering grow (as the dataset becomes more noisy/redundant) or shrink (as even simple methods have enough signal).

Bottom line: The experiments convincingly demonstrate that G-SNR outperforms random selection and is broadly competitive with Superfiltering, with particular advantages in full-parameter fine-tuning. The ablation in Table 2 provides strong evidence that both components of G-SNR (relative gradient drop and uncertainty penalty) are independently necessary. However, the single-seed, single-proxy, single-objective nature of the experiments limits the generality of the conclusions, and the absence of a LESS comparison leaves open the question of whether GRADFILTERING advances the state-of-the-art or provides a cheaper alternative with some performance trade-off. The convergence analysis (Figure 3) is suggestive but limited to one configuration and lacks the Random Split baseline that would contextualize the improvement. These are not fatal flaws — the paper's contributions are clearly supported by the evidence provided — but they represent genuine limitations that should temper claims of broad applicability beyond the tested settings.

6. Limitations and Trade-offs

The Weak-to-Strong Transfer From GPT-2 to LLaMA-2 Is Assumed, Not Verified

The assumption or constraint. GRADFILTERING computes G-SNR utilities from the gradient dynamics of a small GPT-2 proxy model (likely GPT-2 small, ~124M parameters) and then transfers the resulting data ranking to the target LLaMA-2 model (7B or 13B parameters) for fine-tuning. The method therefore assumes that examples which produce large, consistent, decaying gradients for GPT-2 will also be informative for LLaMA-2 — that the structural properties of instruction–response pairs that make them effective training signals are invariant across model scales and architectures separated by roughly two orders of magnitude in parameter count. The paper does not test this assumption directly. Section 6 states that "GRADFILTERING is largely model-agnostic" and that the framework "does not assume a particular architecture," but provides no evidence that the GPT-2 G-SNR ranking correlates with LLaMA-2's per-example learning dynamics — no gradient correlation analysis, no comparison of selected subsets when G-SNR is computed from different proxy sizes, and no experiment using the target LLaMA-2 model itself as the proxy to establish the performance ceiling that the weak-to-strong transfer leaves on the table.

The consequence. If the transfer assumption breaks down — for instance, if GPT-2 finds certain examples challenging due to limited capacity while LLaMA-2 handles them easily using its pre-trained knowledge, or conversely if GPT-2 saturates quickly on instruction formats that LLaMA-2 genuinely benefits from practicing — then the G-SNR ranking derived from the proxy would systematically mis-rank examples for the target model. Examples that are genuinely valuable for LLaMA-2 but happen to produce uninformative gradient patterns for GPT-2 would be discarded; examples that are trivial or noisy for LLaMA-2 but produce clean gradient drops for GPT-2 would be selected. The downstream consequence is that GRADFILTERING's performance advantage relative to content-based methods like Superfiltering (which also uses GPT-2 but through IFD scores rather than gradient dynamics) could shrink or reverse for model families or scales where the transfer is weaker. The paper's ablation in Table 2 shows that removing the uncertainty normalization (Alternative #2) degrades performance — but this only tests the importance of the G-SNR formulation given the proxy choice, not the importance of the proxy choice itself. A practitioner deploying GRADFILTERING on a non-LLaMA target model (e.g., Mistral, Gemma, Qwen) or at a much larger scale gap (GPT-2 → 70B) has no guidance on whether the transfer will hold.

What evidence exists in the paper. The paper provides only indirect evidence that transfer works: the downstream evaluation results in Table 1 show that LLaMA-2 models trained on GPT-2-derived G-SNR subsets perform well. But this is a black-box validation — it tells us that the transfer is "good enough" for these specific configurations, not why it works or when it might fail. There is no correlation analysis between GPT-2 G-SNR scores and any measure of LLaMA-2 per-example utility (e.g., LLaMA-2 gradient drop, per-example loss reduction, or influence on held-out performance). There is no experiment varying the proxy model size or family to measure how selection quality changes. There is no comparison using LLaMA-2-7B itself as the proxy to establish the upper bound on what perfect transfer would achieve. The t-SNE visualizations in Figure 1 show that the LoRA ensemble produces structured, non-degenerate trajectories — but this only validates that the proxy-side gradient statistics are meaningful, not that they transfer across model scales.

Mitigation status. Not addressed. The paper acknowledges in Section 7 (Limitations) that "the proxy model must still be fine-tuned with backpropagation, which is cheaper than running a strong teacher or constructing a full gradient datastore, but not free" — this acknowledges the cost of the proxy but not the validity of the transfer. The model-agnostic claim in Section 6 states that "beyond requiring a differentiable backbone and a parameter-efficient adaptation mechanism ... the framework does not assume a particular architecture or instruction format," which implicitly asserts transferability without evidence. The paper does not suggest future work on characterizing or improving the transfer mechanism, leaving a significant open question for practitioners.

The Cost of Proxy Ensemble Training Is Excluded From All Reported Efficiency Numbers

The assumption or constraint. Every result in the paper — the pairwise winning scores in Table 1, the convergence curves in Figure 3, and the headline claim that GRADFILTERING matches full-data performance with 5–15% of the data — compares the target model fine-tuning cost across methods while ignoring the cost of computing the selection itself. GRADFILTERING's selection pipeline requires training a GPT-2 LoRA ensemble of $M=5$ members for $T=2$ epochs on the full 52,000-example dataset, which involves forward and backward passes through the GPT-2 backbone plus five sets of LoRA adapters, and collecting and storing per-example gradient norms for $52{,}000 \times 5 \times 2 = 520{,}000$ gradient computations. In contrast, Superfiltering requires only forward passes through a single GPT-2 model (no training, no ensemble, no gradient collection) to compute IFD scores. Random selection requires zero proxy computation. The paper does not quantify the FLOPs or wall-clock time of any of these selection methods, nor does it present a total-cost comparison (selection + target model fine-tuning) at any selection ratio.

The consequence. A practitioner deciding between data selection methods needs to know the total end-to-end cost, not just the target model fine-tuning cost. If GRADFILTERING's proxy ensemble training takes, say, the equivalent of fine-tuning the target model on an additional 5–10% of data, then a fair comparison would show GRADFILTERING's 10% subset requires the cost of 15–20% of full-data fine-tuning (when proxy cost is amortized over a single target model run) or more. The paper's convergence advantage in Figure 3 shows that GRADFILTERING-selected data trains faster than Superfiltering-selected data under identical step counts — but this doesn't account for the fact that GRADFILTERING's selection took longer to produce than Superfiltering's. In a production setting where data selection is run once and the curated dataset is reused for many downstream fine-tuning jobs, the proxy cost amortizes to near-zero per target model. But for single-use scenarios (a practitioner wants to fine-tune a model once on the best available subset), the proxy cost is a substantial fraction of the total budget and could make GRADFILTERING more expensive overall than simply training on the full dataset with Superfiltering-selected data — or even on the full dataset itself, if the proxy cost exceeds the savings from using a smaller subset for the target model.

What evidence exists in the paper. None. The paper provides no FLOPs counts, no wall-clock times, and no cost model for the proxy ensemble training. The only efficiency claim is qualitative: the proxy is described as "small" (Section 3.2) and the LoRA ensemble as "parameter-efficient" (Section 3.1), and the convergence curves in Figure 3 show that GRADFILTERING-selected data trains faster once selection is done. But there is no experiment that accounts for selection cost. The paper does note in Section 7 (Limitations) that "the proxy model must still be fine-tuned with backpropagation, which is cheaper than running a strong teacher or constructing a full gradient datastore, but not free" — this acknowledges the cost exists but does not quantify it or include it in any comparison.

Mitigation status. The paper partially acknowledges the issue in the Limitations section by stating that proxy training "is cheaper ... but not free," but does not provide the quantitative analysis that would allow a practitioner to make an informed cost trade-off. The amortization argument (compute G-SNR once, reuse for many target models) is implicit but never stated explicitly — the paper doesn't discuss how selection cost scales with repeated use or whether the selected subset remains optimal if the target model or training configuration changes. The paper also does not suggest future work on reducing proxy cost (e.g., through smaller proxies, shorter training, or amortized gradient computation), leaving the cost gap with Superfiltering unaddressed.

No Variance Estimates or Statistical Significance Testing, Making Performance Comparisons Unreliable

The assumption or constraint. Every number in Table 1 and Table 2 is a single point estimate from what appears to be a single run of the full pipeline: one random initialization of the LoRA ensemble → one G-SNR computation → one subset selection → one target model fine-tuning run → one set of pairwise evaluations. The paper does not report standard deviations, confidence intervals, or results from multiple random seeds for any experimental configuration. This is a significant methodological gap because the pipeline has stochasticity at multiple stages: (a) the LoRA ensemble members are randomly initialized with different seeds, and different initializations could produce different G-SNR rankings; (b) the data ordering during proxy training is shuffled, which affects the per-epoch gradient statistics that G-SNR is computed from; (c) the LLM-as-a-judge evaluations involve sampling from the judge models, and pairwise comparisons on open-ended prompts have inherent variability; and (d) the target model fine-tuning itself has randomness from data ordering and optimizer stochasticity. Without any variance estimates, it is impossible to determine whether a reported PWS difference of 0.02 (e.g., GRADFILTERING 1.16 vs. Superfiltering 1.05 on Alpaca LLaMA2-7B Full 5% — a difference of 0.09, which is larger but still a single number) is a reliable signal or within the range of run-to-run noise.

The consequence. The central empirical claim of the paper — that GRADFILTERING "matches or outperform[s] Random and Superfiltering in 19/24 LLM-as-a-judge evaluation cases" — is vulnerable to two failure modes that cannot be assessed without variance estimates. First, false positives: some of the 19 "wins" might be within noise, meaning GRADFILTERING's actual performance advantage is narrower than reported. Second, ranking instability: the relative ordering of GRADFILTERING and Superfiltering might flip across random seeds for configurations where they are closely matched (e.g., the LoRA settings where differences are often 0.01–0.05 PWS), meaning GRADFILTERING is not reliably better than Superfiltering in those regimes. A practitioner who reads Table 1 as a definitive performance ranking might select GRADFILTERING over Superfiltering for a configuration where the two are actually statistically indistinguishable, incurring the additional proxy training cost for no reliable gain. Conversely, a practitioner who dismisses GRADFILTERING because it "loses" in a particular column (e.g., Alpaca-GPT4 LLaMA2-7B Full 5%, where Superfiltering scores 1.15 vs. GRADFILTERING's 1.01) might be over-interpreting a single-run anomaly.

What evidence exists in the paper. The paper includes a small human evaluation (Section 5.3) with five annotators on 100 prompts for one configuration (LLaMA2-13B Full, 10%). This provides some external validation that the LLM-judge preferences align with human judgment for that specific data point, but it is not a substitute for systematic variance analysis — it tests the validity of the evaluation metric for one configuration, not the stability of the selection-then-fine-tuning pipeline across random seeds. The win/tie/lose counts of 44/19/37 (Alpaca) and 49/8/43 (Alpaca-GPT4) are themselves point estimates from a single fine-tuning run; we don't know how much these counts would vary if the same selection ratio were applied to a different G-SNR ranking produced by a different LoRA ensemble initialization.

Mitigation status. Not addressed. The paper does not mention the lack of variance estimates as a limitation, does not discuss the sources of stochasticity in the pipeline, and does not suggest that future work should include multiple-seed evaluation. The use of two judge models (GPT-5.1 and Qwen3-235B-Instruct) and the human evaluation show some awareness of evaluation-side variance but do not address the selection-side variance that could change which examples are chosen. A minimal fix — running the full pipeline 3–5 times with different LoRA ensemble seeds and reporting mean ± standard deviation for the main results — would substantially strengthen the reliability of the empirical claims.

Evaluation Is Restricted to a Single Task Family, Model Family, and Two Datasets of Modest Size

The assumption or constraint. All experiments use: (a) the LLaMA-2 model family (7B and 13B) as target models; (b) Alpaca and Alpaca-GPT4 (52,000 examples each, constructed via automated distillation pipelines from GPT-3.5 and GPT-4 respectively) as training datasets; and (c) open-ended instruction-following prompts from WizardLM (218 instructions) and Vicuna (80 instructions) as evaluation benchmarks, judged by pairwise preference rather than standard NLP accuracy metrics. The paper does not test GRADFILTERING on any other model architecture (e.g., Mistral, Gemma, Qwen), any other type of training objective (e.g., preference optimization via DPO, multi-task mixtures, safety tuning), any larger or more diverse dataset (modern instruction corpora routinely contain hundreds of thousands to millions of examples), or any standard NLP benchmark (e.g., MMLU, HellaSwag, GSM8K) to check whether the data selection preserves broad capabilities beyond instruction following.

The consequence. The claims of "objective-agnostic" and "model-agnostic" selection (Section 6) are supported only for the narrow configuration tested: supervised instruction tuning with next-token prediction loss on LLaMA-2 models. A practitioner deploying GRADFILTERING in a different setting faces several unknown risks:

Model family mismatch: The weak-to-strong transfer from GPT-2 to LLaMA-2 might not replicate for target models with different pre-training distributions, tokenizers, or architectural inductive biases. A model pre-trained primarily on code or multilingual data might produce different gradient dynamics for instruction–response pairs than LLaMA-2, and GPT-2's G-SNR ranking might not transfer.
Objective mismatch: G-SNR is computed from the next-token prediction loss gradients. If the downstream training objective is different — e.g., DPO uses a preference loss that depends on pairs of responses, or RLHF uses a policy gradient — it is unknown whether examples selected for their next-token prediction gradient properties remain optimal. An example that teaches clear instruction–response mapping (high G-SNR for next-token prediction) might be neutral or harmful for preference optimization if it teaches a response style that human raters disfavor.
Dataset scale: At 52,000 examples, both Alpaca and Alpaca-GPT4 are large enough to justify data selection but small enough that random subsets of 15% (7,800 examples) sometimes work reasonably well. On much larger datasets (200K+ examples), the noise and redundancy that motivate GRADFILTERING are more extreme, but the proxy ensemble training cost also scales — 2 epochs on 200K examples with 5 ensemble members is 4× the cost of the current setup. Whether GRADFILTERING's advantages over Superfiltering grow (because gradient dynamics bring more value when noise is higher) or shrink (because the proxy training cost dominates) is unknown.
Capability preservation: Instruction tuning is known to sometimes degrade performance on standard NLP benchmarks (the "alignment tax"). If GRADFILTERING discards 90% of the training data, it might inadvertently discard examples that are important for maintaining broad capabilities — e.g., factual knowledge, reasoning, or code generation — even if they are less central to instruction-following per se. The paper's evaluation only measures instruction-following quality, not whether the model retains its pre-trained capabilities.

What evidence exists in the paper. None that addresses these gaps directly. The paper evaluates on two datasets within the same distribution family (both are GPT-distilled instruction sets, varying primarily in the quality of the distilling teacher). The two model sizes tested (7B and 13B) are both from LLaMA-2 — a valuable comparison for checking intra-family transfer but not for establishing cross-family generality. The paper acknowledges in Section 6 that "we instantiate and evaluate GRADFILTERING only on supervised instruction tuning, but we argue that the same G-SNR principle can naturally extend to other objectives (e.g., multi-task mixtures), which we leave for future work" — this is an honest scoping statement but also confirms that the "objective-agnostic" claim is aspirational beyond the tested setting.

Mitigation status. Partially acknowledged for objective scope (Section 6 acknowledges the limitation to supervised instruction tuning), but not for model family, dataset size, or capability preservation. The paper's Limitations section in the main text (bottom of page 9) states that "GRADFILTERING has several limitations" but lists only gradient direction neglect, snapshot scheme sensitivity, proxy training cost, and delayed credit assignment — none of which address the single-model-family, single-dataset, single-task evaluation scope. A practitioner reading the abstract's claim of "strong empirical gains" needs to understand that these gains are demonstrated within a narrow envelope; the paper does not provide evidence that they generalize beyond it.

G-SNR Ignores Gradient Direction, Potentially Missing Complementarity and Diversity Signals

The assumption or constraint. GRADFILTERING's utility score is computed entirely from gradient norms (the magnitudes of per-example gradient vectors) and their statistics across ensemble members and epochs. The paper explicitly states in Section 7 (Limitations) that GRADFILTERING "operates on gradient norms and their variance, ignoring gradient direction; examples that are locally uninformative in norm but crucial for aligning with rare or long-horizon behaviors may be under-valued." This is a structural limitation of the method, not an implementational shortcut — using only norms reduces the computational and storage requirements from storing full gradient vectors (potentially millions of entries per example per member per epoch) to storing a single scalar per example per member per epoch, but at the cost of discarding all information about which direction in parameter space the gradient points.

The consequence. By ignoring gradient direction, G-SNR cannot detect two important properties of training data that are visible in the full gradient structure:

Gradient diversity and complementarity: Two examples might have identical gradient norms across epochs and ensemble members (hence identical G-SNR scores), but their gradient directions might be orthogonal — they push the model parameters in completely different directions, teaching complementary skills. G-SNR would rank them similarly, potentially selecting both if their scores are high or neither if they're moderate. A direction-aware method could identify that selecting both provides more coverage of the parameter space than selecting two examples with similar gradient directions. This is particularly relevant for instruction tuning, where the goal is broad instruction-following capability across diverse prompt types — coverage matters as much as individual example quality.
Gradient interference: Two examples might have high G-SNR scores individually but have gradient directions that are anti-correlated — learning from one makes the model worse at the other. This is a known phenomenon in multi-task learning where gradient conflicts cause optimization difficulties. G-SNR cannot detect such conflicts because it only sees how strongly each example pulls, not which way. A subset selected purely by G-SNR might accidentally concentrate on examples with mutually interfering gradients, degrading final performance relative to a subset that balances gradient directions.

The paper acknowledges the first issue (under-valuing "examples that are locally uninformative in norm but crucial for aligning with rare or long-horizon behaviors") but does not discuss the second (gradient interference). Both are concrete failure modes that a direction-aware method could potentially avoid.

What evidence exists in the paper. The ablation in Table 2 shows that using only gradient norms (as G-SNR does) outperforms simpler norm-based alternatives, but this doesn't test whether incorporating direction would help further — there is no comparison to a direction-aware utility. The paper does not report any analysis of gradient direction similarity among high-G-SNR examples, so we don't know whether GRADFILTERING-selected subsets are directionally diverse or concentrated. The convergence advantage in Figure 3 is consistent with G-SNR selecting examples that produce efficient descent (which directional alignment would promote) but doesn't distinguish between the norm-based and direction-based contributions to that efficiency — it might be that high-G-SNR examples happen to have well-aligned gradient directions as a side effect of being "clean" examples, in which case the direction neglect is less harmful than it could be, but this is speculation without evidence.

Mitigation status. The paper explicitly acknowledges this limitation as the first item in its Limitations section: "First, it operates on gradient norms and their variance, ignoring gradient direction; examples that are locally uninformative in norm but crucial for aligning with rare or long-horizon behaviors may be under-valued." This is a fair and honest admission, but the paper does not analyze how severe the problem is in practice — e.g., by computing the pairwise cosine similarity of gradients among top-G-SNR examples to see whether they cluster directionally, or by comparing against a toy direction-aware utility to estimate the ceiling of what directional information could add. The paper also does not propose directions for incorporating gradient direction while maintaining computational efficiency (e.g., through low-rank approximations of the gradient outer product, or learned projections of gradient vectors), leaving this as an open problem for future work with no concrete path forward suggested.

The Early/Late Snapshot Scheme (Epochs 1–2) and Ensemble Size (M=5) Are Chosen Empirically and Not Ablated, Making Their Optimality Unknown Across Settings

The assumption or constraint. The G-SNR utility depends on two critical hyperparameters that control the quality and cost of the gradient dynamics signal: (a) the choice of early and late snapshots $s=1$ and $t=2$ , justified by the observation in Figure 1 that "the trajectories of the LoRA ensemble members change substantially between epochs 1 and 2, and then remain relatively stable afterwards," and (b) the ensemble size $M=5$ , which the paper states is a practical choice but does not justify through ablation or cost-benefit analysis. The paper provides no experiments testing whether different snapshot pairs (e.g., epochs 1→3, 1→4, 2→3) produce better or worse G-SNR rankings, nor whether a larger ensemble (e.g., $M=10$ ) would improve uncertainty estimates enough to justify the additional proxy training cost, nor whether a smaller ensemble (e.g., $M=3$ ) would suffice with minimal degradation. These choices are baked into every result in Tables 1–2 and Figure 3, making it impossible to assess how sensitive the method's performance is to these parameters.

The consequence. The choice of snapshots and ensemble size represents a hidden cost-quality trade-off that practitioners cannot navigate without additional experiments. Specifically:

Snapshot timing: The paper observes that ensemble trajectories stabilize after epoch 2 on Alpaca and Alpaca-GPT4 with GPT-2. But on a larger or more diverse dataset, or with a different proxy model, the trajectory might stabilize later (epoch 3, 5, or 10). If a practitioner uses $s=1, t=2$ on such a dataset, the "late" snapshot would capture mid-training dynamics where the gradient drop is still in progress, producing G-SNR scores that are noisy underestimates of the true learning progress. The method provides no diagnostic to detect this — a practitioner would need to replicate the t-SNE analysis in Figure 1 for their specific dataset and proxy, which requires engineering effort and interpretational judgment beyond what the paper documents. Conversely, on a very easy dataset where learning saturates in epoch 1, waiting until epoch 2 to take the "late" snapshot adds unnecessary proxy training cost.
Ensemble size: The paper uses $M=5$ and reports that the ensemble trajectories show "non-degenerate" diversity (Figure 1), but does not quantify how much diversity is needed for reliable G-SNR estimates. With $M=5$ , the variance estimate $V^{(t)}_i$ (equation 4) is based on only 5 samples per example, making it a high-variance estimator itself — an example might appear to have low ensemble disagreement simply because the 5 randomly initialized members happened to align, not because the example is genuinely unambiguous. A larger ensemble would produce more reliable variance estimates but at linear cost in proxy training time; a smaller ensemble would be cheaper but might produce G-SNR rankings that are less stable across different ensemble initializations. Without an ensemble-size ablation, a practitioner cannot make this trade-off.
Interaction between the two: The optimal snapshot pair might depend on ensemble size. A larger ensemble might reach consensus faster (requiring fewer epochs) or might maintain diversity longer (requiring more epochs to see meaningful variance reduction). The paper's fixed $M=5, T=2$ configuration is a single point in a two-dimensional hyperparameter space, and there is no evidence that it is near-optimal or even representative.

What evidence exists in the paper. The t-SNE visualizations in Figure 1 provide qualitative justification for the $T=2$ choice by showing that trajectories change substantially between epochs 1 and 2 and then stabilize. This is suggestive but not dispositive — it shows that gradient profiles change between epochs 1 and 2, but doesn't show that the epoch-2 gradient statistics produce better G-SNR rankings than epoch-3 or epoch-4 statistics. The paper does not report any correlation between G-SNR rankings computed from different snapshot pairs, which would directly test robustness. For ensemble size, there is no justification beyond stating $M=5$ as a practical choice in Section 3.2.

Mitigation status. Partially acknowledged for ensemble size (the paper notes it is a "practical" choice in Section 3.2) and snapshot timing (Section 3.2 justifies $T=2$ from Figure 1), but not for the sensitivity of results to these choices. The Limitations section acknowledges that "G-SNR relies on a specific early/late snapshot scheme and a modest ensemble size ( $M=5$ ); although our t-SNE analyses indicate that these settings already induce diverse trajectories, different training schedules or proxies could change the behavior" — this is an honest admission but stops short of quantifying the risk. The paper does not ablate either hyperparameter, does not provide guidance on how to choose them for new datasets or proxies, and does not suggest diagnostic tools (like the t-SNE analysis) that practitioners could apply in their own settings. A practitioner adopting GRADFILTERING on a new dataset would need to replicate the t-SNE analysis from scratch to have any confidence in their snapshot choice, and would need to run their own ensemble-size ablation to balance cost and reliability — both of which require substantially more experimentation than simply running the method with the paper's default settings.

7. Implications and Future Directions

How This Work Changes the Landscape

GRADFILTERING introduces a conceptual reframing rather than a paradigm shift: it recasts data selection for instruction tuning as a gradient signal processing problem rather than a content-quality scoring problem. This is not a wholesale replacement of existing methods — Superfiltering remains competitive in many settings, and the paper does not claim to obsolete it — but it changes what practitioners pay attention to when curating instruction data. Before this work, the dominant mental model (embodied by Superfiltering, LESS, and perplexity-based filtering) was that data utility is an intrinsic property of each example, measurable by a forward pass through a proxy model or a similarity search against validation gradients. GRADFILTERING challenges this by demonstrating that data utility is better understood as an emergent property of training dynamics — revealed by how gradient magnitudes evolve across ensemble members and epochs, not by any single-snapshot assessment.

The specific shift is from static, point-estimate scoring toward dynamic, uncertainty-aware valuation. The evidence supporting this shift comes primarily from the ablation in Table 2: removing the uncertainty normalization (Alternative #2, the relative gradient drop without ensemble variance) consistently degrades performance relative to the full G-SNR, demonstrating that how consistently an example drives learning matters as much as how much it drives learning. The paper's naming of its utility as a "Signal-to-Noise Ratio" is more than rhetorical — it encodes the design principle that a good data selection criterion should jointly optimize for high signal (gradient drop) and low noise (ensemble agreement), and that neither component alone suffices.

This work also reconciles a latent tension in the instruction-tuning data selection literature between two goals that seemed orthogonal: being task-agnostic (not requiring a target validation set, as Superfiltering achieves) and being gradient-aware (using parameter-space information about how examples drive optimization, as LESS achieves but at the cost of task-specificity). GRADFILTERING demonstrates that these goals are not in tension — you can have gradient-based valuation without task-specificity by extracting the signal from the training dynamics themselves (gradient drop + ensemble uncertainty) rather than from alignment with a downstream task. This opens a middle path that prior work had not recognized: objective-agnostic gradient-based data selection. The implications for how researchers think about data valuation are significant: instead of choosing between cheap-but-static methods (Superfiltering) and expensive-but-gradient-based methods (LESS), the field can pursue methods that use gradient information from small proxies, keeping costs low while capturing the optimization-level dynamics that forward-pass-only methods miss.

The research directions that become more attractive after this work include:

Training-dynamics-based data curation more broadly — using gradient trajectories, loss landscapes, or ensemble disagreement to characterize data quality, not just for instruction tuning but for any fine-tuning regime where training data is noisy or abundant.
Uncertainty quantification for data pipelines — extending the LoRA ensemble idea to detect label errors, distribution shift, or annotation ambiguity in training corpora, treating the ensemble disagreement as a general-purpose data diagnostic.
Weak-to-strong dynamics transfer — the paper's implicit assumption that GPT-2 gradient dynamics transfer to LLaMA-2 opens the question of whether this transfer is systematic, predictable, and exploitable (can we train a model to predict which examples will transfer?).

Directions that become relatively less attractive (or at least require stronger justification):

Static single-score filtering alone — the ablation shows that pure relative gradient drop (essentially a learned, dynamic difficulty score) already captures much of G-SNR's power, but the additional gain from uncertainty normalization suggests that methods ignoring epistemic uncertainty leave performance on the table. A new static scoring method would need to demonstrate that it captures something beyond what G-SNR's dynamics-based approach already provides.
Full-scale gradient datastores for every application — GRADFILTERING shows that useful gradient information can be extracted from a small proxy model (GPT-2) rather than requiring gradient computation on the target model (as LESS does). This makes the cost case for full-scale gradient datastores harder to justify unless the application demands the highest possible selection quality and can amortize the storage cost.

The paper also establishes convergence speed as a distinct and measurable benefit of good data selection (Figure 3), separate from final model quality. Prior work on data selection for instruction tuning focused almost exclusively on final evaluation metrics (win rates, benchmark scores). GRADFILTERING demonstrates that the choice of training examples affects not just where the optimizer ends up but how efficiently it gets there. This is a practically important dimension that future data selection methods should report — faster convergence means lower training cost even at the same subset size, and a method that achieves the same final accuracy as a competitor but converges in half the steps has a genuine deployment advantage.

Follow-Up Research This Work Enables

1. Directly testing the weak-to-strong gradient dynamics transfer hypothesis. The paper assumes, but never verifies, that GPT-2's gradient dynamics (specifically, which examples show large, consistent gradient drops) correlate with LLaMA-2's learning dynamics. A direct test would: (a) compute G-SNR for all 52,000 Alpaca examples using both GPT-2 small and LLaMA-2-7B as the proxy (with identical LoRA ensemble settings), (b) compute the Spearman rank correlation between the two G-SNR rankings, and (c) measure whether the LLaMA-2-proxy G-SNR selections produce better downstream models than the GPT-2-proxy selections. If the correlation is high (say, ρ > 0.7) and the downstream performance gap is small (say, < 0.02 PWS difference), the transfer assumption is validated and practitioners can confidently use GPT-2. If the correlation is low or the performance gap is large, the paper's central efficiency claim — that a cheap proxy suffices — needs qualification: the proxy must be "close enough" to the target model in some specifiable sense (similar pre-training data? similar architecture? similar scale?). A multi-proxy scaling experiment (GPT-2 small → GPT-2 medium → GPT-2 large → LLaMA-2-7B) would reveal how selection quality improves with proxy capability and where diminishing returns set in, providing a cost-quality Pareto frontier that practitioners can use to choose their proxy budget.

2. Direction-aware G-SNR using low-rank gradient approximations. The paper's primary acknowledged limitation (Section 7) is that G-SNR uses only gradient norms, ignoring gradient direction. This matters for two reasons: gradient diversity (two examples with identical G-SNR might teach complementary skills if their gradient directions are orthogonal) and gradient interference (two high-G-SNR examples might have anti-correlated gradients that cancel each other's updates). A follow-up could extend G-SNR to incorporate directional information without requiring full gradient storage. One approach: instead of storing full per-example gradient vectors, store low-dimensional projections — e.g., the top-K singular vectors of the gradient matrix, or random projections (Johnson-Lindenstrauss) of the gradient vectors. The G-SNR utility would then include a diversity bonus: examples whose projected gradients span diverse directions in the low-dimensional space receive higher priority. The experiment would compare subset quality when selected by norm-only G-SNR vs. direction-aware G-SNR as a function of the projection dimensionality K (trading off storage cost against directional fidelity). If direction-aware selection substantially improves performance — especially on diverse evaluation sets like WizardLM that require broad instruction-following capability — it would justify the additional computation and storage. The paper's current results show that norm-only G-SNR already performs well, so the ceiling for directional improvement might be modest, but establishing this ceiling (even if it's small) would be a valuable contribution that bounds the importance of the paper's main limitation.

3. G-SNR for preference optimization objectives (DPO, RLHF). The paper evaluates G-SNR only on supervised instruction tuning with next-token prediction loss. But the dominant paradigm for aligning LLMs is now preference optimization — DPO, RLHF, or variants — where the training objective is not per-token likelihood but a contrastive or policy-gradient loss that depends on pairs of responses (chosen and rejected). It is unclear whether G-SNR computed from next-token prediction gradients transfers to these objectives. A direct experiment would: (a) compute G-SNR on a standard instruction dataset using next-token prediction gradients from the GPT-2 proxy (as in the paper), (b) select a subset, (c) use that subset to train a LLaMA-2 model with DPO (not SFT), and (d) evaluate on a standard preference benchmark (e.g., AlpacaEval, MT-Bench). If the G-SNR-selected subset works well for DPO, it suggests that "being informative for next-token prediction" and "being informative for preference learning" are correlated properties of instruction data — a non-obvious finding that would expand G-SNR's practical scope substantially. If it fails, it would reveal that G-SNR's "objective-agnostic" claim has a boundary: the utility is tied to the objective used to compute gradients, and switching objectives requires recomputing G-SNR with the new objective's gradients (or developing an objective-transfer method). The experiment could also test a stronger baseline: compute G-SNR using the preference loss gradients directly (by fine-tuning the GPT-2 proxy with a DPO objective rather than SFT), and compare this objective-matched G-SNR against the SFT-derived G-SNR for DPO downstream performance. This would distinguish between "G-SNR is objective-specific" (bad for the current claims) vs. "G-SNR captures objective-invariant properties of good instruction data" (good).

4. Online, adaptive G-SNR that adjusts selection during target model training. The paper's G-SNR is computed once from a proxy model and then fixed — the selected subset is static throughout target model training. But the insight that useful examples produce gradient drops suggests an alternative: online G-SNR, where the target model itself computes per-example gradient statistics during its own training, and examples are dynamically re-weighted or resampled based on their evolving gradient characteristics. Concretely: start fine-tuning LLaMA-2 on a random 20% subset; after each epoch, compute the gradient drop for each example between epochs e and e+1 (using the target model's own gradients, not a proxy); at epoch boundaries, drop examples with low G-SNR and add new examples from the remaining pool that are predicted to have high G-SNR (based on their gradient behavior in the initial epoch of their inclusion). This would test whether the dynamics-based selection principle works when applied to the target model directly, in an online curriculum learning setting. The experiment would compare: (a) static GRADFILTERING (proxy G-SNR → fixed subset → target model training), (b) online G-SNR as described, and (c) a baseline that uses random resampling. If online G-SNR matches or exceeds static GRADFILTERING, it demonstrates that the selection principle is robust and can be applied without a proxy at all — the target model's own early-training gradients contain sufficient signal to guide dynamic data selection, potentially eliminating the proxy training cost entirely. If it underperforms, it would reveal that the proxy's gradient dynamics capture something that the target model's early-training dynamics miss (perhaps because the target model learns too quickly or too slowly to produce informative gradient drops in a short window).

5. G-SNR as a diagnostic for instruction dataset quality and automated corpus cleaning. The paper shows that G-SNR can rank examples within a dataset, but the utility distributions themselves might contain valuable information about corpus quality. A follow-up could characterize the statistical properties of G-SNR score distributions across different instruction datasets — e.g., does Alpaca (GPT-3.5 distilled) have a different G-SNR distribution shape (mean, variance, skewness, tail behavior) than Alpaca-GPT4 (GPT-4 distilled) or than human-written instruction datasets? Do datasets known to be noisy (e.g., unfiltered web-scraped instructions) have systematically different G-SNR distributions — higher variance, fatter left tails (more examples with negative or zero gradient drop), or higher ensemble disagreement overall? If so, G-SNR could serve as a dataset-level quality diagnostic: before fine-tuning, a practitioner computes the G-SNR distribution for their instruction corpus and uses summary statistics to flag potential quality issues (e.g., "30% of your examples have negative gradient drop — this corpus may be unusually noisy"). A concrete experiment: collect a benchmark of instruction datasets with known quality labels (human-annotated for noise, redundancy, and helpfulness), compute G-SNR distributions for each, and train a simple classifier to predict dataset quality from G-SNR summary statistics. If the classifier achieves non-trivial accuracy, G-SNR becomes a tool not just for example selection but for corpus-level quality assessment, which is currently done through expensive manual inspection or downstream trial-and-error.

6. G-SNR for multi-turn conversation and long-form generation data. The paper's experiments use single-turn instruction–response pairs (Alpaca format). But modern instruction tuning increasingly uses multi-turn conversations (e.g., ShareGPT, OpenHermes) where each example is a dialogue with multiple user-assistant exchanges. Extending G-SNR to this setting is non-trivial because the gradient for a multi-turn example is the sum of per-turn gradients, and the "gradient drop" signal would conflate learning progress on early turns (which the model might master quickly) with learning progress on later turns (which require multi-turn coherence skills). A follow-up could decompose G-SNR per turn: compute per-turn gradient norms, track per-turn gradient drops, and use the minimum or harmonic mean across turns as the example-level G-SNR (so that an example is only considered "learned" if all its turns show gradient reduction). The experiment would compare per-turn G-SNR against naive full-example G-SNR on a multi-turn instruction dataset (e.g., ShareGPT filtered to 52K examples for direct comparability with the paper's setting), evaluating on multi-turn evaluation benchmarks like MT-Bench. If per-turn G-SNR outperforms, it demonstrates that the gradient dynamics principle extends naturally to structured outputs with appropriate decomposition. If it underperforms, it reveals a fundamental limitation: G-SNR's gradient-drop heuristic assumes examples are atomic learning units, and this assumption breaks for composite examples where different components are learned at different rates.

Practical Applications and Downstream Use Cases

1. Cost-efficient batch fine-tuning for multi-task model deployments. An organization maintaining multiple instruction-tuned model variants for different product surfaces (e.g., a coding assistant, a writing helper, a customer support agent) currently faces a choice: fine-tune a single model on a massive combined dataset (expensive, risks negative transfer between tasks) or fine-tune separate models on task-specific subsets (requires curating those subsets, which is manual or heuristic). GRADFILTERING offers a third path: run the G-SNR pipeline once on the combined dataset to produce a universal ranking of all instruction–response pairs by their gradient-based utility, then select task-specific subsets by intersecting the top-k G-SNR examples with task-relevant filters (e.g., keyword matching, embedding similarity). The key practical benefit is that the expensive proxy ensemble training is amortized across all downstream fine-tuning runs — you pay the GPT-2 training cost once, then reuse the G-SNR ranking to curate subsets of any size for any task, with the confidence (from Table 1) that models trained on 5–15% G-SNR-ranked data match or exceed full-data performance. For an organization running weekly model updates, the proxy training cost (perhaps a few GPU-hours on a single A100 for GPT-2 with LoRA ensemble) is negligible compared to the savings from training target models on 5–15% of the data instead of 100%, each week, across multiple tasks. The paper's result that G-SNR-selected 5% subsets on Alpaca LLaMA2-7B (Full) achieve PWS = 1.16 — outperforming the full-data model — suggests these savings come without quality degradation.

2. Data cleaning and deduplication for open-source instruction dataset releases. Many open-source instruction datasets (e.g., OpenOrca, Dolly, self-instruct corpora) are released with millions of examples, but end users — researchers or small companies with limited compute — often cannot afford to fine-tune on the full dataset and lack principled ways to subsample. GRADFILTERING could be run once by the dataset releasers and distributed alongside the dataset as a "quality ranking" file: a single column of G-SNR scores, one per example, computed by the dataset curators using a standard proxy model (e.g., a specified GPT-2 checkpoint and LoRA configuration). Downstream users could then select the top-k fraction that fits their compute budget with confidence that the ranking is grounded in training dynamics evidence rather than ad-hoc heuristics. This transforms dataset release from "here are 1M examples, good luck" to "here are 1M examples ranked by gradient-based utility; we've validated that the top 50K match full-dataset performance on representative evaluations." The paper's result that G-SNR transfers from GPT-2 to LLaMA-2 (at least for the tested configurations) provides initial evidence that a single proxy-based ranking could serve many downstream model families, though further testing on non-LLaMA models would be needed for the claim to generalize. The practical infrastructure needed is modest: a script that runs the GPT-2 LoRA ensemble training and outputs per-example G-SNR scores, which dataset curators could integrate into their release pipelines.

3. Warm-start data selection for continual fine-tuning and model updates. When an instruction-tuned model needs to be updated — because new user feedback reveals failure modes, because the domain shifts, or because new capabilities are needed — practitioners typically fine-tune on a combination of new data and a replay buffer of old data to prevent catastrophic forgetting. Selecting which old examples to include in the replay buffer is currently heuristic (random sampling, uniform subsampling). GRADFILTERING provides a principled criterion: include old examples with high G-SNR scores, because these are the examples that the model genuinely learned from (large, consistent gradient drops) and that are therefore most at risk of being forgotten. The experiment would: (a) train a base model on the full Alpaca dataset, (b) select a replay buffer using G-SNR (top-10% by G-SNR) vs. random (random 10%) vs. no replay, (c) fine-tune on a new instruction dataset (e.g., Alpaca-GPT4, representing a domain shift), and (d) evaluate both on the new task (Alpaca-GPT4 evaluation) and the old task (Alpaca evaluation) to measure forward transfer and forgetting. If G-SNR-selected replay buffers reduce forgetting more than random buffers at the same size, it demonstrates that gradient dynamics identify not just which examples are "good" in isolation, but which examples created lasting parameter changes that need reinforcement — a practically valuable property for any system that undergoes iterative updates.

4. Lightweight data valuation for model auditing and transparency. As regulation and organizational policy increasingly require documentation of training data provenance and quality, practitioners need tools to audit their instruction-tuning corpora — to answer questions like "which examples in our training set most influenced the model's behavior on this specific type of query?" GRADFILTERING provides a lightweight, proxy-based alternative to full influence functions for this purpose. By computing G-SNR for all training examples and correlating high-G-SNR examples with downstream evaluation performance (as the paper implicitly does), a practitioner can produce a data influence report: the top-K training examples ranked by G-SNR, with qualitative inspection to verify that they represent desired model behaviors. This is not as rigorous as per-example influence functions (which measure impact on specific test predictions), but it is dramatically cheaper — the GPT-2 LoRA ensemble training is a one-time cost rather than the per-query Hessian-vector products that influence functions require — making it feasible for routine auditing rather than only for high-stakes investigations. The paper's finding that G-SNR-selected subsets produce human-preferred responses (Section 5.3, with 44/19/37 win/tie/lose for Alpaca at 10% subset) provides some evidence that high-G-SNR examples correlate with desirable downstream behavior, supporting the auditing use case.

When to Prefer This Method

GRADFILTERING is most appropriate when:

You need task-agnostic data selection: The downstream evaluation is broad instruction following (like the paper's WizardLM and Vicuna evaluations), not a specific narrow task with a predefined validation set. If you had a clean validation set, LESS's gradient similarity search might produce better-targeted subsets (though the paper doesn't test this head-to-head).
Your proxy training cost amortizes well: You plan to reuse the selected subset many times (multiple model versions, multiple adaptation runs, or as a released dataset with quality rankings), so the GPT-2 LoRA ensemble training cost is negligible per use. If you're doing a single one-off fine-tuning run, Superfiltering's forward-pass-only scoring is cheaper and often competitive (Table 1 shows Superfiltering close to or slightly ahead of GRADFILTERING in many LoRA settings).
You are doing full-parameter fine-tuning: Table 1 shows GRADFILTERING's largest advantages over Superfiltering in Full fine-tuning settings (e.g., Alpaca LLaMA2-7B Full 5%: GRADFILTERING 1.16 vs. Superfiltering 1.05). This is consistent with the paper's claim that "optimization dynamics and sample interactions are more pronounced" when all parameters are updated, making gradient-based selection more valuable.
You care about convergence speed as well as final quality: Figure 3 demonstrates that GRADFILTERING-selected subsets train faster than Superfiltering-selected subsets at identical step counts. If training time or compute budget is the binding constraint (rather than just final model quality), this convergence advantage provides a distinct benefit beyond the pairwise preference scores.
Uncertainty quantification matters for your use case: If your dataset is known to be noisy or contains out-of-distribution examples (e.g., web-scraped instructions with minimal filtering), G-SNR's variance penalty provides a principled way to down-weight unreliable examples that static scoring methods cannot distinguish from genuinely hard-but-clean examples.

Superfiltering remains preferable when:

Proxy training cost cannot be amortized (single-use fine-tuning, tight deadline, no GPU availability for even a small proxy ensemble run).
LoRA-based target model fine-tuning (where Table 1 shows the two methods are often within 0.01–0.05 PWS, making GRADFILTERING's additional proxy cost harder to justify).
You want the simplest possible pipeline (forward passes through a single GPT-2 model vs. training a 5-member LoRA ensemble for 2 epochs with per-example gradient collection — Superfiltering is operationally simpler to implement and debug).