Fast and Accurate Causal Parallel Decoding using Jacobi Forcing

ArXiv: 2512.14681

🎯 Pitch

You can turn autoregressive LLMs into parallel decoders that run nearly 4× faster without retraining from scratch—just teach the model to fix its own mistakes using Jacobi trajectories. No bidirectional attention, no masked data, and no quality drop.

---

1. Executive Summary

This paper introduces Jacobi Forcing, a progressive distillation paradigm that transforms standard autoregressive (AR) language models into efficient parallel decoders without modifying their causal attention mechanism, by training models on their own Jacobi decoding trajectories under a progressively increasing noise schedule and a noise-conditioned causal mask. Evaluated on coding benchmarks (HumanEval, MBPP) using Qwen2.5-Coder-7B-Instruct and math benchmarks (GSM8K, MATH) using Qwen2.5-Math-7B-Instruct, the resulting Jacobi Forcing Model achieves up to 3.8× wall-clock speedup over AR decoding with minimal accuracy degradation—substantially outperforming both diffusion language model baselines (by 5.3–7.4×) and prior consistency-distilled AR decoders. Two complementary inference optimizations—rejection recycling (reusing high-quality n-grams discarded during Jacobi iterations as candidate drafts) and multi-block decoding (maintaining and refining multiple blocks simultaneously, decoding future blocks even when preceding blocks remain unconverged)—further boost token acceptance per iteration to 4.5× and overall speedup to nearly 4.0×, establishing that AR models can be trained as competitive parallel decoders without the pretrain-to-posttrain mismatch introduced by bidirectional attention or masked-data objectives, provided the noise schedule controls the length of noisy context dependencies during training.

2. Context and Motivation

The Bottleneck: Autoregressive Decoding Limits Parallelism

The fundamental performance bottleneck in large language model inference is that autoregressive (AR) decoding generates tokens one at a time, sequentially. As formalized in Equation 1, a model with parameters $θ$ produces a response of length $n$ by computing $n$ separate forward passes through the entire network:

$y_i = \arg\max_y \; p_{\theta}(y \mid \boldsymbol{y}_{<i}, \boldsymbol{x}), \quad \text{for } i=1,\dots,n$

Each token depends on all previous tokens, so the model must wait for position $i-1$ to finish before it can start computing position $i$. The cost of this is not primarily in FLOPs — modern GPUs have massive compute capacity — but in memory bandwidth and the inability to batch operations across sequence positions. Each forward pass reads the model's full weight matrix from memory but only produces a single new token, resulting in low arithmetic intensity and poor hardware utilization. As models scale to hundreds of billions of parameters and applications demand increasingly long generations (complex reasoning chains, multi-step coding tasks, agentic workflows), this sequential bottleneck dominates end-to-end latency.

Why This Matters: The Practical and Theoretical Stakes

Practical impact. If an LLM could generate multiple correct tokens in a single forward pass, the arithmetic intensity would increase proportionally: reading the weights once to produce $k$ tokens instead of one. This directly translates to lower user-facing latency in interactive applications (chat assistants, code completion, real-time translation), lower cost per token in batch inference pipelines, and better hardware utilization on accelerators that are increasingly optimized for throughput rather than single-token latency. The paper frames this explicitly in terms of modern hardware trends: "modern AI accelerators whose abundant FLOPs could otherwise be leveraged to decode more future tokens per iteration" (Section 1, paragraph 2). In other words, the hardware is already provisioned for higher throughput — the algorithm just cannot exploit it.

Theoretical significance. Multi-token decoding is not merely an engineering optimization. It challenges a core assumption of language modeling: that left-to-right causality is the only viable inductive bias for coherent generation. Finding a way to predict future tokens jointly — without degrading generation quality below that of sequential AR decoding — would reshape our understanding of how causal structure interacts with model capacity and training objectives. The paper's approach (training AR models to predict tokens under noisy context without modifying the causal mask) is particularly interesting because it suggests that parallel generation capability can be learned as a skill without altering the model's fundamental architecture, treating future-token prediction as a form of denoising rather than a departure from causality.

Prior Approaches and Their Shortcomings

The paper identifies three families of prior work on multi-token generation, each with specific limitations that Jacobi Forcing is designed to address.

1. Diffusion Language Models (dLLMs)

Diffusion-based language models abandon left-to-right causality entirely. They model the entire sequence jointly — typically by learning a denoising process that starts from fully masked tokens and iteratively refines them into coherent text. Because all positions can be denoised in parallel, dLLMs offer the promise of high-throughput generation. The paper acknowledges this:

"dLLMs… relax left-to-right generation by modeling the entire sequence jointly and decoding via full-sequence denoising. This, in turn, enables highly parallelizable computation." (Section 1, paragraph 1)

However, open pretrained dLLMs suffer from two interconnected problems:

Weaker generation quality due to training objective mismatch. dLLMs are trained with a negative evidence lower bound (NELBO) objective, which the paper characterizes as "a loose bound on AR's negative log-likelihood (NLL) that is proven less efficient" (Section 1, paragraph 1, citing cheng2025sdar, niescaling, arriola2025block). This means that even at equivalent model scale, the diffusion training formulation is less sample-efficient than AR training — the model learns less per training token — resulting in weaker final performance at comparable compute budgets. As the paper notes, "open pretrained dLLMs underperform AR models in generation quality" (Section 1, paragraph 1, citing Dream-7B, LLaDA-15, and Large Language Diffusion Models).

Pretrain-to-posttrain mismatch in AR-to-dLLM adaptation. A natural workaround is to take a high-quality pretrained AR model and convert it into a dLLM through post-training, thereby inheriting the AR model's superior pretrained knowledge. This is exactly what recent methods like SDAR (cheng2025sdar) and fast-dLLM (wu2025fast_dllm) attempt. The procedure involves:

1. Taking pretraining data and applying block-wise perturbations (randomly masking tokens) following the dLLM recipe.
2. Modifying the attention mask from causal to block-wise bidirectional (tokens within a block attend to all other tokens in that block).
3. Replacing the training objective from NLL to NELBO.

The paper identifies this as a pretrain-to-posttrain mismatch with two distinct components:

• Bidirectional attention conflicts with the causal prior. The AR model was pretrained with strictly causal attention — each token can only attend to previous tokens. When suddenly forced to process bidirectional context within blocks during post-training, the model's learned representations are disrupted. The paper notes that "SDAR suffers substantial quality drops when large block sizes (e.g., 64 or 128) are adopted" (Section 1, paragraph 2). This is not a minor fine-tuning issue; it reflects a fundamental distribution shift in what attention patterns the model was optimized for.

• Masked data distribution deviates from natural data. During post-training, the model sees randomly masked tokens that never appeared in its pretraining data. The paper describes this as a "sharp deviation from the natural data distribution seen during pretraining, making the adaptation difficult to learn" (Section 1, paragraph 2). The model essentially has to learn a new denoising skill on top of its language modeling skill, and large block sizes make this harder because the proportion of masked tokens increases.

The consequence is captured in Figure 1: AR-adapted dLLMs are "costly to train" and "fail to scale speedup reliably with larger block sizes." As block size increases, the paper reports that these models cannot consistently decode more correct tokens per iteration — the quality drops faster than the throughput increases, "thereby underutilizing modern AI accelerators" (Section 1, paragraph 2).

2. Jacobi Decoding and Consistency Distillation (CLLMs)

Jacobi decoding reframes AR generation as solving a fixed-point iteration. Instead of generating one token at a time, we initialize a block of $n$ random tokens and iteratively update all positions in parallel using the AR model:

$\{y_1^{(j+1)} = \arg\max_y p_{\theta}(y \mid \boldsymbol{x}), \quad y_2^{(j+1)} = \arg\max_y p_{\theta}(y \mid y_1^{(j)}, \boldsymbol{x}), \quad \dots\}$

At each iteration $j$, the model processes all $n$ positions in a single forward pass (using a causal attention mask so that position $i$ sees the previous iteration's tokens at positions $<i$). The iteration converges when $\boldsymbol{y}^{(k)} = \boldsymbol{y}^{(k-1)}$, and it is provable that the converged fixed point matches greedy AR decoding (song2021accelerating; santilli2023accelerating).

The key problem: "Jacobi decoding achieves little speedup over standard AR decoding, as it rarely predicts more than one correct token within one fixed-point iteration" (Section 2.2). Even though all $n$ positions are computed in parallel, only the first few tokens (often just one) actually match the final fixed point at each step. The rest are discarded and recomputed. This means the wall-clock speedup is marginal despite the parallel forward pass.

CLLMs address this through consistency distillation (kou2024cllms_consistency_large_language_models). The idea is to train the model to map any point along a Jacobi trajectory directly to the converged fixed point. Given a Jacobi trajectory $\mathcal{J} = \{\boldsymbol{y}^{(0)}, \dots, \boldsymbol{y}^{(k)}\}$ where $\boldsymbol{y}^{(k)} = \boldsymbol{y}^*$ is the fixed point, the consistency loss encourages the model's output distribution when conditioned on the noisy intermediate point $\boldsymbol{y}_{B_i}$ to match the teacher's output distribution when conditioned on the clean fixed point $\boldsymbol{y}_{B_i}^*$:

$\mathcal{L}_c = \mathbb{E}_{i \sim \mathcal{U}\{1,\dots,N\}, \boldsymbol{y}_{B_i} \sim \mathcal{J}_i} \left[ D_{KL}\left( p_{\theta^-}(\boldsymbol{y}_{B_i}^* \mid \dots) \;\|\; p_{\theta}(\boldsymbol{y}_{B_i} \mid \dots) \right) \right]$

The intuition: if the model learns to predict the correct next token (the fixed point's token) even when conditioned on noisy/unconverged previous tokens, then more tokens will be correct in early Jacobi iterations and convergence will accelerate. CLLMs achieve up to 2× speedup over AR decoding with minimal quality loss.

But CLLMs hit a scaling wall with larger block sizes. The paper identifies the root cause: as block size $n$ increases, predicting future tokens becomes increasingly difficult because they must be conditioned on longer spans of noisy (unconverged) context. Consider Equation 6:

$y_f = \arg\max_y \; p(y \mid \boldsymbol{x}_c, \boldsymbol{y}'_{c:f-1}), \quad \text{for } f = c+1,\dots,n$

Here, $\boldsymbol{x}_c = [\boldsymbol{x}, \boldsymbol{y}_{<c}]$ is the clean context (tokens already accepted) and $\boldsymbol{y}'_{c:f-1}$ is the noisy context (unconverged tokens from the current iteration). When $n$ is large and the model is early in the Jacobi trajectory, $\boldsymbol{y}'_{c:f-1}$ can be a long sequence of incorrect tokens. The model must predict $y_f$ correctly despite conditioning on many wrong preceding tokens — a fundamentally hard task. The paper notes that "CLLMs face a similar limitation as AR-adapted dLLMs: as block size increases, the number of tokens correctly decoded per iteration remains essentially constant" (Section 1, paragraph 3).

3. Speculative Decoding

The paper explicitly positions Jacobi Forcing as distinct from speculative decoding approaches, noting that these methods "don't serve as parallel decoders without supplemental architecture modifications (e.g., via additional heads) or separate draft models" (Section 4.1). Methods like Medusa (cai2024medusa), EAGLE/EAGLE-2/EAGLE-3 (li2024eagle; li2024eagle2; li2025eagle3), and standard speculative decoding (leviathan2022speculative_decoding) all require either an auxiliary draft model, additional prediction heads, or feature-level reuse mechanisms. Jacobi Forcing aims to make the base model itself a parallel decoder without these additions.

How Jacobi Forcing Positions Itself

The paper positions Jacobi Forcing as resolving a specific tension: how to train AR models to generate multiple correct tokens per iteration while maintaining causal attention and avoiding the pretrain-to-posttrain mismatch of dLLM adaptation. The key innovations that distinguish it:

1. No attention mask modification. Unlike dLLM adaptation methods (SDAR, fast-dLLM) that switch to bidirectional attention, Jacobi Forcing preserves the causal mask throughout training. The model always processes tokens left-to-right, even when conditioning on noisy context. This means there is no disruption to the pretrained attention patterns — the model is simply learning a denoising skill within the causal framework it already knows.

2. Progressive noise schedule addresses the long-noisy-context problem. CLLMs fail at large block sizes because they condition token predictions on long spans of noisy context (Equation 6). Jacobi Forcing breaks a large block into smaller sub-blocks and applies a cyclic progressive noise schedule where the noise ratio $t_i$ (fraction of noisy tokens in sub-block $i$) increases linearly from 0 to 1 across each window of size $w$:

   $W = \left\{0, \frac{1}{w}, \dots, \frac{w-1}{w}\right\}, \quad t_i = W[j], \quad j = i \bmod w$

   This reduces the longest span of consecutive noisy tokens from $O(nN)$ (if all blocks were fully noisy, as in the random schedule) to $O(\lceil tn \rceil)$ — a dramatic reduction that makes the prediction task learnable. The paper empirically verifies that this progressive schedule "significantly outperforms" both random and reverse progressive schedules (Table 4).

3. Noise-conditioned causal attention. Unlike CLLMs, which condition each noisy block only on clean previous blocks (Figure 2a), Jacobi Forcing uses a noise-conditioned mask where each block conditions on the actual noisy states of previous blocks (Figure 2b). This means the model learns to predict correctly conditioned on partially incorrect context — exactly what it will encounter during inference. The ablation in Table 5 confirms that this noise-conditioned mask is more effective than the clean-context alternative at producing speedup while maintaining quality.

4. Self-generated progressive distillation. Jacobi Forcing is iterative: after training on trajectories from the original AR model, the paper generates new trajectories using the already-trained Jacobi Forcing Model itself (with progressively larger block sizes) and trains on those. This creates a self-reinforcing cycle where the model learns from its own improved parallel decoding behavior, yielding an additional 20% speedup with "only minor performance degradation" (Section 3.1, final paragraph). This departs from CLLMs, which train on a single round of trajectories from the original AR model.

5. Inference-time optimizations exploit learned behavior. The paper observes that Jacobi Forcing Model produces qualitatively different Jacobi trajectories compared to CLLMs: "fixed-point segments emerge within the noisy tokens of the unconverged point" and "these segments progressively extend, even under noisy context" (Section 3.2). This property — longer runs of correct tokens appearing in later positions even when earlier positions are still noisy — is what enables the rejection recycling and multi-block decoding optimizations. These optimizations are not generic; they are specifically designed to exploit the characteristic that Jacobi Forcing training produces.

Summary: The Gap Jacobi Forcing Fills

The paper addresses a specific technical gap: prior work either abandoned causal attention entirely (dLLMs, suffering pretrain-to-posttrain mismatch and quality degradation) or preserved causal attention but could not scale to large block sizes (CLLMs, hitting a wall because long noisy contexts made prediction too hard). Jacobi Forcing preserves the causal attention that AR models were pretrained with — avoiding the mismatch — while using a progressive noise schedule and noise-conditioned mask to make large-block parallel prediction learnable. This enables the model to serve as a standalone parallel decoder without auxiliary drafters, modified attention masks, or separate denoising objectives, achieving speedups that neither prior approach could reach.

3. Technical Approach

3.1 Reader Orientation

This paper develops a training paradigm — not a new architecture — that progressively distills a standard autoregressive (AR) language model into a parallel decoder that can generate multiple correct tokens in a single forward pass without modifying the model's causal attention or training objective. The core problem is that prior methods for multi-token generation either abandon the causal prior (diffusion language models, introducing catastrophic pretrain-to-posttrain mismatch) or preserve it but fail to scale beyond small block sizes (CLLMs, because predicting tokens conditioned on long noisy contexts is too difficult). Jacobi Forcing solves this by constructing a curriculum of training sequences where the model learns to predict the correct next tokens while conditioned on progressively longer spans of noisy (unconverged) predecessors, controlled by a cyclic noise schedule that caps the difficulty of any single prediction, and it couples this with a noise-conditioned causal attention scheme that exactly mirrors the conditions the model will face during inference.

3.2 Big-Picture Architecture (Diagram in Words)

The Jacobi Forcing pipeline has four major stages that operate in a cycle:

1. Trajectory generation. The current model (initially the base AR model, later the partially-trained Jacobi Forcing Model) runs Jacobi decoding on training prompts to produce Jacobi trajectories — sequences of intermediate states that evolve from a random initialisation toward the converged fixed point (which matches greedy AR decoding). Each trajectory records, for each block of $n$ tokens, every intermediate point $\boldsymbol{y}^{(j)}$ and the final converged point $\boldsymbol{y}^*$.

2. Noise schedule mapping and sequence packing. A progressive noise schedule assigns to each block $i$ in the trajectory a target noise ratio $t_i \in [0, 1]$ (fraction of tokens that should be unconverged/noisy). For each block, the trajectory point whose actual fraction of unconverged tokens is closest to $t_i$ is selected as the noisy block $\boldsymbol{y}_{B_i}^{t_i}$. The corresponding clean block $\boldsymbol{y}_{B_i}^*$ is the fixed point. All noisy and clean blocks across the full response are interleaved into a single training sequence, and a custom sparse attention mask is constructed so that each noisy block conditions on the actual noisy states of preceding blocks (noise-conditioned) while each clean block conditions on preceding clean blocks (standard causal).

3. Progressive distillation training (Jacobi Forcing loss). The model processes this packed sequence in a single forward pass. The training objective combines (a) a progressive consistency loss that penalises the KL divergence between the teacher distribution conditioned on clean context (from clean blocks) and the student distribution conditioned on noisy context (from noisy blocks), and (b) a standard autoregressive (AR) loss on the clean blocks to maintain generation quality. Both losses are computed simultaneously using the single packed sequence and the dual-purpose attention mask.

4. Iterative retraining with expanded block sizes. After training converges, the resulting Jacobi Forcing Model is used to generate new Jacobi trajectories with larger block sizes. These higher-quality trajectories (since the model now produces more correct tokens per iteration) are fed back into the same training pipeline for a second round of distillation, yielding further speedup gains.

At inference time, the trained Jacobi Forcing Model runs standard Jacobi decoding with two optional optimisations — rejection recycling (caching high-quality n-grams from rejected drafts) and multi-block decoding (maintaining multiple parallel blocks that can decode future tokens even when current blocks are unconverged) — that exploit the model's learned ability to produce long runs of correct tokens in trailing positions.

3.3 Roadmap for the Deep Dive

• Jacobi decoding mechanics and trajectories (Section 3.4.1): The fundamental mechanism that generates training data — how it works, why naive Jacobi decoding is slow, and how the trajectories encode the information Jacobi Forcing learns from.
• The progressive noise schedule (Section 3.4.2): The core insight that makes large-block parallel prediction learnable — how noise ratios are assigned, why a cyclic progressive schedule reduces the difficulty of the prediction task, and how this contrasts with CLLMs' approach.
• Sequence packing and noise-conditioned causal attention (Section 3.4.3): How multiple noisy and clean blocks are interleaved into one training sequence, how the custom attention mask enables a single forward/backward pass to compute all losses, and why conditioning on noisy context during training is essential.
• The progressive consistency loss and AR loss (Section 3.4.4): The training objective — what is being optimised, why KL divergence is the right metric, how the AR loss prevents quality degradation, and how the combined loss is computed from the packed sequence.
• Iterative progressive distillation (Section 3.4.5): Why one round of training saturates, how regenerating trajectories from the partially-trained model creates a virtuous cycle, and the specific block size configurations used.
• Inference-time optimisations (Section 3.4.6): Rejection recycling (the n-gram pool, verification, and selection mechanism) and multi-block decoding (real-active vs. pseudo-active blocks, acceptance criteria, block promotion) — how they exploit Jacobi Forcing Model's characteristic trajectory properties.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily an empirical systems paper whose core technical contribution is a training paradigm that constructs a learnable curriculum for parallel decoding by controlling the length of noisy-context dependencies during distillation, combined with inference-time techniques that leverage the resulting model's distinctive generation behaviour. Every design choice flows from a single observation: predicting a token becomes exponentially harder as the number of immediately preceding incorrect tokens grows, so the key to enabling large-block parallel decoding is to bound this span during training while still teaching the model to handle it at inference time.

---

3.4.1 Jacobi Decoding as Training Data Generator

The paper does not invent Jacobi decoding — it uses the fixed-point formulation from prior work (song2021accelerating; santilli2023accelerating) as the mechanism for generating training data and as the inference procedure for the trained model. Understanding why naive Jacobi decoding is slow is essential to understanding what Jacobi Forcing trains the model to overcome.

The fixed-point formulation. Standard AR decoding generates $n$ tokens sequentially (Equation 1). Jacobi decoding reformulates this as solving a system of $n$ nonlinear equations (Equation 2):

$f(y_i, \boldsymbol{y}_{<i}, \boldsymbol{x}) = 0, \quad \text{for } i = 1,\dots,n$

where

$f(y_i, \boldsymbol{y}_{<i}, \boldsymbol{x}) := y_i - \arg\max_y \, p_{\theta}(y \mid \boldsymbol{y}_{<i}, \boldsymbol{x})$

Here $\boldsymbol{x}$ is the prompt, $\boldsymbol{y}_{<i} = \{y_1, \dots, y_{i-1}\}$ are the previously generated tokens, and $p_{\theta}$ is the AR model's next-token distribution. The function $f$ equals zero when the token at position $i$ exactly matches what the AR model would predict given the preceding tokens — zero means "this position is consistent with AR decoding given its left context." The system of $n$ such equations encodes the constraint that every position must be consistent with the AR model's prediction given the tokens to its left.

The Jacobi iteration (Equation 3). Starting from a randomly initialised $n$-token sequence $\boldsymbol{y}^{(0)} = \{y_1^{(0)}, \dots, y_n^{(0)}\}$, each iteration $j$ updates all positions in parallel:

$\begin{aligned} y_1^{(j+1)} &= \arg\max_y \, p_{\theta}(y \mid \boldsymbol{x}) \\ y_2^{(j+1)} &= \arg\max_y \, p_{\theta}(y \mid y_1^{(j)}, \boldsymbol{x}) \\ &\vdots \\ y_n^{(j+1)} &= \arg\max_y \, p_{\theta}(y \mid \boldsymbol{y}_{<n}^{(j)}, \boldsymbol{x}) \end{aligned}$

What this computes, in operational terms. At iteration $j$, the model takes the previous iteration's tokens $\boldsymbol{y}^{(j)}$ as input, runs a single forward pass with a standard causal attention mask (so position $i$ attends to positions $<i$ from the same input $\boldsymbol{y}^{(j)}$), and produces $n$ new token predictions — one for each position, all computed simultaneously in that single forward pass. Position 1 is predicted from the prompt alone (it has no left context to update). Position 2 is predicted from the prompt plus the previous iteration's token at position 1. Position $n$ is predicted from the prompt plus the previous iteration's tokens at positions $1$ through $n-1$. The iteration halts when $\boldsymbol{y}^{(k)} = \boldsymbol{y}^{(k-1)}$ — the output has stabilised and no position changes from one step to the next. This converged state $\boldsymbol{y}^* = \boldsymbol{y}^{(k)}$ is provably identical to what greedy AR decoding would produce token-by-token.

The Jacobi trajectory. Let $\mathcal{J} = \{\boldsymbol{y}^{(0)}, \boldsymbol{y}^{(1)}, \dots, \boldsymbol{y}^{(k)}\}$ denote the full sequence of states from random initialisation to convergence. This trajectory encodes rich information: each intermediate point $\boldsymbol{y}^{(j)}$ contains some positions that already match the fixed point $\boldsymbol{y}^*$ (these will not change in future iterations — they are "converged") and some positions that do not match (these are "noisy" or "unconverged"). The key property is that tokens converge from left to right: position 1 typically converges first (it has no noisy left context), then position 2 (once position 1 is correct), and so on. This left-to-right convergence pattern mirrors the causal structure of AR generation.

Why naive Jacobi decoding is slow. The paper states that Jacobi decoding "rarely predicts more than one correct token within one fixed-point iteration" (Section 2.2). This is because the model was trained to predict tokens given correct left context (AR training). During Jacobi iteration, positions 2 through $n$ are predicted from the previous iteration's tokens at positions $<i$, which are likely wrong early in the trajectory. The model has never been trained to predict tokens given incorrect predecessors, so it performs poorly — only the first position (which has clean context from the prompt) and possibly a few early positions update toward the correct value quickly. Most positions oscillate or remain wrong until their left context stabilises through many iterations. The result: $n$ tokens are computed in parallel each iteration, but only $\sim\!1$ is actually correct and can be "accepted," so the effective speedup is minimal.

Jacobi trajectories as training data. The key insight of consistency distillation (from CLLMs) is that Jacobi trajectories provide exactly the right kind of training data to teach models to overcome this bottleneck. For each prompt in the training set, the paper runs Jacobi decoding with the current model (initially the base AR model) to collect full trajectories. For each block $B_i$ of $n$ tokens in the response, the trajectory contains multiple intermediate states $\boldsymbol{y}_{B_i}^{(j)}$ at varying noise levels (some tokens converged, some not) and the final fixed point $\boldsymbol{y}_{B_i}^*$. By training the model to map noisy intermediate states to the clean fixed point, the model learns to predict correct tokens even when conditioned on incorrect predecessors — exactly the capability that naive Jacobi decoding lacks.

Block-by-block generation for long responses. For responses longer than $n$ tokens, Jacobi decoding is applied sequentially over blocks. After the $i$-th block converges to $\boldsymbol{y}_{B_i}^*$, these tokens are committed to the KV cache, and a new block of $n$ positions is randomly initialised. The full output $\boldsymbol{l}$ is the concatenation of fixed points from consecutive blocks (Equation 4):

$\boldsymbol{l} = [\boldsymbol{y}_{B_1}^*, \dots, \boldsymbol{y}_{B_N}^*]$

where $N = \lceil L/n \rceil$ is the number of blocks. This block-by-block procedure means the training data consists of independent trajectories for each block, each with its own prompt context (the prompt plus all previously committed blocks).

---

3.4.2 The Progressive Noise Schedule

This is the paper's core mechanism for solving the scaling problem that CLLMs face. The issue, restated precisely: in Equation 6, to predict token $y_f$ at position $f$ in a block, the model conditions on $\boldsymbol{y}'_{c:f-1}$ — the noisy (unconverged) tokens from positions $c$ through $f-1$. When the block size $n$ is large and the iteration is early in the trajectory, this noisy span can be very long. Predicting correctly across a long noisy span is hard because the model has limited training signal for this exact condition.

The noise schedule as a difficulty curriculum. Instead of training on arbitrary points from Jacobi trajectories (as CLLMs do, sampling uniformly), Jacobi Forcing constructs training blocks with controlled noise ratios. The noise schedule is a sequence $\{t_1, \dots, t_N\}$ where each $t_i \in [0, 1]$ specifies the target fraction of noisy tokens in the $i$-th block of the training sequence. For each block $i$, the training procedure selects the trajectory point whose actual fraction of unconverged tokens is closest to $t_i$, and uses that point to form the noisy block $\boldsymbol{y}_{B_i}^{t_i}$.

The cyclic progressive schedule (Equation 7). The schedule is defined over a window of size $w$ blocks:

$W = \left\{0, \frac{1}{w}, \dots, \frac{w-1}{w}\right\}, \quad t_i = W[j], \quad j = i \bmod w$

What this means operationally. Within a window of $w$ consecutive blocks, the noise ratio starts at 0 (the first block is completely clean — all tokens match the fixed point), increases by $1/w$ per block, and reaches $(w-1)/w$ at the final block of the window (almost completely noisy). After $w$ blocks, the pattern repeats from 0. If the block size $n = 16$ and the window size $w = 16$, then within each window:

• Block 0 has $t = 0$: fully clean (0 noisy tokens).
• Block 1 has $t = 1/16$: approximately 1 noisy token per block of 16.
• Block 8 has $t = 8/16 = 0.5$: approximately 8 noisy tokens.
• Block 15 has $t = 15/16$: approximately 15 noisy tokens.

The cycle then restarts at $t = 0$ for block 16.

Why this form reduces the longest noisy span. Consider the worst case: a block with $t_i = 1$ (fully noisy). In CLLMs or a random schedule, any block could be fully noisy at any time, and the model might need to predict a token conditioned on an arbitrarily long preceding span of noisy tokens (up to $O(nN)$ if all blocks happen to be fully noisy). Under the progressive schedule, when a block is fully noisy (at the end of a window), the preceding blocks in the same window have lower noise ratios — they are partially clean. Specifically, the block immediately before it has $t = (w-2)/w$, which is only slightly less noisy, but blocks further back become progressively cleaner. The new block starting the next window has $t = 0$ (fully clean), providing a clean anchor point. The paper claims this reduces the longest span of consecutive noisy tokens from $O(nN)$ to $O(\lceil tn \rceil)$, where $t$ is the current block's noise ratio — the bound is now proportional to a single block's noise level rather than the entire sequence length. This is because the cyclic reset provides clean context that "resets" the accumulation of error.

Why progressive, not random or reverse. The ablation in Table 4 tests three schedules:

• Random: $t_i \sim \mathcal{U}(1, \dots, N)$, each block gets an independently sampled noise ratio. This provides no structure to the difficulty curriculum — the model sometimes sees hard blocks early, sometimes easy blocks late.
• Linear progressive: noise increases from 0 to $1 - 1/w$ within each window, as described. This creates a smooth difficulty ramp: early blocks in each window are easy (mostly clean), later blocks are harder (mostly noisy).
• Reverse progressive: noise decreases from $1 - 1/w$ to 0 within each window — the hardest blocks come first, then become easier.

The progressive schedule significantly outperforms both alternatives. The intuition: starting each window with clean blocks provides the model with easy examples to anchor its learning, and progressively increasing the noise ratio throughout the window gradually increases the difficulty of the next-token prediction task. This mirrors effective curriculum learning strategies where training examples are ordered from easy to hard.

Configuration used. The default configuration in the main experiments uses block size $n = 16$, window size $w = 16$, giving noise ratios $\{0, 1/16, 2/16, \dots, 15/16\}$ across each window of 16 blocks.

---

3.4.3 Sequence Packing and Noise-Conditioned Causal Attention

Training on progressive noise ratios requires an efficient mechanism to compute losses across all blocks simultaneously. The paper's sequence packing and attention mask design is what makes this computationally feasible — it reduces what would be $O(N)$ forward/backward passes to a single forward/backward pass.

The packed training sequence. For a training sample with $N$ blocks, the sequence packing interleaves noisy and clean blocks as follows:

$\text{packed} = [\boldsymbol{y}_{B_1}^{t_1}, \boldsymbol{y}_{B_1}^*, \boldsymbol{y}_{B_2}^{t_2}, \boldsymbol{y}_{B_2}^*, \dots, \boldsymbol{y}_{B_N}^{t_N}, \boldsymbol{y}_{B_N}^*]$

Here $\boldsymbol{y}_{B_i}^{t_i}$ is the noisy block (the trajectory point with noise ratio closest to $t_i$) and $\boldsymbol{y}_{B_i}^*$ is the clean block (the fixed point). Each block has size $n$ tokens, so the total packed sequence has $2nN$ tokens. This interleaving means that for every position in the original response, there are now two copies in the packed sequence: a noisy version (used for the consistency loss — teaching the model to predict correctly from noisy context) and a clean version (used for the AR loss — maintaining standard language modelling capability).

The dual-purpose attention mask. The critical design choice is the attention mask applied to this packed sequence. Figure 2 illustrates two options:

Clean-context conditioned mask (Figure 2a — CLLM-style). Each noisy block $\boldsymbol{y}_{B_i}^{t_i}$ attends only to preceding clean blocks $\boldsymbol{y}_{B_1}^*, \dots, \boldsymbol{y}_{B_{i-1}}^*$ and to the prompt (not shown in the figure). The noisy block does not attend to preceding noisy blocks. This means the model learns to predict correct tokens from noisy context only when the preceding blocks' actual states are clean — a much easier task, but one that does not match inference conditions (during inference, preceding blocks are noisy).

Noise-conditioned mask (Figure 2b — Jacobi Forcing). Each noisy block $\boldsymbol{y}_{B_i}^{t_i}$ attends to the actual noisy states of preceding blocks $\boldsymbol{y}_{B_1}^{t_1}, \dots, \boldsymbol{y}_{B_{i-1}}^{t_{i-1}}$. The clean block $\boldsymbol{y}_{B_i}^*$ attends only to preceding clean blocks (standard causal). This means the noisy block is conditioned on exactly what it would see during inference — noisy (unconverged) tokens from earlier blocks. The model must learn to predict correctly despite this accumulated noise, which is precisely the capability needed for fast Jacobi decoding.

Why this matters (Table 5 ablation). The paper trains Jacobi Forcing Model with two mask variants: noise-conditioned (NC) and noise-conditioned with intra-window clean context (NC-IC). The NC-IC variant provides a hybrid: within each window, noisy blocks attend to clean blocks from their own window (easier) but to noisy blocks from previous windows. Intuitively, this makes prediction easier because tokens in later blocks of a window have less noisy left context. However, Table 5 shows that the noise-conditioned mask (NC) is more effective — it produces higher speedup while maintaining generation quality. The interpretation: training under harder conditions (conditioning on actual noisy context) better prepares the model for the conditions it encounters during inference, even though the training signal is noisier. The NC-IC variant provides an easier training curriculum but creates a train-test mismatch.

Computational efficiency: O(1) passes instead of O(N). Without sequence packing, computing the consistency loss for $N$ blocks would require $N$ separate forward passes (each with a different noisy block as input) and $N$ separate backward passes. The packed sequence and custom attention mask enable all noisy blocks to be processed in a single forward pass — the mask ensures each position only attends to what it should, and the logits for all positions are produced simultaneously. Similarly, the clean blocks and their AR loss are processed in the same single forward pass (since they are interleaved in the packed sequence). The total number of forward and backward passes per training step is reduced from $O(N)$ to $O(1)$. This is not a minor optimisation — it makes training on long sequences with many blocks computationally tractable, enabling the progressive distillation over hundreds of thousands of prompts.

How the mask is implemented (block-wise sparse attention). The attention mask is constructed as a custom sparse pattern rather than a standard causal or bidirectional mask. For each query token in the packed sequence, the mask specifies which key tokens it may attend to:

• If the query belongs to a clean block $\boldsymbol{y}_{B_i}^*$, it attends to: (1) all prompt tokens, (2) all tokens in clean blocks $\boldsymbol{y}_{B_1}^*$ through $\boldsymbol{y}_{B_{i-1}}^*$, and (3) previous tokens within the same clean block $\boldsymbol{y}_{B_i}^*$ (standard causal within-block).
• If the query belongs to a noisy block $\boldsymbol{y}_{B_i}^{t_i}$, it attends to: (1) all prompt tokens, (2) all tokens in preceding noisy blocks $\boldsymbol{y}_{B_1}^{t_1}$ through $\boldsymbol{y}_{B_{i-1}}^{t_{i-1}}$, and (3) previous tokens within the same noisy block $\boldsymbol{y}_{B_i}^{t_i}$ (causal within-block).

Crucially, a noisy block does not attend to any clean blocks — it sees only the noisy versions of the preceding context. This is what enforces noise-conditioned prediction during training.

---

3.4.4 The Progressive Consistency Loss and AR Loss

The training objective has two components, combined with a tunable weight (Equation 9):

$\mathcal{L}(\theta) = \mathcal{L}_{\text{pc}} + w \, \mathcal{L}_{\text{AR}}$

where $\mathcal{L}_{\text{pc}}$ is the progressive consistency loss (teaching parallel decoding from noisy context) and $\mathcal{L}_{\text{AR}}$ is the standard autoregressive loss (maintaining language modelling quality). The weight $w$ is a hyperparameter that balances these two objectives.

The progressive consistency loss (Equation 8).

$\mathcal{L}_{\text{pc}} = \frac{1}{N} \sum_{i=1}^{N} D_{\text{KL}}\big( p_{\theta^-}(\cdot \mid \boldsymbol{x}, \boldsymbol{y}_{B_1}^*, \ldots, \boldsymbol{y}_{B_{i-1}}^*) \;\|\; p_{\theta}(\cdot \mid \boldsymbol{x}, \boldsymbol{y}_{B_1}^{t_1}, \ldots, \boldsymbol{y}_{B_{i-1}}^{t_{i-1}}) \big)$

where $\theta^- = \text{stopgrad}(\theta)$ means the teacher model's parameters are treated as constants (no gradient flows through the teacher), $D_{\text{KL}}$ denotes the Kullback-Leibler divergence aggregated across all $n$ token positions in block $i$, and $N$ is the total number of blocks.

What this computes, position by position. For each block $i$ in the packed sequence, the teacher model $p_{\theta^-}$ is given clean context — the correct fixed-point tokens from all preceding blocks $\boldsymbol{y}_{B_1}^*, \dots, \boldsymbol{y}_{B_{i-1}}^*$ plus the prompt $\boldsymbol{x}$. This teacher produces a target distribution over the $n$ tokens in the block — representing "what the model would predict if it saw the correct history." The student model $p_{\theta}$ is given the same prompt but with noisy context — the actual noisy intermediate states $\boldsymbol{y}_{B_1}^{t_1}, \dots, \boldsymbol{y}_{B_{i-1}}^{t_{i-1}}$. The student produces its own distribution over the $n$ tokens. The KL divergence measures how different these two distributions are.

Restating in operational terms. For each token position $f$ within block $i$, the teacher predicts $p_{\theta^-}(y_f \mid \text{clean left context})$ and the student predicts $p_{\theta}(y_f \mid \text{noisy left context})$. The KL divergence penalises the student when its predictions diverge from the teacher's. Minimising this loss forces the student to produce the same next-token distribution (and hence the same argmax prediction) as the teacher, even though the student sees noisy context that the teacher does not see. This teaches the student to "look through" the noise in its left context and predict the correct token anyway — exactly the capability needed to accelerate Jacobi decoding.

Why KL divergence rather than cross-entropy or MSE. The KL divergence $D_{\text{KL}}(P \;\|\; Q)$ measures how much information is lost when using the student's distribution $Q$ to approximate the teacher's distribution $P$. For discrete token prediction, this has a natural interpretation: the teacher produces a probability vector over the vocabulary (in practice, a one-hot or near-one-hot distribution from argmax plus temperature), and the student must match this entire distribution, not just the argmax. This provides richer gradient signal than cross-entropy to the ground-truth token alone (which would ignore the teacher's uncertainty) and is more appropriate than MSE (which treats token indices as Euclidean distances rather than categorical outcomes). The stop-gradient on the teacher $\theta^-$ prevents the teacher from co-adapting with the student, which would lead to representation collapse (both models drifting together without improving).

Why $1/N$ averaging across blocks. Summing the KL divergence across all $N$ blocks and dividing by $N$ computes the mean per-block loss. This treats each block equally regardless of its noise ratio $t_i$. An alternative would be to weight blocks by their noise ratio (harder blocks get more weight) or by their position (later blocks condition on more accumulated context). The uniform averaging is the simplest choice and implicitly assumes that all blocks contribute equally to the overall parallel decoding capability — the model needs to be good at predicting from both lightly noisy context (early in windows) and heavily noisy context (late in windows).

The AR loss. The standard autoregressive loss is computed on the clean blocks $\boldsymbol{y}_{B_i}^*$ using a conventional causal language modelling objective (cross-entropy between predicted next-token distribution and the ground-truth token). This term ensures that the model does not forget how to generate coherent text when conditioned on clean context — without it, the consistency loss alone could cause the model to specialise in denoising at the expense of standard generation quality. The paper notes that prior work (kou2024cllms_consistency_large_language_models) observed the same necessity, and the Jacobi Forcing model's preliminary experiments confirmed that using only the consistency objective degrades output quality.

How both losses are computed simultaneously. Because the packed sequence interleaves noisy and clean blocks, a single forward pass produces logits for every token position. The consistency loss $\mathcal{L}_{\text{pc}}$ is computed using the logits at positions corresponding to noisy blocks (comparing against the teacher's logits at the corresponding clean block positions). The AR loss $\mathcal{L}_{\text{AR}}$ is computed using the logits at positions corresponding to clean blocks (comparing against the ground-truth next tokens). Both losses are summed and backpropagated in a single backward pass.

---

3.4.5 Iterative Progressive Distillation

The paper observes that training on trajectories from the original AR model saturates: "speedup scales with training steps and saturates at large step counts, likely due to significant data distribution shifts from extensively trained models" (Section 3.1, final paragraph). The issue is that as the model improves at parallel decoding, the data it was trained on (trajectories from the original AR model) becomes stale — the original model's trajectories have low-quality intermediate states (few correct tokens, short convergent runs), limiting how much the trained model can learn about handling its own improved decoding behaviour.

Regenerating trajectories from the partially-trained model. After the first round of training (10k steps, block size 16, window size 16), the resulting Jacobi Forcing Model is used to generate new Jacobi trajectories on the training prompts. Because this model has learned some parallel decoding capability, its trajectories have different properties: more intermediate tokens are correct, convergent segments appear earlier in the trajectory, and the overall noise patterns differ from the base AR model's trajectories. Training on these new trajectories exposes the model to a data distribution that better matches its own improved behaviour.

Expanded block sizes in the second round. The second round uses "progressively larger block sizes" — specifically, block size 32 and window size 8, trained for another 10k steps. Doubling the block size (from 16 to 32) while halving the window size (from 16 to 8) changes the noise curriculum: each window now contains 8 blocks of 32 tokens each, with noise ratios $\{0, 1/8, \dots, 7/8\}$. The per-block prediction task is now harder (longer noisy contexts within each block, since $n = 32$ tokens must be predicted from noisy predecessors) but the window resets more frequently (every 8 blocks instead of 16), providing more frequent clean anchor points.

Yield: additional 20% speedup. The paper reports that this second round "yields a further 20% speedup with only minor performance degradation" (Section 3.1, final paragraph). This is significant because it demonstrates that Jacobi Forcing is not a one-shot process — it can be applied iteratively, with each round generating better training data from the improved model and enabling larger block sizes. The paper does not report whether a third round would provide further gains, but the principle of iterative self-improvement through trajectory regeneration is clearly established.

Why this is progressive distillation, not just more training. Traditional knowledge distillation trains a student to match a frozen teacher. Here, the "teacher" is the fixed point of Jacobi decoding (equivalent to greedy AR output), which is constant across rounds. But the training data (the intermediate noisy states along Jacobi trajectories) changes between rounds because the model generating those trajectories has changed. This creates a self-reinforcing cycle: the model gets better at parallel decoding → its trajectories have better-quality intermediate states → training on those trajectories provides a more informative learning signal → the model gets even better. The progressive block size increase ensures that the task difficulty scales with the model's improving capability.

---

3.4.6 Inference-Time Optimisations: Rejection Recycling and Multi-Block Decoding

These two techniques are not part of the training procedure — they are algorithmic modifications to the Jacobi decoding inference process that exploit specific characteristics of Jacobi Forcing Model's trajectories. The paper motivates them by observing qualitative differences between Jacobi Forcing Model's trajectories and those of baseline models.

Observed trajectory characteristics (Figure 4). The paper visualises a partial Jacobi trajectory and colour-codes tokens: blue for accepted (match the fixed point at their position), black for unconverged/noisy, and red for runs of three or more consecutive tokens that match the fixed point even though they are not yet "accepted" (they appear in later positions that haven't been verified as stable). Two key observations:

1. Fixed-point segments emerge within noisy context. Long runs of correct tokens (red) appear deep in the block, far from the accepted prefix, even when earlier positions are still noisy. This means the model is predicting correct future tokens despite conditioning on incorrect intermediate tokens — exactly what the noise-conditioned training taught it to do.
2. Segments progressively extend across iterations. From point 1 to point 2 in Figure 4, the number of red tokens increases, indicating that as the left context stabilises, even more trailing tokens become correct. This suggests that high-quality n-grams in the unconverged tail can be "harvested" and reused.

Rejection recycling. This technique builds an n-gram pool from discarded draft tokens to accelerate future iterations. The procedure:

1. N-gram pool construction. During Jacobi decoding, whenever a block's draft tokens fail verification (they are "rejected"), any contiguous sequences of tokens in the rejected draft — particularly those at the tail, where correct runs tend to occur in Jacobi Forcing Model — are stored in a fixed-size pool $\mathcal{N}$. Each stored n-gram records the exact token sequence.

2. Candidate generation during verification. In subsequent iterations, when verifying a new draft, the system checks the n-gram pool: if the pool contains an n-gram whose first token matches the last accepted token of the current point, it creates candidate sequences by appending the remaining tokens of that n-gram to the accepted prefix. For example, if the accepted prefix ends with token { and the pool contains an n-gram {, \n, d, e, f}, candidates like {\n, {\n de, {\n def are generated.

3. Parallel verification. These candidates are appended along the batch dimension and verified in parallel (along with the standard draft) in a single forward pass. The model's logits determine whether the candidate tokens match greedy AR predictions.

4. Selection. Among all candidates (the standard draft plus any n-gram-derived candidates), the system selects the one that yields the largest number of newly accepted tokens. This is a greedy selection: maximise tokens-per-iteration.

What this enables. In Figure 4, point 3 contains a fixed-point segment (red tokens) in its tail. Rejection recycling stores this segment in the n-gram pool when point 3's draft is partially rejected. In a later iteration, the pool provides a candidate that matches this high-quality tail, and if verified, the system can skip directly from point 3 to point 5 — bypassing the intermediate noisy iterations that would otherwise be needed to stabilise those positions. The key insight is that Jacobi Forcing Model produces reusable correct n-grams in its rejected drafts; naive Jacobi decoding discards these entirely.

Multi-block decoding (Algorithm 1). This technique maintains and refines up to $K$ blocks simultaneously to exploit the model's ability to produce correct tokens in later blocks even when earlier blocks are unconverged. The algorithm introduces two types of blocks:

• Real-active block ($RA$): The block closest to the effective KV cache boundary. Only tokens within $RA$ are actually committed to the KV cache and considered as accepted output. This block is the "frontier" of committed generation.
• Pseudo-active blocks: Up to $K-1$ additional blocks that are maintained beyond $RA$. These blocks are decoded and refined in parallel with $RA$, but their tokens are only pseudo-accepted — they condition on preceding blocks (including $RA$) but are not committed to output until they are promoted.

The multi-block decoding loop (simplified from Algorithm 1):

1. Initialisation. The real-active block $RA$ is initialised with randomly sampled draft tokens of length $n$. Its accepted tokens $a_{RA}$ start empty. All other blocks are empty, marked as pseudo-active.
2. Forward pass. The input is assembled by concatenating: $q_{RA}$ (the current draft for $RA$), then for each pseudo-active block $b$, its accepted tokens $a_b$ (which contribute to context but don't produce logits) followed by its draft $q_b$ (which produces logits for verification). A single forward pass processes this concatenated sequence.
3. Verification with rejection recycling. For each block, the model's greedy predictions $g = \arg\max \text{logits}$ are compared against the block's current draft $q_b$. The longest prefix of $q_b$ that matches $g$ is accepted (for $RA$, candidates from the n-gram pool are also considered). For $RA$, if end-of-sequence (EOS) is encountered in the accepted region, generation halts and the committed output is returned.
4. Tail update. If the draft is only partially accepted, the non-accepted tail plus new predictions from $g$ form the new draft for that block. If the draft is fully accepted, the block becomes empty (its work is done).
5. Cache management. The KV cache is trimmed to the committed length: prompt tokens plus all accepted tokens from $RA$ and any pseudo-active blocks' committed tokens.
6. Spawning. When a block $b$ reaches $|a_b| \geq s = \lceil r n \rceil$ (a threshold fraction $r$ of the block size $n$), and fewer than $K$ blocks are active, a new pseudo-active block is spawned: it copies $q_{RA}$, pads to length $n$, and is initialised as pseudo-active. This launches a new "lookahead" block that begins decoding before $RA$ has finished.
7. Promotion. When $RA$ fills its accepted tokens to $|a_{RA}| \geq n$, it is "full." The system selects a pseudo-active block $b$ with $|a_b| > 0$ (some tokens already accepted), rebuilds its draft to length $n$ (since it may have been partially consumed), verifies it by marking all accepted tokens for commitment, and promotes it to be the new $RA$. The old $RA$'s tokens are now permanently committed.

Key hyperparameters and their roles:

• $K$: number of active blocks. The paper reports that performance gains "saturate at block size $= 2$" (Section 4.3), meaning $K = 2$ — one real-active and one pseudo-active — provides most of the benefit. Additional blocks beyond two degrade quickly because later drafts are based on increasingly stale/unconverged context.
• $r$: the initialisation threshold, defined as the fraction of $RA$ completed before the next block is spawned. The paper finds $r = 0.85$ to be consistently optimal across verification sizes 2 to 8 with block size 64. This means the pseudo-active block is spawned when $RA$ is 85% complete, giving it a head start on decoding the next region while $RA$ finishes.
• Verification size: the number of candidate sequences verified in parallel (from rejection recycling). The paper uses verification size 4, chosen via grid search (Figure 8) to maximise tokens-per-second under the hardware FLOPs budget.

Why both techniques are lossless. Both rejection recycling and multi-block decoding use greedy rejection sampling for token acceptance in the real-active block (as in standard speculative decoding, leviathan2022speculative_decoding). This means tokens are accepted only if they match what the model would produce under greedy AR decoding — the generation distribution is identical to sequential greedy AR. The speedup comes purely from parallelism without any change to the output distribution.

Hardware-aware configuration. The inference configuration (block size 64, verification size 4, $K = 2$, $r = 0.85$) is chosen based on profiling (Appendix D, Figure 7) to operate near the "knee" of the GPU roofline. On B200 GPUs, decoding latency is nearly constant as parallel tokens increase up to $\sim\!256$ — processing $64 \times 4 = 256$ tokens in parallel (block size × verification size) maximises FLOPs utilisation without incurring the linear latency penalty that begins beyond this threshold. On A100 GPUs, the knee is at $\sim\!128$, so a smaller configuration would be appropriate (though the paper primarily reports B200/H200 results for the optimised variant).

4. Key Insights and Innovations

Innovation 1: The Pretrain-to-Posttrain Mismatch as a Unifying Diagnosis for Why Prior Parallel Decoding Methods Failed

The paper's most fundamental conceptual contribution is not a new method but a diagnostic framework that explains why two seemingly disparate families of prior work — AR-adapted diffusion language models (dLLMs) and consistency-distilled AR decoders (CLLMs) — both fail to scale speedup reliably with larger block sizes. The paper identifies a common root cause: pretrain-to-posttrain mismatch, which manifests in two distinct but equally destructive forms.

For AR-adapted dLLMs (SDAR, fast-dLLM), the mismatch is explicit and architectural. These methods take a model pretrained with strictly causal attention and force it to operate with block-wise bidirectional attention during post-training, while simultaneously feeding it masked tokens that never appeared during pretraining. The paper argues this creates a double distribution shift: the attention patterns the model learned over billions of pretraining tokens are disrupted, and the input data distribution shifts from natural text to randomly masked sequences. The empirical signature of this mismatch is that "SDAR suffers substantial quality drops when large block sizes (e.g., 64 or 128) are adopted" — the model cannot maintain generation quality when the block size pushes it too far from its pretrained operating regime. The conceptual insight is that causal attention is not merely a convenient inductive bias; it is baked into the model's representations so deeply that switching to bidirectional attention is destructive rather than additive.

For CLLMs, the mismatch is subtler but equally limiting. CLLMs preserve causal attention, so they avoid the architectural disruption. However, they train the model to predict tokens from arbitrary intermediate points on Jacobi trajectories — including points where the model must predict a token conditioned on a very long span of noisy (unconverged) preceding tokens. The paper's key diagnostic observation is that this creates a task difficulty mismatch: during CLLM training, the model encounters prediction tasks of wildly varying difficulty (some blocks mostly clean, some mostly noisy, with no structure to the sequence), and the hardest instances — long noisy spans — are essentially unlearnable because the model was never pretrained to handle them. The consequence is that CLLMs "face a similar limitation as AR-adapted dLLMs: as block size increases, the number of tokens correctly decoded per iteration remains essentially constant." The model learns to predict a few tokens correctly but cannot scale to the long-noise-span predictions that large-block Jacobi decoding demands.

This framing is intellectually significant because it converts a set of seemingly disconnected negative results into a coherent picture with a single explanatory principle: any post-training method for parallel decoding must respect the pretraining distribution's constraints on attention patterns and token-prediction difficulty, or it will hit a scaling wall. This is not an obvious observation — the field had largely treated "adapt AR models to parallel decoding" as an engineering problem of choosing the right training objective (NELBO vs. consistency loss), not a fundamental distribution-matching problem. The paper reframes it as the latter, which changes what solutions are worth pursuing: rather than designing better losses, design better training data distributions.

This diagnosis is supported by the paper's own negative results with dLLM baselines (Table 1, where dLLMs achieve substantially lower speedup and accuracy than Jacobi Forcing Model), and by the CLLM comparison (CLLM* with the paper's sequence packing but without progressive training shows limited speedup). It is also consistent with the prior literature the paper cites — SDAR's quality degradation at large block sizes (cheng2025sdar) and the CLLM observation that token acceptance per iteration plateaus (kou2024cllms_consistency_large_language_models) — but prior work had not unified these under a single diagnostic framework.

Innovation 2: Noise-Conditioned Causal Attention as a Third Way Between Causal and Bidirectional

The paper makes a distinctive architectural choice that challenges a tacit assumption in the parallel decoding literature: that multi-token prediction requires relaxing the causal constraint. dLLMs abandon causality entirely with bidirectional attention. CLLMs preserve it but in a limited way — during training, noisy blocks attend only to clean preceding blocks (Figure 2a), which means the model never actually learns to predict from noisy left context during training. Both approaches implicitly assume that causal attention and parallel decoding are in tension, and that the way to get parallelism is to weaken causality.

Jacobi Forcing introduces a noise-conditioned causal attention (Figure 2b) where each noisy block attends to the actual noisy states of preceding blocks, exactly as it would during inference. This is not bidirectional attention — the mask is still strictly causal (each position sees only positions to its left) — but the content being attended to is the noisy intermediate states rather than the clean fixed points. The paper's key conceptual move is to decouple the attention pattern (which remains causal) from the context quality (which becomes noisy). This means the model never experiences a distribution shift in its attention mechanism — it always processes tokens left-to-right, exactly as during pretraining — but it learns to "see through" noise in that left context.

The significance of this choice is validated by a subtle ablation (Table 5). The paper trains a variant with "noise-conditioned mask with intra-window clean context" (NC-IC), where within each training window, noisy blocks attend to clean blocks from the same window (making prediction easier) but to noisy blocks from previous windows. This hybrid mask is intuitively appealing — it provides a gentler training curriculum — but it underperforms the fully noise-conditioned mask (NC) in both speedup and accuracy. The interpretation is profound: train-test mismatch in context quality matters more than training difficulty. The NC-IC variant makes training easier but creates a gap between training conditions (partially clean context) and inference conditions (fully noisy context), and the model fails to generalise across this gap. The fully noise-conditioned mask is harder to train under but produces a model that performs exactly the task required at inference time.

This finding reframes how to think about training for parallel decoding. The conventional wisdom — make training as easy as possible, then hope the model generalises — is wrong here. Instead, training should match inference conditions as closely as possible, even if that makes training harder, because the train-test distribution shift is more damaging than the increased training difficulty. This is a general principle that potentially extends beyond Jacobi decoding to any setting where models must operate under distribution shift at test time.

The evidence for this innovation's importance is not just the ablation in Table 5 but the overall speedup results: Jacobi Forcing Model achieves 3.6–3.8× speedup on coding benchmarks (Table 1) while CLLM — which uses clean-context conditioning (Figure 2a) — achieves only ~2× speedup in prior work. The noise-conditioned mask is a significant contributor to this gap, along with the progressive noise schedule.

Innovation 3: Progressive Noise Schedule as a Curriculum That Bounds Prediction Difficulty

This is the paper's most elegant theoretical move, and it operates at the level of training data design rather than model architecture or loss function. The problem is stated clearly in Section 3.1: predicting a token $y_f$ conditioned on noisy context $\boldsymbol{y}'_{c:f-1}$ becomes exponentially harder as the noisy span grows. CLLMs fail at large block sizes because they provide no control over this span — any training block could have an arbitrarily long noisy prefix, and the model must learn to handle the worst case. The training signal for long-noise-span predictions is weak because such instances are rare in the training data (most Jacobi trajectory points have only a short noisy prefix near the beginning of convergence) and hard when they do occur.

Jacobi Forcing's solution is to impose a cyclic progressive noise schedule (Equation 7) that caps the maximum noisy span any prediction must contend with. By splitting a large block into smaller sub-blocks and controlling the noise ratio $t_i$ per sub-block with a sawtooth pattern (linearly increasing from 0 to nearly 1 within each window, then resetting), the longest span of consecutive noisy tokens that any single prediction depends on is reduced from $O(nN)$ to $O(\lceil tn \rceil)$. The key idea is that the window reset (back to $t = 0$) provides a "clean anchor" — the model never has to predict across an unbounded accumulation of noise because every window starts fresh from clean context.

This is fundamentally a curriculum learning insight applied to the structure of Jacobi trajectories. The progressive schedule does not change what the model learns to do (predict correct tokens from noisy context) but how it learns it: easy examples (low noise ratio) come first in each window, building the model's capability, and harder examples (high noise ratio) come later, once the model has clean anchors to ground its predictions. The cyclic repetition ensures the model sees this easy-to-hard progression many times across a long sequence, rather than encountering a single long span of uniformly hard predictions.

What makes this intellectually distinctive is that it identifies the structure of the noise — not just its presence — as the critical variable. Prior work treated noise in Jacobi trajectories as an undifferentiated quantity: CLLMs sample uniformly from trajectories, implicitly assuming that any noisy point is equally useful for training. Jacobi Forcing shows that which noisy points you select, and in what order, dramatically affects what the model can learn. The noise schedule is not an implementation detail; it is the core mechanism that makes large-block parallel decoding learnable.

The evidence is decisive. Table 4 compares three noise schedules (random, progressive, reverse progressive) and shows that progressive significantly outperforms both alternatives in speedup while maintaining comparable accuracy. The random schedule — which is closest to CLLM's uniform sampling — produces substantially lower speedup, confirming that uncontrolled noise exposure is the bottleneck CLLMs face. The reverse progressive schedule (hardest blocks first) performs even worse, which is consistent with the curriculum learning interpretation: starting with hard examples provides no foundation for the model to build on. The ablation directly validates the theoretical claim that noise structure, not just noise quantity, determines parallel decoding capability.

Innovation 4: Self-Reinforcing Progressive Distillation as a Scalable Training Paradigm

The paper introduces an iterative training loop that is more than just "train for more steps." After the first round of Jacobi Forcing training on trajectories from the base AR model, the paper regenerates trajectories using the partially-trained Jacobi Forcing Model itself, with larger block sizes, and trains a second round on these new trajectories. This yields an additional ~20% speedup.

The conceptual innovation here is that the training data improves as the model improves, creating a virtuous cycle. Standard distillation trains a student on a fixed dataset generated by a frozen teacher. Jacobi Forcing's iterative variant is closer to self-play in reinforcement learning: the model generates its own training data, and because it has improved at parallel decoding, that data is of higher quality (more correct intermediate tokens, longer convergent segments in trajectories) than the original base model's data. Training on higher-quality data further improves the model, which could generate even better data in a subsequent round.

This matters because it suggests that Jacobi Forcing is not a one-shot post-training technique with diminishing returns, but potentially a scalable paradigm where capability increases with each iteration as the data distribution shifts toward higher-quality trajectories. The paper explicitly notes that the first round's training "saturates at large step counts, likely due to significant data distribution shifts from extensively trained models" — the data from the original model becomes a bottleneck. The iterative regeneration breaks this bottleneck by aligning the training data distribution with the model's improving capability.

The significance of this is primarily conceptual rather than empirical (the paper reports only one iterative round, so the scalability beyond 20% gains is unproven). But it reframes the goal: rather than maximising performance from a fixed dataset (CLLMs' approach), design a training procedure where each round produces both a better model and better training data for the next round. This connects Jacobi Forcing to broader research on self-improving systems (STaR, ReST, self-play) but applies the principle to the specific problem of parallel decoding capability. The evidence from Table 1 and Table 2 shows that the iterative training (10k steps at block size 16, then 10k steps at block size 32) produces models that substantially outperform both single-round baselines and dLLMs, but the paper does not isolate the marginal contribution of the second round versus simply training for 20k steps in a single round.

Innovation 5: Inference-Time Optimisations as Exploitation of Learned Trajectory Structure

The paper's rejection recycling and multi-block decoding techniques might appear to be standard inference hacks, but they are conceptually distinctive because they exploit a specific, empirically observed property of Jacobi Forcing Model's trajectories that does not exist in baseline models. Figure 4 visualises this property: "fixed-point segments emerge within the noisy tokens of the unconverged point" and "these segments progressively extend, even under noisy context." In operational terms, Jacobi Forcing Model produces long runs of correct tokens in the tail of each block, even when the prefix of the block is still unconverged.

This property is a direct consequence of the noise-conditioned training: the model learned to predict correct tokens despite noisy left context, so when it encounters partially noisy context during inference, it can "look through" the noise and generate correct tokens at later positions. Baseline models (CLLMs, naive Jacobi decoding) do not exhibit this behaviour — their trailing tokens are typically wrong until the prefix stabilises. This means the inference optimisations are not generic acceleration techniques; they are specifically tuned to the capabilities Jacobi Forcing training produces.

Rejection recycling exploits this by harvesting the correct tail n-grams that Jacobi Forcing Model generates in rejected drafts and reusing them as candidate drafts in later iterations. This would be useless for a standard model whose rejected drafts contain no correct tokens to harvest. Multi-block decoding exploits the property by launching pseudo-active blocks that decode future regions even when the current block is unconverged — feasible because the model can generate correct tokens in later blocks despite noise in earlier ones.

The conceptual contribution is the tight coupling between training behaviour and inference algorithm design. The paper does not propose these as generic optimisations that any parallel decoder could use; it explicitly grounds them in the observed trajectory characteristics and validates that they provide outsized gains specifically for Jacobi Forcing Model. Table 3 shows that multi-block decoding with rejection recycling adds over 30 TPS compared to vanilla Jacobi decoding on B200 GPUs, and Figure 5 shows that the fast-forward token count increases with block size specifically for Jacobi Forcing Model with these optimisations — a property that would not hold for baseline models.

This coupling suggests a broader principle: inference algorithms for parallel decoders should be co-designed with the training objective, exploiting the specific capabilities the training produces rather than treating the model as a black-box parallel token generator. This contrasts with speculative decoding, where the draft model and verification algorithm are largely independent.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper evaluates on two task domains: (1) coding benchmarks — HumanEval (chen2021evaluating) and MBPP (austin2021program), trained on OpenCodeInstruct (ahmad2025opencodeinstruct); and (2) math benchmarks — GSM8K (cobbe2021gsm8k) and MATH (hendrycks2021math), trained on the math split of Openthought2 (guha2025openthoughtsdatarecipesreasoning). For OpenCodeInstruct, the paper curates question instances that come with generations passing all unit tests, yielding 450k prompts for trajectory generation and training. For Openthought2, only mathematical prompts are considered with the same training settings. All datasets are publicly available and preprocessing steps are fully documented.

• Base model(s). For coding benchmarks, the paper uses Qwen2.5-Coder-7B-Instruct (hui2024qwen2coder). For math benchmarks, it uses Qwen2.5-Math-7B-Instruct (yang2024qwen25mathtechnicalreportmathematical). Both are 7B-parameter models from the Qwen2.5 family, chosen because they represent strong contemporary open-weight models at a scale where inference latency is practically relevant. The paper does not experiment with larger or smaller model scales — all results are at 7B parameters.

• Metrics. Two primary metrics are reported:

  • Pass@1 accuracy / solve rate (%). For coding, this is strict accuracy on greedy generations for HumanEval and MBPP. For math, it is the problem solve rate (test@1) on GSM8K and MATH. The paper uses the standard evaluation protocols for each benchmark (the MATH grading function from Hendrycks et al., 2021, and the standard HumanEval/MBPP test suites).
  • Generation speedup. Measured as wall-clock speedup relative to the AR baseline on the same hardware. Two sub-metrics are reported: tokens per second (TPS) and fast-forward (FF) token count per iteration. Speedup is measured on A100-80GB, H200, and B200 GPUs, with specific hardware noted per table. The speedup ratio is computed as the TPS of the method divided by the TPS of the AR baseline under greedy decoding.

• Baselines. The paper compares against three families:

  • Diffusion language models (dLLMs): LLaDA-7B (nie2025llada), Dream-7B (ye2025dream7b), fast-dLLM (wu2025fast_dllm), fast-dLLM v2 (wu2025fastdllmv2), D2F (wang2025diffusion_forcing), SDAR (cheng2025sdar), and distilled dLLMs via dParallel (chen2025dparallel). For Dream-7B and D2F comparisons in Tables 1–2, the paper selects Dream-7B as the backbone because "it's significantly faster with similar or better performance than LLaDA-7B."
  • AR-based parallel decoders: Vanilla Jacobi decoding (santilli2023accelerating) and CLLM (kou2024cllms_consistency_large_language_models). CLLM* denotes a variant trained with the paper's new sequence packing technique but without progressive training on larger block sizes, representing an ablation of the progressive distillation contribution.
  • Speculative decoding: EAGLE-3 (li2025eagle3) and HASS (zhang2025hass), included in Appendix B (Table 6) as stronger baselines than earlier methods like Medusa.

• Generation budget / compute accounting. Speedup comparisons are measured by wall-clock tokens-per-second on matched hardware. For the main results (Tables 1–2), all methods use a block size of 128 except Jacobi Forcing Model (MR), which uses block size 64 with verification size 4. The paper explicitly notes that MR "takes extra FLOPs for multiblock decoding and parallel verification," so the block size is configured to maximise hardware utilisation under the GPU roofline (Appendix D, Figure 7): block size 64 × verification size 4 = 256 parallel tokens on H200/B200 (the "knee" of the roofline), while block size 128 on A100 (knee at ~128). This FLOPs-aware configuration is critical — the paper does not claim equal-FLOPs comparison but equal-hardware wall-clock comparison, with the explicit acknowledgment that MR consumes additional compute for its speedup.

• Cross-validation / statistical protocol. The paper does not report cross-validation for the main benchmark results. For inference configuration selection (Appendix E), a grid search is performed over block sizes and verification sizes, measuring TPS for each configuration on the target GPU. A 2D polynomial surface is fit to the discrete measurements and used as a surrogate for continuous hyperparameter selection. The paper reports that $K = 2$ (number of active blocks) and $r = 0.85$ (initialisation threshold) are fixed as "training-optimal values from a separate grid search" (Section 4.3), but details of this search and the criteria for optimality are not provided beyond the statement that these values are used.

---

Main Quantitative Results

Performance and Speedup on Coding Benchmarks (Table 1, Table 3)

The headline result appears in Table 1: on HumanEval, Jacobi Forcing Model achieves 3.6× speedup over the AR baseline (Qwen2.5-Coder-7B-Instruct) while maintaining comparable accuracy (79.9% pass@1 vs. 78.7% for AR). On MBPP, the speedup is 3.8× with accuracy 70.7% vs. 70.4% for AR. These results are on A100 GPUs with block size 128 for Jacobi Forcing Model (vanilla Jacobi decoding without MR).

The positioning against baselines in Table 1 reveals substantial gaps:

• Against dLLMs (community): Jacobi Forcing Model achieves 53.3–74× speedup compared to community dLLMs (LLaDA-7B, Dream-7B) at similar or better accuracy. LLaDA-7B achieves only 15.5 TPS on MBPP (vs. Jacobi Forcing Model's 135.6 TPS) with comparable accuracy (71.0% vs. 70.7%). Dream-7B shows 68.8% accuracy at 18.3 TPS on HumanEval — Jacobi Forcing Model is ~7.4× faster with 79.9% accuracy (+11.1 absolute points).
• Against optimised dLLMs: Fast-dLLM with bidirectional dual cache and pipelined parallel decoding achieves 43.7 TPS on HumanEval at 77.4% accuracy. Jacobi Forcing Model is roughly 2× faster (83.3 TPS on A100 at 79.9%). D2F achieves 51.4 TPS on HumanEval but with degraded accuracy (71.6% vs. Jacobi Forcing Model's 79.9%) — Jacobi Forcing Model is ~1.8× faster with higher accuracy.
• Against CLLM: The CLLM* baseline (the paper's improved CLLM variant with sequence packing but without progressive training) shows essentially no speedup over the AR baseline — 1.0× on HumanEval and MBPP, with slightly degraded accuracy (75.0% vs. 78.7% on HumanEval). This is a critical result: it establishes that the speedups come from Jacobi Forcing's progressive distillation and noise-conditioned mask, not from the sequence packing optimisation alone.
• Against speculative decoding baselines (Table 6, Appendix B): Jacobi Forcing Model (MR) achieves 116.7 TPS on HumanEval with B200, compared to EAGLE-3 at 107.3 TPS and HASS at 86.4 TPS. The accuracy comparison is complicated by the fact that EAGLE-3 and HASS are lossless with respect to their specific AR backbone checkpoints, which are not the same as Qwen2.5-Coder-7B-Instruct (the paper notes this caveat explicitly: "in principle EAGLE-3 and HASS are lossless in comparison with greedy AR checkpoints if they were trained with the Qwen2.5-7B backbone"). Jacobi Forcing Model (MR) shows 82.3% accuracy (above the AR baseline's 78.7%, likely due to the progressive distillation's AR loss preserving quality).
• Against distilled dLLM (dParallel): Jacobi Forcing Model (MR) achieves 29% higher accuracy (82.3% vs. 53.7%) and more than 80% higher TPF and TPS on HumanEval (Table 6).

With inference optimisations (MR): Adding multi-block decoding and rejection recycling to Jacobi Forcing Model further boosts speedup to nearly 4.0× on B200 GPUs (Table 3). The running window method (an optimised Jacobi decoding variant that replenishes draft tokens as they are accepted) achieves 1.6× over A100 AR baseline, while Jacobi Forcing Model (MR) reaches 3.6× — more than double the running window speedup. The paper also reports that MR adds "over 30 TPS compared to vanilla Jacobi decoding on a B200 GPU" (Appendix A).

Performance and Speedup on Math Benchmarks (Table 2)

On GSM8K and MATH using Qwen2.5-Math-7B-Instruct, Jacobi Forcing Model achieves 3.7× speedup while preserving or improving accuracy:

• GSM8K: AR baseline achieves 94.2% solve rate at 22.7 TPS. Jacobi Forcing Model achieves 95.0% (+0.8 absolute points) at 84.9 TPS (3.74× speedup). This is a notable result because the method actually improves accuracy while running nearly 4× faster.
• MATH: AR baseline achieves 77.0% solve rate at 40.5 TPS. Jacobi Forcing Model achieves 77.4% (+0.4 absolute points) at 150.7 TPS (3.72× speedup). The paper highlights this explicitly: "Jacobi Forcing Model delivers a 150.7 TPS while even slightly improving the solve rate from 77.0% to 77.4%, highlighting its ability to achieve both high efficiency and accuracy."

The paper does not report math benchmark results for dLLM baselines in the main tables (Tables 1–2 are coding-only for dLLMs). Appendix B mentions that on GSM8K, Jacobi Forcing Model "improves accuracy by 8 absolute points with about 20% higher TPF and TPS" compared to dParallel, but these numbers are not in a formal table with hardware specification.

Speedup Scaling with Hardware (Table 3, Figure 5)

Table 3 reports speedup on B200 GPUs, a more modern accelerator than A100. The key finding is that speedup scales with hardware capability — Jacobi Forcing Model achieves higher TPS on B200 relative to A100 because B200's greater FLOPs capacity allows more tokens to be processed in parallel before hitting the roofline.

• AR baseline on B200: 33.2 TPS (1.0× relative to A100 AR baseline, with the note that this is a speedup ratio over A100).
• Jacobi Forcing Model (vanilla Jacobi decoding, block size 128): 83.3 TPS (2.5× over A100 AR, 3.0× over A100 AR when accounting for the relative improvement).
• Jacobi Forcing Model (MR, block size 64, verification size 4): 116.7 TPS (3.5× over A100 AR baseline). The 30+ TPS gain from MR over vanilla is specifically from the combination of multi-block decoding and rejection recycling.

Figure 5 provides the mechanistic explanation for these speedups:

• Figure 5a shows wall-clock speedup versus block size (log-scaled) at fixed fast-forward counts per iteration, measured on H200. Up to block size 64, decoding latency shows "no penalty" — the GPU has spare FLOPs and extra tokens are nearly free. At block size 128, there is "only minor degradation" particularly at high fast-forward rates. The key insight: "up to 126 tokens can be decoded in parallel with shared KV without significant latency overhead."
• Figure 5b reports fast-forward count versus block size on HumanEval using three strategies. Multi-block decoding with rejection recycling (the top curve) shows increasing fast-forward count as block size grows, reaching 4–5 tokens per iteration at block size 256, compared to ~2 tokens for vanilla Jacobi decoding and ~1.5 for the running window method at the same block size. This directly supports the claim that MR achieves "4.5× higher token acceptance count per iteration" — the baseline (vanilla Jacobi) accepts ~1 token per iteration, while MR accepts ~4.5.

---

Ablation Studies and Robustness Checks

Training noise schedules (Table 4): The paper evaluates three schedules on a reduced-scale training configuration (Qwen2.5-Coder-7B-Instruct, 10k randomly sampled instances, block size 256, $N = 16$, $t_{\text{min}} = 0.0$, $t_{\text{max}} = 1.0$, tested on HumanEval with A100). Linear progressive achieves 74.4% accuracy with no reported speedup (the table reports accuracy only, but the paper states linear progressive "significantly outperforms the other two when the window size is 8"). For the window size 16 setting used in main experiments, the progressive schedule shows substantially better speedup than random (the ablation trains on 10k instances rather than the full 450k, so absolute speedup numbers are not comparable to Table 1). Reverse progressive is described as "significantly worse" — only one window size configuration was tested for it. The critical finding: the structure of the noise schedule matters independently of the total amount of noise, because all three schedules have the same range of noise ratios but different ordering. Progressive ordering (easy-to-hard within each window) enables the model to learn parallel decoding; random and reverse ordering do not.

Training mask types (Table 5): Comparing noise-conditioned mask (NC) with noise-conditioned mask with intra-window clean context (NC-IC), evaluated on HumanEval with A100. NC achieves higher TPS (the table shows relative speedup; exact numbers are in the paper's table) while maintaining comparable accuracy. The NC-IC variant — which provides cleaner context within each window — is intuitively easier to train under but produces worse inference performance. The paper interprets this as evidence for train-test mismatch: NC-IC conditions training predictions on partially clean context, but at inference time the context is fully noisy, so the model's learned capability does not transfer. This is an important negative result for the "make training easier" approach.

Inference FLOPs utilisation analysis (Figure 5, Appendix D Figure 7): The paper profiles generation-only latency versus total number of parallel decoded tokens across three hardware platforms. On H200 and B200, latency is "essentially flat" up to ~256 parallel tokens and grows approximately linearly beyond ~512. On A100, the knee is at ~128 tokens with steep growth by 256. This profiling directly determines the inference configuration: block size 64 with verification size 4 on H200/B200 (64 × 4 = 256, at the knee), and block size 128 on A100 (at the knee). The ablation is hardware-specific and demonstrates that the speedups are not purely algorithmic — they depend on choosing configurations that match the GPU's computational capacity.

Inference configuration search (Appendix E, Figure 8): Grid search over block sizes {8, 16, 32, 64, 128, 256} and n-gram verification sizes {1, 2, 4, 8, 12} with $K = 2$ and $r = 0.85$ fixed. The fitted TPS surface shows a clear optimum near block size 64 and verification size 4. Very small configurations (block size 8, verification 1) underutilise FLOPs; very large configurations push past the roofline and degrade wall-clock latency. The paper does not report the TPS values at each grid point, only the smoothed surface and contour map, so the exact sensitivity to these parameters cannot be assessed from the provided figures.

Effect of block size on fast-forward count (Figure 5b): Multi-block decoding with rejection recycling shows fast-forward count increasing roughly logarithmically with block size, reaching 4–5 tokens per iteration at block size 256. This is substantially higher than vanilla Jacobi decoding (~2 tokens) and the running window method (~1.5 tokens), both of which plateau at moderate block sizes. The gap between MR and baselines widens as block size increases, suggesting that MR specifically benefits from the larger block sizes that Jacobi Forcing Model's noise-conditioned training enables.

Number of active blocks ($K$): The paper reports that "performance gains from additional blocks saturate at block size = 2" (Section 4.3), meaning $K = 2$ (one real-active + one pseudo-active) provides most of the benefit. Beyond $K = 2$, "later drafts degrade quickly." The paper does not show the data supporting this claim in a table or figure — it is stated as an observation from the inference configuration search.

Initialisation threshold ($r$): The paper reports $r = 0.85$ as consistently optimal across verification sizes 2 to 8 with block size 64, determined via grid search. This means the pseudo-active block is spawned when the real-active block is 85% complete, giving it a head start on decoding the next region while the current block finishes. The paper does not provide ablation data showing sensitivity to this parameter.

Progressive distillation rounds: The paper reports that a second round of training (block size 32, window size 8, 10k steps) after the first round (block size 16, window size 16, 10k steps) "yields a further 20% speedup with only minor performance degradation" (Section 3.1). The paper does not isolate this contribution in the main results tables — the Jacobi Forcing Model results in Tables 1–2 include both rounds. There is no ablation showing single-round-only performance at the full data scale, so the marginal contribution of the second round cannot be quantified from the reported results.

Noise-conditioned mask implementation details: The paper compares two attention mask implementations (Figure 2). The ablation in Table 5 is the primary evidence that noise-conditioned masking is superior. No additional ablations on mask sparsity patterns, attention window sizes, or alternative conditioning schemes are reported.

---

Critical Assessment

Does Jacobi Forcing achieve 3.8× wall-clock speedup with minimal loss in performance?

The evidence in Tables 1–2 strongly supports this claim for the specific benchmarks and models tested. On HumanEval, Jacobi Forcing Model (vanilla Jacobi decoding, no MR) achieves 3.6× speedup at 79.9% accuracy vs. AR baseline 78.7% on A100. On MBPP, 3.8× speedup at 70.7% vs. 70.4%. On GSM8K, 3.74× at 95.0% vs. 94.2%. On MATH, 3.72× at 77.4% vs. 77.0%. In all four cases, accuracy is preserved or slightly improved, and speedup ratios are consistently in the 3.6–3.8× range.

However, several qualifications are necessary:

• Block size 128 vs. hardware-optimal configuration. The 3.6–3.8× numbers are for Jacobi Forcing Model with block size 128 on A100. Per the hardware profiling in Figure 7c, A100's "knee" is at ~128 parallel tokens — block size 128 is at the hardware limit where latency just begins to grow. On H200/B200, the knee is at ~256, so a larger block size could potentially yield higher speedup, but the paper does not report vanilla Jacobi Forcing Model (without MR) at block size 256. The ~4.0× speedup from MR on B200 is the highest reported number, but MR consumes additional FLOPs for parallel verification — this is not an apples-to-apples comparison with the AR baseline's FLOPs-per-token.

• Single model scale (7B). All experiments use 7B-parameter models. The paper does not demonstrate that Jacobi Forcing scales to larger models (13B, 70B, etc.). The speedup mechanism depends on having spare FLOPs to process multiple tokens in parallel without latency penalty — larger models have different compute-to-memory-bandwidth ratios, and the "knee" in the roofline would shift. Whether the 3.6–3.8× speedup holds at different model scales is unverified.

• The baselines are strong but not exhaustive. The dLLM comparison is thorough (LLaDA, Dream, fast-dLLM, fast-dLLM v2, D2F, SDAR, dParallel), and the CLLM comparison is informative. The speculative decoding comparison (EAGLE-3, HASS) shows Jacobi Forcing Model is competitive but not dominant — EAGLE-3 achieves 107.3 TPS vs. Jacobi Forcing Model's 116.7 TPS on B200, a relatively narrow gap. The paper does not compare against the strongest possible speculative decoding configuration (e.g., EAGLE-3 with the same Qwen2.5-Coder-7B backbone, which could be lossless), so the claim that Jacobi Forcing Model is "faster and more accurate" than speculative decoding is only partially supported.

Does Jacobi Forcing Model produce "higher quality draft n-grams from future tokens within each block"?

The visualisation in Figure 4 provides qualitative evidence for this claim: fixed-point segments (red tokens) emerge in the tail of unconverged blocks and extend across iterations. Quantitatively, Figure 5b shows that MR's fast-forward count reaches 4–5 tokens per iteration at large block sizes, compared to ~2 for vanilla Jacobi decoding. This is consistent with the claim that Jacobi Forcing Model produces more reusable correct tokens in its drafts.

However, the paper does not provide a direct quantitative metric of "draft quality" (e.g., average percentage of correct tokens in the tail of unconverged blocks, measured across the full test set). The fast-forward count in Figure 5b is a downstream metric that conflates draft quality with verification efficiency. A cleaner ablation would report the per-position correctness rate of draft tokens as a function of distance from the accepted prefix — this would directly test whether Jacobi Forcing training improves tail-token prediction. The absence of this measurement leaves some ambiguity about whether the MR speedup comes from better drafts, better verification, or both.

Do rejection recycling and multi-block decoding boost speedup to "nearly 4.0×"?

Table 3 shows Jacobi Forcing Model (MR) at 116.7 TPS on B200 for HumanEval, compared to 33.2 TPS for the AR baseline on B200 — a ratio of 3.5× relative to B200 AR, or "nearly 4.0×" as stated in the abstract when compared to A100 AR baseline (33.2 / 8.3 ≈ 4.0×, if A100 AR is ~8.3 TPS — the paper's reporting baseline cross-hardware makes exact validation difficult). The addition of MR improves TPS from 83.3 (vanilla Jacobi Forcing Model on B200) to 116.7, a ~40% relative improvement from the inference optimisations alone.

The "4.5× higher token acceptance count per iteration" claim in the abstract refers to token acceptance count, not speedup — supported by Figure 5b where MR reaches 4.5 tokens per iteration at block size 256 compared to ~1 for vanilla Jacobi decoding. The distinction between token acceptance and wall-clock speedup is important because MR's parallel verification consumes additional FLOPs. The paper is explicit about this tradeoff but the abstract's phrasing ("up to 4.5× higher token acceptance count per iteration and nearly 4.0× wall-clock speedup, effectively trading additional compute for lower inference latency") is appropriately qualified.

Genuine weaknesses and missing experiments

Missing scale dimension. The paper evaluates only 7B models on two task families (coding, math). The claims about Jacobi Forcing as a general paradigm for parallel decoding would be substantially strengthened by results at the 1B and 13B scales (or larger), and on additional task types (general text generation, instruction following, multi-turn dialogue). The hardware roofline analysis is specific to 7B model inference — whether the speedups generalise to models with different FLOPs-to-memory characteristics is unverified.

No isolation of the second distillation round's contribution. The paper reports that iterative progressive distillation adds ~20% speedup, but this is not isolated in the main results tables. The Jacobi Forcing Model results in Tables 1–2 are from the full two-round training procedure. Without a "Jacobi Forcing Model (single round)" baseline, the marginal value of iterative distillation versus simply training longer in a single round cannot be assessed. This matters for the paper's conceptual claim that self-reinforcing distillation is a scalable paradigm (Innovation 4 in Section 4).

Difficulty estimation and length generalisation. The paper does not analyse how speedup varies with generation length or problem difficulty. All reported speedups are aggregate across the full benchmark. It is plausible that Jacobi Forcing Model's parallel decoding advantage is larger for structured, repetitive outputs (code with consistent indentation patterns) and smaller for highly diverse or unpredictable text — the coding benchmarks may represent a best-case scenario. The math results suggest the effect is not code-specific, but a difficulty-stratified analysis would clarify the boundary conditions.

Training cost is not quantified. The paper does not report the FLOPs, GPU-hours, or wall-clock time required to collect Jacobi trajectories and train the model. The iterative distillation process requires two rounds of trajectory generation (running Jacobi decoding on 450k prompts to collect trajectories, then again after the first training round) and two rounds of 10k-step training. For practitioners considering adoption, the training cost matters — the speedup at inference is a win only if it amortises the training investment. The paper frames Jacobi Forcing as a post-training method for existing AR models, but the cost of this post-training relative to the cost of pretraining is not discussed.

Hardware-specific optimisation limits generality. The optimal inference configuration (block size 64, verification size 4) is tuned to B200/H200 roofline characteristics. On different hardware (A100, consumer GPUs, TPUs), the optimal configuration would differ, and the speedup ratio may be lower. The paper provides profiling data (Figure 7, Appendix D) to help practitioners adapt the configuration, but the headline speedup numbers assume access to the specific GPU models used. A practitioner with older or different hardware cannot assume the same speedup ratios.

The accuracy comparison with AR baseline is favourable but not conclusive. In all four benchmark settings, Jacobi Forcing Model slightly improves accuracy over the AR baseline (e.g., 79.9% vs. 78.7% on HumanEval, 77.4% vs. 77.0% on MATH). This is a positive result but it raises a question: is the accuracy improvement from the AR loss term in training (which provides additional fine-tuning on the training data), or from the Jacobi Forcing training itself? The CLLM* baseline, which uses the same training data and sequence packing but without progressive distillation, shows degraded accuracy (75.0% vs. 78.7% on HumanEval). This suggests that standard consistency distillation on the same data hurts accuracy, while Jacobi Forcing's progressive schedule preserves or improves it — but the paper does not train a baseline that receives equivalent total training steps without Jacobi Forcing to isolate the effect of additional fine-tuning from the effect of the specific distillation method.

Small test sets. HumanEval has 164 problems, MBPP has ~500 (the standard test split), GSM8K has 1,319, and MATH has 5,000. The coding benchmarks in particular are quite small — 164 and ~500 problems — which means small absolute differences in pass@1 could arise from a handful of problems. The paper does not report confidence intervals, statistical tests, or multiple evaluation seeds. For the coding benchmarks, a difference of 1.2 percentage points on HumanEval (79.9% vs. 78.7%) corresponds to roughly 2 problems out of 164 — well within the range of sampling variation.

No comparison with vanilla consistency distillation at equivalent training data. The CLLM* baseline is trained with the paper's sequence packing technique but without progressive training. It achieves 1.0× speedup — essentially no improvement. This demonstrates that the sequence packing alone does not produce speedup, but it does not compare against a CLLM trained with the original CLLM recipe (random sampling from trajectories, no progressive schedule, no noise-conditioned mask) on the same data — such a comparison would isolate the contribution of the progressive schedule and noise mask more cleanly than the CLLM* ablation, which conflates "no progressive schedule" with "our sequence packing."

6. Limitations and Trade-offs

Single Model Scale and Architecture — No Evidence Generalises Beyond 7B Parameters

The assumption or constraint. All experiments in the paper use exactly one parameter scale — 7B — across two variants of the Qwen2.5 family (Qwen2.5-Coder-7B-Instruct and Qwen2.5-Math-7B-Instruct). The paper does not evaluate Jacobi Forcing on models of different sizes (e.g., 1B, 13B, 70B), different architectures (e.g., non-Qwen models, mixture-of-experts), or different pretraining paradigms. The authors do not explicitly acknowledge this as a limitation in the main text.

The consequence. The speedup ratios reported (3.6–3.8×) depend on the relationship between the model's FLOPs-per-token and the GPU's available compute and memory bandwidth — the "roofline" analysis in Appendix D. A 7B model running on an H200/B200 leaves substantial headroom for parallel token processing before hitting the hardware knee (~256 tokens on B200, ~128 on A100). A larger model (e.g., 70B) has a different FLOPs-to-memory-bandwidth profile: each forward pass consumes more compute and more KV cache bandwidth, shifting the roofline knee to a lower parallel-token threshold. If the knee drops below the block size needed for effective Jacobi decoding (e.g., block size 64 becomes too large because the GPU saturates at 32 parallel tokens for a 70B model), the speedup ratio would degrade sharply, potentially to the point where Jacobi Forcing provides no benefit. Conversely, a very small model (e.g., 1B) might be so lightweight that the hardware is never saturated even at very large block sizes, but the model's capacity to learn parallel decoding under noisy context may be limited by its smaller representational capacity — a different failure mode. Neither regime is tested.

What evidence exists in the paper. None. The paper provides no scaling experiments, no FLOPs profiling for models other than 7B, and no discussion of how the speedup ratio might change with model size. The hardware profiling in Figure 7 and Appendix D is for the 7B model only. The roofline analysis is presented as a general methodology for selecting inference configurations under a FLOPs budget, but it is instantiated with 7B-specific latency measurements. A practitioner with a different model size cannot extrapolate the reported speedups.

Mitigation status. Not addressed. The paper does not claim results at other scales, so there is no misrepresentation, but the absence is consequential for practitioners considering deployment. The method's applicability to the 13B-70B range — where inference latency is most practically painful — is entirely unverified.

---

Training Cost and Trajectory Collection Overhead Are Not Quantified

The assumption or constraint. Jacobi Forcing requires a multi-stage training pipeline: (1) collecting Jacobi trajectories from the base AR model on 450k prompts, (2) training for 10k steps with block size 16, (3) regenerating trajectories from the partially-trained model with block size 32, and (4) training for another 10k steps. The paper reports the training hyperparameters (learning rate 10⁻⁶, batch size 4, max sequence length 2048, 8× A100-80GB or 8× H200 GPUs) but does not report the total GPU-hours, wall-clock training time, or FLOPs consumed. The trajectory collection step involves running full Jacobi decoding on the training set — a procedure that, by the paper's own analysis, achieves "little speedup over standard AR decoding" for the untrained base model (Section 2.2), meaning it is roughly as expensive as generating the training set via standard AR decoding, twice (once before each training round).

The consequence. Practitioners evaluating whether to adopt Jacobi Forcing cannot assess the total cost of ownership. The 3.6–3.8× inference speedup is a win only if the training cost amortises over the inference volume. For a low-volume deployment (e.g., a research prototype processing thousands of queries), the training cost may dominate any inference savings. For a high-volume production system (millions of queries), the inference speedup likely dominates, but the break-even point is impossible to calculate without training cost data. Additionally, the trajectory regeneration in step (3) creates a dependency loop — the model must be good enough at parallel decoding to generate useful trajectories, but if the first round of training is insufficient (e.g., on a different model family or task domain), the second round's trajectories may be low-quality and provide no benefit, wasting the regeneration cost.

What evidence exists in the paper. The paper states that training uses 8× A100-80GB or 8× H200 GPUs (Section 4.1) and reports 10k + 10k training steps, but provides no cost numbers. The trajectory collection cost is acknowledged indirectly — the paper notes that Jacobi decoding trajectories are collected "by running Jacobi decoding with p_θ on a set of prompts" (Section 3.1, discussion of progressive distillation) — but the computational scale of this step is not discussed. The paper frames Jacobi Forcing as a "post-training method" (Section 3.1) but does not position it relative to the cost of the original pretraining, which is the relevant baseline for assessing whether this post-training is "cheap."

Mitigation status. Not addressed. The paper does not frame training cost as a limitation or provide guidance on amortisation. A single sentence estimating GPU-hours would substantially improve deployability assessment.

---

Evaluated on Only Two Task Families, Both with Clean Correctness Signals

The assumption or constraint. All results are on coding benchmarks (HumanEval, MBPP, trained on OpenCodeInstruct) and math benchmarks (GSM8K, MATH, trained on Openthought2-math). Both task families share specific properties that may favour Jacobi Forcing: they have unambiguous ground-truth answers (unit tests for code, numeric/expression matching for math), the outputs tend to have regular structure (code blocks with indentation, mathematical derivations with consistent formatting), and the training data is curated to contain only correct solutions (OpenCodeInstruct instances with passing unit tests, Openthought2's math prompts). The paper does not evaluate on general text generation, instruction following, multi-turn dialogue, creative writing, translation, summarisation, or any task without a clean correctness oracle.

The consequence. The Jacobi Forcing training pipeline depends critically on the ability to identify correct vs. incorrect tokens in Jacobi trajectories — this is what defines "noisy" (unconverged) vs. "clean" (fixed-point) blocks during trajectory construction (Section 3.4.1, Section 3.4.2). For coding and math, correctness is binary and verifiable (does the output pass unit tests? does the answer match the ground truth?). For open-ended generation tasks, there is no equivalent signal. The Jacobi trajectory's "fixed point" is defined by greedy AR decoding convergence, but the desirability of that fixed point cannot be assessed without a task-specific verifier. If the AR model produces a coherent but suboptimal response, Jacobi Forcing would train the model to converge to that suboptimal response faster — potentially amplifying biases or errors rather than improving output quality. The paper's framing of "correct" tokens (Section 2.2, footnote 1: "By correctness, we mean alignment with the AR decoding result under a greedy sampling strategy") elides this distinction: alignment with greedy AR is not the same as task success for open-ended tasks.

Additionally, the regular structure of code and math outputs may inflate the speedup ratios. Code has highly predictable patterns (closing braces, consistent indentation, repeated variable names) that make multi-token prediction easier — the model can "guess ahead" because the structure is constrained. Less structured text (creative prose, conversational responses) may yield lower token acceptance rates per iteration, reducing the effective speedup.

What evidence exists in the paper. None directly comparing across task types. The coding and math results are consistent (3.6–3.8× speedup across all four benchmarks), which the paper presents as evidence of robustness. But the consistency could equally reflect that both task families share the properties described above — they test different content but similar structural regularity. The paper does not discuss how the training pipeline would be adapted for tasks without correctness oracles, nor does it provide qualitative examples of Jacobi Forcing Model's behaviour on unstructured text.

Mitigation status. Not addressed. The paper does not claim results beyond coding and math, but it also does not flag this as a scope limitation. The framing in Section 1 ("multi-token generation has emerged as a promising paradigm for accelerating transformer-based large model inference") implies generality that the experimental coverage does not support. A practitioner working on dialogue systems or content generation cannot assume the reported speedups apply.

---

Difficulty Estimation and Problem-Dependent Performance Variation Are Not Analysed

The assumption or constraint. The paper reports all speedup and accuracy results as aggregate numbers across entire benchmarks. It does not stratify results by problem difficulty, generation length, or any measure of instance-level complexity. The only exception is Figure 5b, which reports fast-forward count as a function of block size aggregated over HumanEval, but this is not broken down by problem characteristics.

The consequence. The aggregate speedup numbers (3.6–3.8×) obscure potentially large variance in per-instance performance. It is plausible — and consistent with the paper's own mechanism — that Jacobi Forcing provides larger speedups on "easy" instances where the model makes confident, predictable token choices, and smaller (or negative) speedups on "hard" instances where predictions are uncertain and Jacobi iterations require many steps to converge. If the speedup is concentrated on already-fast instances while slow instances remain slow, the user-facing latency distribution (e.g., p50 vs. p95 vs. p99 latency) may be much less favourable than the mean speedup suggests. This is a common pitfall in inference optimisation: mean speedup improvements that leave tail latency essentially unchanged. For interactive applications, tail latency determines user experience far more than mean latency.

Additionally, without difficulty stratification, a practitioner cannot predict whether Jacobi Forcing will help on their specific workload. If their prompts tend to be harder than the average MATH or HumanEval problem (producing longer, more complex responses with fewer predictable patterns), the speedup may be substantially lower than reported.

What evidence exists in the paper. None. The paper does not report latency distributions, percentile speedups, or difficulty-stratified results. The only speedup metric is aggregate TPS, which is a mean throughput measure. The paper does not discuss per-instance variance or worst-case behaviour.

Mitigation status. Not addressed. The paper does not acknowledge this as a missing analysis or recommend that practitioners profile their own workload-specific speedup. Given that the method's core mechanism (predicting correct tokens under noisy context) is inherently more effective when predictions are confident, the absence of difficulty analysis is a significant gap in the evaluation.

---

The Comparison with Speculative Decoding Is Incomplete and Does Not Establish Clear Dominance

The assumption or constraint. The paper positions Jacobi Forcing as distinct from and superior to speculative decoding approaches (Section 4.1, Appendix B). However, the comparison in Table 6 is limited and partially confounded. The EAGLE-3 and HASS baselines are evaluated using their publicly released checkpoints, which were not trained on the Qwen2.5-Coder-7B backbone that Jacobi Forcing Model uses. The paper acknowledges this: "in principle EAGLE-3 and HASS are lossless in comparison with greedy AR checkpoints if they were trained with the Qwen2.5-7B backbone" (Table 6 footnote). This means the accuracy numbers are not directly comparable — the speculative decoding baselines' accuracy reflects their own base model's capability, not Qwen2.5-Coder-7B's.

The consequence. The claim that Jacobi Forcing Model is faster than speculative decoding (116.7 TPS vs. 107.3 TPS for EAGLE-3 on B200) is confounded by the backbone mismatch. A properly controlled comparison would train EAGLE-3 on Qwen2.5-Coder-7B-Instruct and measure both accuracy and TPS. Given that EAGLE-3 is designed to be lossless with respect to its base model, it would likely achieve accuracy matching the AR baseline (78.7% on HumanEval) — slightly below Jacobi Forcing Model's 82.3%. The speed comparison would then be: 116.7 TPS (Jacobi Forcing Model MR, 82.3% accuracy) vs. some TPS for EAGLE-3 on the same backbone (unknown, but likely competitive given the 107.3 TPS achieved on a different backbone). Without this controlled comparison, neither dominance nor parity is established.

Furthermore, the paper does not compare against the combination of Jacobi Forcing and speculative decoding — a natural baseline where a Jacobi Forcing Model serves as the draft model for a larger target model. This combination could potentially yield speedups beyond either method alone.

What evidence exists in the paper. Table 6 provides the only head-to-head speculative decoding comparison. The paper acknowledges the backbone mismatch in a footnote but does not discuss how it affects the interpretability of the comparison. No controlled-comparison experiments are reported.

Mitigation status. Partially addressed through transparency (the footnote acknowledges the issue), but not resolved. The paper does not train EAGLE-3 or HASS on the Qwen2.5 backbone, nor does it provide EAGLE-3-style draft heads for Jacobi Forcing Model to enable an architecture-matched comparison. A practitioner choosing between Jacobi Forcing and speculative decoding cannot make an informed decision based on the reported numbers.

---

The Second Distillation Round's Contribution Is Not Isolated, and Self-Reinforcing Scalability Is Unproven

The assumption or constraint. The paper claims that iterative progressive distillation — regenerating trajectories from the partially-trained model with larger block sizes and training a second round — "yields a further 20% speedup with only minor performance degradation" (Section 3.1, final paragraph). This is presented as evidence for a self-reinforcing training paradigm where "the training data improves as the model improves" (as discussed in Section 4, Innovation 4). However, the main results in Tables 1 and 2 report Jacobi Forcing Model performance after both rounds of training (10k steps at block size 16 + 10k steps at block size 32). There is no "single round only" baseline at the full data scale.

The consequence. The 20% contribution of the second round relative to the first cannot be verified from the reported results. Several alternative explanations are consistent with the data: (1) the second round genuinely adds 20% through better-aligned trajectory data; (2) the second round's benefit comes primarily from the additional 10k training steps, not from the trajectory regeneration — training a single round for 20k steps might achieve similar speedup; (3) the second round's larger block size (32 vs. 16) is the primary driver of speedup, and a single round at block size 32 from the start might match the two-round result. Without isolating the contribution (e.g., a single-round-20k baseline, or a two-round baseline without trajectory regeneration), the claim of self-reinforcing distillation remains an interesting hypothesis rather than an empirically validated mechanism.

This matters for the paper's broader conceptual contribution. If the benefit is simply "more training steps" or "larger block size," then Jacobi Forcing is an effective but conventional post-training recipe, not a new paradigm of self-improving distillation. The paper's positioning of iterative regeneration as a core innovation (Section 1: "Jacobi Forcing repeats this distillation procedure for the trained model and involves more noisy data with a larger block size for progressive distillation") requires stronger evidence of the specific contribution of the regeneration step.

What evidence exists in the paper. The 20% claim appears in Section 3.1 (final paragraph) as a qualitative statement without a supporting table or figure. The main results tables do not disentangle the two rounds. The CLLM* baseline (trained with sequence packing but without progressive training) serves as an ablation of the progressive schedule, but it is not directly informative about the second round's contribution — it tests whether the schedule matters, not whether regeneration matters.

Mitigation status. Not addressed. The paper does not report a single-round Jacobi Forcing Model at the full data scale, nor does it discuss alternative explanations for the second round's benefit. A practitioner cannot determine whether the two-round process is necessary or whether a simpler single-round large-block training recipe would suffice.

7. Implications and Future Directions

How This Work Changes the Landscape

Jacobi Forcing reframes the parallel decoding problem from an architectural question — "should we use causal or bidirectional attention for multi-token generation?" — into a distribution-matching problem: can we construct training data that teaches a model to predict correctly under noisy context without ever changing the attention patterns it was pretrained with? This is a genuine conceptual reframing, not merely a new method in an existing category. The paper's diagnostic of pretrain-to-posttrain mismatch as the shared failure mode of both dLLM adaptation (architectural disruption) and CLLM-style consistency distillation (uncontrolled task difficulty) provides a unifying lens through which prior negative results become coherent. The field previously saw SDAR's quality collapse at large block sizes and CLLM's plateaued token acceptance as separate empirical disappointments; Jacobi Forcing shows they are manifestations of the same underlying principle — any post-training for parallel decoding must respect both the attention structure and the token-prediction difficulty distribution of the pretraining regime, or it will hit a scaling wall.

The practical consequence of this reframing is that it redirects research effort away from designing novel attention masks or loss functions and toward designing better training data curricula. The paper demonstrates that a simple KL-divergence loss combined with a carefully constructed noise schedule outperforms architecturally sophisticated alternatives (bidirectional attention, NELBO objectives, dual KV caches). This is a methodological shift: the primary lever for parallel decoding capability is not the training objective but the structure of the training data. For practitioners, this means that improving Jacobi Forcing likely involves better noise schedules, more principled trajectory sampling, or hardware-aware block partitioning — not architectural modifications.

The paper also resolves a specific tension in the speculative decoding literature. Jacobi decoding was known to be theoretically capable of parallel token generation (song2021accelerating; santilli2023accelerating) but practically useless because standard AR models accept only ~1 token per iteration. CLLMs improved this to ~2× speedup by training models to accelerate convergence, but could not push further. The implicit message from prior work was that Jacobi-based approaches are inherently capped and that auxiliary draft models (speculative decoding) or full architectural conversion (dLLMs) are necessary for substantial gains. Jacobi Forcing demonstrates this is false — a properly trained AR model using Jacobi decoding can achieve speedups competitive with or exceeding state-of-the-art speculative decoding, without any auxiliary draft model or architectural modification. This reopens Jacobi decoding as a viable parallel decoding paradigm, particularly attractive because it requires no additional parameters at inference time.

At a broader level, the paper establishes that the ability to predict correct future tokens from noisy left context is a learnable skill that can be systematically improved through iterative self-distillation. This finding connects Jacobi Forcing to the broader literature on self-improving LLMs (STaR, ReST, self-play), but applies the principle to a specific capability — parallel token prediction — rather than general task performance. The iterative distillation loop (train → generate better trajectories → train on better trajectories) suggests a template that could be applied to other capabilities where the model's own outputs can generate progressively better training data.

However, the scope of the demonstrated shift is limited by the paper's experimental coverage. The results are at a single model scale (7B), on two task families with clean correctness signals and regular output structure. The conceptual claims about distribution matching and noise-conditioned training are well-supported by the ablations (Tables 4–5), but whether they generalise to larger models, unstructured text generation, or tasks without correctness oracles remains unverified. The reframing is therefore credible and well-motivated within its demonstrated scope but not yet established as a universal principle. The most important open question is whether the pretrain-to-posttrain mismatch diagnosis applies equally to larger models or whether larger models, with their greater capacity, can tolerate more distribution shift and benefit from simpler training recipes.

Follow-Up Research This Work Enables

Scaling Jacobi Forcing to 13B–70B models and measuring how the speedup ratio degrades (or improves). The paper's results are exclusively at 7B parameters. A direct replication at 13B, 34B, and 70B scales — using the same Qwen2.5 model family to control for architecture — would answer the most pressing open question: does Jacobi Forcing's speedup scale with model size, stay constant, or degrade? The mechanism depends on hardware rooflines (Appendix D, Figure 7), which shift as model FLOPs-per-token increases. A strong follow-up would profile the GPU roofline at each model scale (as the paper does for 7B in Figure 7), measure the maximum block size that fits under the knee, train Jacobi Forcing Models at each scale following the same two-round recipe, and report speedup vs. AR baseline on HumanEval and GSM8K. The key measurement is whether the speedup ratio holds, degrades gradually (e.g., 3.8× at 7B → 2.5× at 70B), or collapses at some threshold. If degradation is gradual, Jacobi Forcing remains practically useful across scales; if it collapses sharply, the method is specific to small models where spare FLOPs are abundant. A negative result — speedup dropping below 2× at 34B — would be equally informative, bounding the method's practical applicability.

Extending Jacobi Forcing to general text generation tasks without correctness oracles. The current training pipeline relies on identifying "correct" vs. "noisy" tokens via convergence to greedy AR output on tasks with verifiable ground truth (passing unit tests, matching answer keys). For open-ended generation — dialogue, summarisation, creative writing, instruction following — there is no equivalent correctness signal, and the "fixed point" of Jacobi decoding may be a coherent but suboptimal response. A concrete extension would adapt the trajectory collection step: instead of defining the clean block $\boldsymbol{y}_{B_i}^*$ as the greedy AR output (which could be any coherent completion), use a reward model or LLM-as-judge to score multiple AR-generated completions for each prompt, select the highest-scoring one as the "target" fixed point, and collect Jacobi trajectories relative to that target. The Jacobi Forcing training objective would then teach the model to converge to high-quality outputs, not just any AR output. The experiment would: (1) train Jacobi Forcing Models on a general instruction-following dataset (e.g., Alpaca, OpenAssistant, or a curated subset of Chatbot Arena conversations) using reward-model-selected targets, (2) evaluate on MT-Bench or AlpacaEval for quality, and (3) report speedup relative to AR decoding. A negative result — Jacobi Forcing degrades output quality on open-ended tasks even with reward-model targets — would reveal that the method's success depends on the structural regularity of code/math, not just the training paradigm.

Hardware-aware adaptive block sizing that dynamically adjusts the block size per generation step based on real-time GPU utilisation. The paper's inference configuration (block size 64, verification size 4 on B200) is statically tuned to the roofline knee. In practice, GPU utilisation varies with prompt length, batch size, and generation length. A follow-up could implement an adaptive controller that monitors GPU utilisation (e.g., via CUDA profiling counters) during generation and increases block size when headroom is detected (early in generation, short prompts) and decreases it when utilisation saturates. The experiment would compare adaptive block sizing against the static configuration on HumanEval and report both mean speedup and tail latency (p95, p99) — the hypothesis being that adaptive sizing improves tail latency by avoiding the latency cliff when the static block size overshoots the roofline on long prompts. A negative result (adaptive sizing provides no benefit over a well-tuned static configuration) would simplify practical deployment by confirming that one configuration suffices.

Isolating the contribution of iterative distillation vs. extended single-round training. The paper reports that a second distillation round with regenerated trajectories adds ~20% speedup (Section 3.1), but this is confounded with additional training steps (10k → 20k) and larger block size (16 → 32) in the second round. A clean ablation would train three variants at the full 450k-prompt data scale: (A) single round, block size 16, 20k steps (extended training, no regeneration, same block size); (B) single round, block size 32, 20k steps (larger block size, no regeneration); (C) two rounds as in the paper. Comparing A vs. the paper's two-round result isolates the contribution of regeneration + larger block size; comparing B vs. C isolates the regeneration step specifically (both use block size 32 in the final stage). The experiment would clarify whether the self-reinforcing distillation claim (Innovation 4) is empirically supported or whether the same speedup could be achieved by a simpler recipe of training longer with larger block sizes on the original trajectories. A negative result — B matching C — would substantially reduce the novelty of the iterative distillation component.

Combining Jacobi Forcing with speculative decoding for compound speedup. Jacobi Forcing Model produces high-quality draft tokens (Figure 4, Figure 5b) but still requires per-token verification. A natural extension is to use a Jacobi Forcing Model as the draft model in a speculative decoding pipeline with a larger target model. The experiment: (1) train a Jacobi Forcing Model at 7B; (2) pair it with a larger target model (e.g., Qwen2.5-Coder-32B or 72B) using standard speculative decoding (the Jacobi Forcing Model proposes $k$ tokens per iteration, the target model verifies in one forward pass); (3) measure speedup and accuracy relative to AR decoding with the target model alone, and relative to using a standard (non-Jacobi-Forcing) draft model of the same size. The hypothesis is that Jacobi Forcing's drafts are higher quality than standard draft models (more correct tokens per proposal, higher acceptance rates), leading to better compound speedup than either method alone. A negative result — the Jacobi Forcing draft model's speed advantage is absorbed by the target model's verification overhead — would suggest that Jacobi Forcing and speculative decoding address different bottlenecks.

Difficulty-stratified and length-stratified analysis of token acceptance rates. The paper reports only aggregate fast-forward counts. A detailed analysis would stratify the HumanEval problems by (a) solution length (short < 100 tokens, medium 100–500, long > 500) and (b) problem difficulty (using the standard HumanEval pass@1 of the base AR model as a difficulty proxy), and report fast-forward count per iteration and per-problem speedup within each stratum. The specific question: does Jacobi Forcing's speedup concentrate on easy/short problems while providing minimal benefit on hard/long ones? If so, the effective speedup for the hardest quartile of problems — precisely those where latency matters most — may be substantially lower than 3.8×. This analysis would directly inform practitioners about expected speedup on their specific workload distribution. A strong negative result (speedup < 1.5× on the hardest quartile) would bound the method's practical value for latency-sensitive applications where tail latency dominates user experience.

Practical Applications and Downstream Use Cases

Low-latency code completion in IDEs and coding assistants. The paper reports 3.6× speedup on HumanEval and 3.8× on MBPP at 7B parameters (Table 1), with accuracy preserved or slightly improved (79.9% vs. 78.7% on HumanEval). For a coding assistant like GitHub Copilot or Cursor, where latency directly determines user experience — developers expect completions within a few hundred milliseconds — reducing generation time by nearly 4× means the difference between a completion appearing instantly and a perceptible delay that breaks flow. A 7B model running Jacobi Forcing on an A100 or H200 could serve multiple concurrent users with sub-200ms latency for typical code completions (tens to low hundreds of tokens), where an AR baseline might exceed 500ms. The accuracy preservation is critical: speedup is worthless if the completions are less useful. The method requires no architectural changes to the deployed model, so integration into existing serving stacks (vLLM, TGI, TensorRT-LLM) is straightforward — it only requires implementing the Jacobi decoding inference loop with the custom attention mask for the draft+verify step.

Batch inference for math reasoning or code evaluation at scale. For organisations running large-scale batch evaluation — e.g., grading student code submissions, generating training data for math reasoning models, or evaluating model outputs against test suites — throughput (tokens per second per GPU) determines cost. The paper reports 150.7 TPS on MATH for the 7B model vs. 40.5 TPS for AR (Table 2), a 3.7× improvement. In a batch inference pipeline processing millions of problems, this directly translates to 3.7× lower GPU-hours and cost, or the ability to process 3.7× more problems in the same time window with the same hardware. The method is particularly attractive here because batch inference is typically latency-tolerant (individual requests don't need real-time response) and the regular structure of code/math outputs favours high token acceptance rates. The iterative distillation cost can be amortised over the inference volume — for pipelines processing billions of tokens, the training cost (~20k steps on 8 GPUs) is negligible relative to the inference savings.

On-device or edge deployment where larger models are infeasible. The paper demonstrates that a 7B Jacobi Forcing Model can achieve speedups competitive with speculative decoding baselines (Table 6: 116.7 TPS vs. 107.3 TPS for EAGLE-3 on B200) without requiring an auxiliary draft model or additional prediction heads. For edge deployment scenarios — e.g., a code completion model running locally on a developer's laptop with a consumer GPU — memory is the binding constraint. Jacobi Forcing provides speculative-decoding-level throughput using only the base model's weights, saving the memory that a separate draft model or Medusa-style heads would consume. The 7B model on a consumer GPU (e.g., RTX 4090, ~24 GB VRAM) can run comfortably with Jacobi decoding; adding an EAGLE-3-style draft head structure or a separate draft model would push memory limits. The paper's hardware profiling (Appendix D, Figure 7) provides the methodology for tuning block size to any GPU's roofline, making it deployable on consumer hardware where the knee will be at a lower parallel-token count.

When to Prefer This Method

The paper positions Jacobi Forcing explicitly against three alternatives — dLLM adaptation, CLLM-style consistency distillation, and speculative decoding — and the experimental results in Tables 1–2 and Table 6 provide clear decision boundaries. The tradeoffs are grounded in specific numbers from the paper rather than general principles.

Prefer Jacobi Forcing over dLLM adaptation (SDAR, fast-dLLM, D2F) when: (a) you are starting from a strong pretrained AR model that you want to preserve the generation quality of, and (b) you have the budget for custom trajectory collection and distillation training rather than using off-the-shelf diffusion training recipes. The paper shows dLLMs achieve 15–51 TPS on coding benchmarks vs. Jacobi Forcing Model's 84–136 TPS (Table 1), with Jacobi Forcing also delivering higher accuracy. The gap is large enough (2–7×) that Jacobi Forcing dominates on both speed and quality. The cost of this advantage is the training pipeline complexity: trajectory collection + two-round distillation vs. dLLMs' standard masked-token training. If you lack the infrastructure for custom Jacobi trajectory generation on your training data, dLLM adaptation may be simpler to implement despite lower performance.

Prefer Jacobi Forcing over CLLM-style consistency distillation when: you need speedups beyond ~2× or block sizes beyond small values. The paper's CLLM* baseline (Table 1) achieves essentially no speedup (1.0×) with the paper's improved sequence packing, and prior CLLM work achieved at most ~2×. Jacobi Forcing's progressive noise schedule directly addresses the long-noisy-context bottleneck that caps CLLMs — if you attempted to push CLLMs to block size 128 or 256, accuracy would degrade sharply (as the paper argues SDAR does under bidirectional attention). If your application can tolerate 2× speedup and you want the simplest possible training recipe, CLLMs may be sufficient, but the paper demonstrates that CLLM-style uniform trajectory sampling does not scale.

Prefer Jacobi Forcing over speculative decoding (EAGLE-3, HASS, Medusa) when: (a) you cannot afford additional parameters at inference time (memory-constrained deployment, edge devices), or (b) you want the base model itself to serve as the parallel decoder without auxiliary draft heads or draft models. The paper shows Jacobi Forcing Model MR achieves 116.7 TPS vs. EAGLE-3's 107.3 TPS on B200 (Table 6), with competitive or better accuracy. The advantage stems from requiring no additional weights — Jacobi Forcing Model is a single model; EAGLE-3 requires a separate draft head structure trained on top of the base model. However, the speculative decoding comparison in the paper is not fully controlled (different backbones for EAGLE-3/HASS vs. Jacobi Forcing), so the TPS advantage should be treated as indicative rather than definitive. If you already have a strong draft model or draft heads deployed and they achieve acceptable speedup, switching to Jacobi Forcing may not justify the retraining cost.

Prefer Jacobi Forcing with MR (multi-block + rejection recycling) when: (a) you are deploying on hardware with abundant FLOPs relative to the model size (H200, B200), and (b) you are willing to trade additional FLOPs-per-token for lower wall-clock latency. The paper explicitly frames MR as "effectively trading additional compute for lower inference latency" (abstract). On B200, MR adds ~30 TPS over vanilla Jacobi decoding (Appendix A), reaching nearly 4.0× speedup. On A100, the hardware knee is at ~128 tokens (Figure 7c), so MR's parallel verification may saturate the GPU and provide less benefit — the paper uses block size 128 without MR on A100, suggesting that MR's extra FLOPs are not cost-effective on that hardware tier.