TiDAR: Think in Diffusion, Talk in Autoregression

ArXiv: 2511.08923

🎯 Pitch

TiDAR is the first architecture to match autoregressive quality while delivering up to 5.91× higher tokens per second—not by choosing between diffusion and autoregression, but by fusing them into a single forward pass that drafts multiple tokens in parallel and verifies them sequentially using the same model weights.

---

1. Executive Summary

This paper introduces TiDAR, a sequence-level hybrid architecture that combines diffusion and autoregressive generation within a single model forward pass, drafting multiple tokens in parallel via diffusion ("thinking") and sampling them autoregressively ("talking") using structured attention masks and rejection sampling. Evaluated on coding, math, and reasoning benchmarks using models initialized from Qwen2.5 and Qwen3 at 1.5B and 8B scales, TiDAR is compared against AR baselines, Block Diffusion, Dream, Llada, and speculative decoding (EAGLE-3). TiDAR achieves lossless quality relative to its AR counterpart at 1.5B while delivering 4.71× relative throughput speedup in tokens per second, and at 8B delivers 5.91× throughput speedup with minimal quality loss—the first diffusion-based architecture to close the quality gap with AR models while substantially surpassing speculative decoding throughput. The paper establishes that parallel diffusion drafting combined with autoregressive rejection sampling can substitute for model scale at inference time, but only when the drafting and verification share the same model weights within a single forward pass, exploiting the "free token slots" available in the memory-bound regime on modern GPUs.

2. Context and Motivation

The Core Problem: The Memory-Bound Bottleneck in Autoregressive Decoding

The fundamental tension this paper addresses is architectural: autoregressive (AR) language models generate high-quality text but are fundamentally inefficient at inference time, while diffusion language models (dLMs) can generate tokens in parallel but suffer from degraded output quality. This is not a surface-level implementation issue — it reflects a deep hardware-algorithm mismatch that has shaped the LLM landscape.

During autoregressive decoding, the model generates one token per forward pass. At small batch sizes (which dominate interactive and latency-sensitive applications), this process is memory-bound rather than compute-bound: the GPU's floating-point units sit idle while memory bandwidth is saturated loading model weights and the key-value cache for each token [5, 6, 10, 11]. As the paper demonstrates in Figure 1 using Qwen3-32B on an NVIDIA H100, the latency of a forward pass remains nearly constant as you add more token slots — up to a certain threshold — because the dominant cost is weight loading, not computation on the input tokens. The paper calls these additional token positions that incur negligible extra latency "free token slots" and "cheap token slots," depending on how far they extend into the compute-bound regime.

This observation creates a clear imperative: if you can generate multiple tokens in a single forward pass without substantially increasing latency, you can improve throughput proportionally. The paper formalizes this intuition in the introduction:

"If for a given $k$ such that both computations are still memory-bound, the forward time of $F([x_{<t-1}; x_t])$ and $F([x_{<t}; m_{t+1}, \ldots, m_{t+k+1}])$ should be similar. We refer to the extra token slots $(m_{t+2}, \ldots, m_{t+k+1})$ as free token slots because carrying them through a single forward incurs minimal to no latency increase as validated by a real-world profiling in Figure 1."

The promise is clear: exploit these free token slots to do useful work — draft multiple candidate tokens — and you get speedup essentially for free. The challenge is that drafting useful tokens requires making predictions about future positions without access to the ground-truth context at those positions, which is precisely where the tension with output quality arises.

Why This Matters: The Throughput-Quality Frontier

The practical implications are substantial. Current LLM serving infrastructure operates in a regime where:

• Interactive applications (chatbots, code assistants, real-time translation) demand low latency, forcing small batch sizes where the memory-bound bottleneck is most severe.
• Batch inference pipelines (evaluating benchmarks, generating training data) can use larger batch sizes to improve compute utilization, but the per-query latency of AR decoding still dominates total cost.
• The gap between GPU compute capacity and memory bandwidth has been widening with each hardware generation, meaning the memory-bound problem is becoming more acute over time, not less.

A method that increases tokens-per-second by 4-6× without sacrificing quality would directly translate to lower serving costs, reduced latency, or both — precisely the kind of improvement that changes deployment economics. Moreover, as the paper notes in Section 1, there is a broader motivation connected to the scaling of AI systems: "how to make the most of compute resources during both training and testing times becomes increasingly important." This positions the work within the larger conversation about compute efficiency that includes pretraining scaling laws (Hoffmann et al., 2022), model distillation, quantization, and speculative decoding.

The Theoretical Basis for the Quality-Parallelism Tradeoff

Before examining prior approaches, it is essential to understand why parallel token generation tends to degrade quality. The paper provides a clean formalization in Section 1 that is worth walking through carefully, as it underpins the entire architectural motivation.

Autoregressive models sample from a chain-factorized joint distribution:

$p_{\text{AR}}(\cdot; \theta) = \prod_i p^i_\theta(x_i \mid x_{<i}; \theta)$

Each token $x_i$ is conditioned on all previously generated tokens $x_{<i}$. This aligns naturally with the sequential structure of language — later words depend on earlier ones in complex ways — and the factorization is exact (no independence assumptions are made).

Diffusion models sample from a different distribution:

$p_{\text{Diff}}(\cdot; \theta) = \mathbb{E}_{\tilde{x} \sim q(\cdot \mid x)} \prod_i p^i_\theta(x_i \mid \tilde{x})$

Here, $\tilde{x}$ is a corrupted version of the full sequence $x$, drawn from a noising distribution $q(\cdot \mid x)$. Each token $x_i$ is predicted from this corrupted context. When decoding one token per denoising step, the decoded token becomes part of the condition for the next step (the model conditions on a progressively less-corrupted version of the sequence), which preserves quality. But when decoding $K$ tokens in a single step, the model computes:

$p_{\text{Diff\_Independent\_K}}(\cdot; \theta) = \mathbb{E}_{\substack{\tilde{x} \sim q(\cdot \mid x) \\ \text{s.t. } \tilde{x}_{i \in K} = [m]}} \prod_{i \in K} p^i_\theta(x_i \mid \tilde{x})$

Because all $K$ mask positions are predicted from the same corrupted context $\tilde{x}$, the predictions are conditionally independent — the model cannot account for dependencies among the $K$ tokens being generated simultaneously. As the paper states:

"The introduced token independence assumption will degrade the quality despite providing more parallelism during sampling."

This is not a minor approximation — it fundamentally violates the structure of language, where adjacent words exhibit strong mutual information. The degradation is empirically substantial: as cited from APD [17], Dream-7B's accuracy on GSM8K drops by 10% when increasing from 1 to 2 tokens decoded per step. This is the central tradeoff that all prior diffusion language models grapple with: parallelism comes at the cost of ignoring inter-token dependencies, and the more tokens you generate in parallel, the worse the quality.

The key insight from this formalization is that the diffusion sampling procedure would fall back to AR if you restricted the decoding order to strict left-to-right, decoded one token per step, and changed the masking strategy to uniform suffix masking during training. This observation — that the two paradigms exist on a continuum — is what makes a hybrid architecture conceptually viable.

Prior Approaches and Their Limitations

The paper situates itself relative to three families of prior work, each of which attempts to address the throughput-quality tension but falls short in specific ways. I will walk through each, explaining the mechanism and where it breaks down.

Pure Diffusion Language Models (Dream, Llada, MDLM)

Diffusion language models [21, 22, 8, 9, 12, 7] apply the denoising diffusion paradigm — originally developed for continuous domains like images — to discrete text tokens. The core idea is to train a model to recover clean tokens from corrupted (masked) sequences, and at inference time, iteratively denoise a fully masked sequence into a coherent output. The promise is that because multiple tokens can be predicted simultaneously in each denoising step, generation can be substantially faster than AR decoding.

Where they fall short: The paper identifies two specific failure modes:

1. The parallelizability-quality contradiction: As discussed above, the best generation quality is achieved when decoding one token at a time — which eliminates the speed advantage. As the paper reports, "the best generation quality is often achieved when decoding strictly one token per denoising step." This is confirmed in Table 2, where diffusion models like Dream-7B and LLaDA-8B are evaluated decoding one token per forward pass (for best quality), and even then, Dream-7B achieves 58.74% average score versus Qwen3-8B's 68.09% — a substantial gap (see Table 2).

2. Lack of exact KV cache support: Because diffusion models use bidirectional attention over the entire sequence (or over blocks), they cannot trivially cache and reuse key-value pairs across generation steps in the way AR models can. The paper notes that Fast-dLLM [13, 24] proposes block-parallel decoding with prefix caching, and d-KV cache [27] selectively caches tokens in a delayed fashion, but both introduce complexity and quality tradeoffs. The absence of exact KV caching means that even if the model can decode multiple tokens per step, a significant fraction of the FLOPs are wasted recomputing representations for tokens that have already been processed.

The paper's assessment is blunt: "current open source SOTA diffusion LLMs, such as Dream [9] and Llada [8], have not yet matched the combined speed–quality profile of strong AR LLMs." This is not a criticism of those models' technical contributions — they advanced the state of the art — but rather a recognition that the pure diffusion paradigm faces a fundamental ceiling imposed by the independence assumption.

Speculative Decoding and Multi-Token Prediction

Speculative decoding [11, 16] accelerates AR generation by using a faster (typically smaller) draft model to propose multiple candidate tokens, which are then verified in parallel by the larger target model using a modified rejection sampling procedure that guarantees the output distribution matches the target model exactly. If the draft model's proposals are accepted at a high rate, the overall throughput increases because the expensive target model is called less frequently (it verifies $K$ tokens at once rather than generating one at a time).

The paper's comparison of speculative methods is organized in Table 1, which I will unpack because it captures the key architectural differences:

Classic speculative decoding [16]: Uses a separate, smaller draft model. Drafting capacity is low (the small model cannot match the target model's predictions), drafting is sequential (the draft model itself generates tokens one at a time), and drafting and verification are sequential (first draft, then verify). These three limitations constrain the maximum speedup: low drafting capacity means low acceptance rates, and sequential drafting adds latency overhead.

EAGLE series [29, 30, 18] and DeepSeek-V3 MTP [19]: Address the capacity limitation by making the draft model share the base model's hidden states — EAGLE adds autoregressive layers on top of the base model's embeddings, and DeepSeek-V3 uses sequential multi-token prediction modules. This increases drafting capacity (the draft model sees rich intermediate representations), but drafting is still sequential (the added layers are autoregressive) and drafting is sequential to verification (you must complete the base model's forward pass before drafting begins). As the paper notes:

"the drafting process does not fully take advantage of the base model, and the maximal speedup is hindered by the lower drafter capacity. In addition, EAGLE and DeepSeek-V3's MTP modules are still autoregressive and sequential to the base verification. These two factors show that they cannot effectively increase the compute density and release the full power of parallel generation."

Apple MTP [20]: Shares weights with the base model and allows parallel decoding of multiple future tokens, but the paper indicates it still has mid-level drafting capacity relative to what TiDAR achieves.

The critical insight from the paper's analysis is that none of these speculative methods achieve all three desirable properties simultaneously: (1) the drafter has the full capacity of the base model, (2) drafting is fully parallel (not autoregressive), and (3) drafting and verification happen in a single forward pass. TiDAR is designed to achieve all three, and Table 1 marks this explicitly with checkmarks in all rows for TiDAR.

Block Diffusion (Semi-Autoregressive Models)

Block Diffusion [12] is the closest architectural predecessor to TiDAR. It interpolates between AR and diffusion by partitioning the sequence into blocks: the probability of each block is conditioned autoregressively on previous blocks, and within each block, tokens are generated via discrete diffusion (bidirectional attention). This enables KV caching across blocks (since block boundaries are causal) and allows parallel generation within each block.

Where Block Diffusion falls short: Despite supporting exact KV caching — a significant advance over pure diffusion — Block Diffusion inherits the same quality-parallelism tradeoff within each block. Generating multiple tokens per block in parallel introduces the independence assumption discussed above, and the paper's own Block Diffusion baseline, trained under the same recipe starting from Qwen2.5-1.5B, achieves only 38.41% average accuracy compared to TiDAR's 44.03% (Table 2, 1.5B scale) and 60.27% versus TiDAR's 65.31% at the 8B scale. The Pareto frontier analysis in Figure 5 shows that Block Diffusion requires trading off tokens-per-forward-pass (T/NFE) against quality in a way that TiDAR does not — Block Diffusion with higher throughput (by accepting lower-confidence predictions) sees sharp quality drops, while TiDAR maintains quality at higher T/NFE.

Moreover, the paper identifies a subtle training limitation of Block Diffusion:

"Note that Block Diffusion cannot compute [next token prediction] loss on the prefix because of the label leakage issue of intra-block bidirectional attention."

Because Block Diffusion uses bidirectional attention within blocks, a token in the middle of a block can attend to tokens that follow it in the sequence, creating a label leakage problem if you try to compute standard next-token prediction loss. This means Block Diffusion cannot leverage the dense NTP signal during training — a limitation TiDAR explicitly addresses by making the prefix fully causal (see Section 3.1).

Conflicting Evidence in the Literature

The paper inherits a research landscape where the evidence about diffusion language models is mixed. On one hand, discrete diffusion for text has achieved impressive results: MDLM [32] demonstrated that masked diffusion could compete with AR models on perplexity, and Llada [8] and Dream [9] scaled diffusion to 8B and 7B parameters respectively with competitive performance on many benchmarks. On the other hand, the practical throughput advantages have been disappointing: to get best quality, these models must decode one token per step, eliminating the speed advantage, and attempts to decode multiple tokens per step cause substantial quality degradation (the 10% GSM8K drop cited from APD [17]).

Similarly, in the speculative decoding literature, methods like EAGLE-3 achieve impressive speedups (2-3× throughput improvement is typical), but they are fundamentally bounded by the sequential nature of the drafting process and the fact that the drafter, even when sharing hidden states, has lower capacity than the full base model. The paper's Figure 4 shows that EAGLE-3 with Qwen3-8B achieves relative throughput speedups of approximately 3.5×, while TiDAR-8B achieves 5.9× — a substantial gap that the paper attributes to parallel drafting and single-forward-pass architecture.

How TiDAR Positions Itself

TiDAR's conceptual positioning is elegant in its simplicity: rather than choosing between AR quality and diffusion parallelism, build a single model that does both in one forward pass. The key architectural insight is that the "free token slots" identified in Figure 1 can be used not just for drafting or verification, but for both simultaneously, using different attention patterns on different parts of the sequence.

The paper makes this positioning explicit by framing TiDAR through two lenses:

From the diffusion perspective: TiDAR solves the quality problem by using autoregressive rejection sampling on the drafted tokens, ensuring that the final output adheres to the chain-factorized joint distribution $p_{\text{AR}}$ even though the drafting was done under the independence-assuming $p_{\text{Diff}}$. The draft tokens only need to be good enough to be accepted at a high rate — they do not need to be perfect. This is why one-step diffusion drafting (with full masking during training) works: the model learns to predict tokens from a fully masked context, and the quality of those predictions, while imperfect, is sufficient to achieve high acceptance rates when verified against the AR distribution.

From the speculative decoding perspective: TiDAR achieves what no prior speculative method has: the draft model is the base model (maximum capacity), drafting is fully parallel (diffusion over all mask tokens simultaneously), and drafting and verification share a single forward pass (no sequential overhead). Table 1 captures this crisply: TiDAR is the only method with checkmarks in all three columns — shared model capacity, high drafting capacity, parallel decoding, and parallel-to-verification.

The paper does not claim to have invented either diffusion language models or speculative decoding — it explicitly builds on Block Diffusion [12] for the hybrid attention mask and on rejection sampling [11] for the verification procedure. The contribution is the architectural synthesis that makes these components work together in a single forward pass, and the empirical demonstration that this synthesis achieves a quality-efficiency frontier that neither pure diffusion nor pure speculative decoding approaches.

There is also an implicit positioning relative to the broader "scaling laws" conversation. Just as the Chinchilla scaling laws (Hoffmann et al., 2022) showed that the optimal allocation of pretraining compute between model size and data is not obvious a priori, TiDAR suggests that the optimal allocation of inference compute between drafting and verification — and between sequential and parallel operations — is not solved by simply making the AR model larger. The 5.91× throughput improvement with minimal quality loss is a concrete instance of the principle that architectural efficiency can substitute for scale at inference time, a theme that resonates with the compute-optimal test-time scaling analysis (Snell et al., 2024) discussed in the reference example.

3. Technical Approach

3.1 Reader Orientation

TiDAR is a single language model that, in one forward pass, simultaneously verifies (via autoregressive sampling) the tokens it drafted in the previous forward pass and pre-drafts (via diffusion) the tokens needed for the next forward pass. The system solves the problem that autoregressive generation is memory-bound and slow (one token per forward pass) while diffusion generation is fast but low-quality (parallel tokens are generated under an independence assumption); TiDAR's shape of solution is to use the fast-but-imperfect diffusion mode to propose candidate tokens and the slow-but-accurate AR mode to verify them, both within the same model and the same forward pass, so that the diffusion computation occupies otherwise-idle GPU capacity.

3.2 Big-Picture Architecture (Diagram in Words)

The TiDAR system has four major components operating within a single transformer model:

1. The Base Transformer Backbone: A standard autoregressive language model (initialized from Qwen2.5 or Qwen3 pretrained weights) whose attention mask is modified during both training and inference to support two distinct attention patterns on different parts of the input sequence.

2. The Causal (AR) Subsystem: The leftmost portion of the input sequence, comprising the prefix tokens and the tokens drafted in the previous step, is processed with standard causal self-attention. This computes the chain-factorized joint distribution $p_{\text{AR}}$ needed for rejection sampling.

3. The Bidirectional (Diffusion) Subsystem: A block of mask tokens appended to the right of the causal section is processed with block-causal bidirectional attention (causal with respect to the prefix, bidirectional within the mask block). This computes the marginal distribution $p_{\text{Diff}}$ needed for parallel drafting of the next step's candidate tokens.

4. The Rejection Sampling and Selection Logic: An algorithmic procedure (not a learned component) that compares the AR predictions against the diffusion-drafted tokens from the previous step, accepts tokens that match, and selects the appropriate pre-drafted tokens for the next step based on how many tokens were accepted.

Information flows through these components in a fixed cycle: (a) the prefix and previously-drafted tokens enter the causal attention region, producing AR logits; (b) simultaneously, mask tokens in the bidirectional region produce diffusion logits conditioned on all possible acceptance outcomes; (c) rejection sampling accepts or rejects each drafted token by comparing against the AR logits; (d) based on the acceptance length, the corresponding pre-drafted mask tokens are selected and become the "drafted tokens" for the next forward pass.

3.3 Roadmap for the Deep Dive

• First, the dual-mode training objective (Section 3.1 in the paper), because understanding how the model learns to compute both $p_{\text{AR}}$ and $p_{\text{Diff}}$ from the same weights is prerequisite to understanding inference. I will explain the hybrid attention mask, the sequence-doubling architecture, the full-mask strategy, and the loss balancing formula.
• Second, the parallel self-speculative generation procedure (Section 3.2), because this is where the speedup originates. I will walk through exactly what happens in a single forward pass — which tokens get which attention pattern, how rejection sampling operates, how pre-drafting works across multiple acceptance hypotheses, and how KV caching interacts with all of this.
• Third, the training and inference optimizations (Section 3.3), including the full-mask training strategy's benefits, the loss balancing mechanism, and the attention mask reuse technique that makes decoding efficient.
• Fourth, I will connect these mechanisms back to the "free token slots" concept from Figure 1 to explain why the architecture works from a hardware perspective — what makes the drafting computation essentially free in the memory-bound regime.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily an architectural design paper whose core idea is that a single transformer model can be trained to operate in two attention modes — causal for AR quality and block-causal bidirectional for diffusion parallelism — and that at inference time these two modes can be composed via rejection sampling within a single forward pass to achieve the speed of diffusion with the quality guarantee of autoregression.

---

Dual-Mode Backbone Training: Teaching One Model to Think in Two Attention Patterns

The fundamental challenge in training TiDAR is that the model must learn to compute two different conditional distributions from the same parameters within the same forward pass: the chain-factorized joint distribution $p_{\text{AR}}(x_i \mid x_{<i})$ (standard next-token prediction under causal attention) and the marginal distribution $p_{\text{Diff}}(x_i \mid \tilde{x})$ where $\tilde{x}$ is a corrupted context containing mask tokens (denoising under bidirectional attention). The paper achieves this through a sequence-level attention hybridization that partitions the input into two regions with different attention rules.

The Hybrid Attention Mask

The core architectural mechanism is the structured attention mask applied during both training and inference. The input sequence — which is double the length of the original text — is divided into two contiguous segments:

Segment 1 (the causal prefix): Contains the original clean tokens of the input sequence. These tokens attend to themselves and to earlier positions using standard causal self-attention: token at position $i$ can attend to tokens at positions $0$ through $i$, but not to any position $j > i$. This region computes the standard autoregressive next-token prediction distribution.

Segment 2 (the diffusion block): Contains only mask tokens [M] of equal length to the original sequence. These mask tokens attend to all clean tokens in Segment 1 (the entire prefix, without causality restriction) and attend to all other mask tokens in Segment 2 bidirectionally (each mask token can attend to every other mask token). Critically, the attention from Segment 1 to Segment 2 is blocked — clean tokens cannot see mask tokens. This region is "block-causal" with respect to the prefix and "intra-block bidirectional" within the mask region.

The paper visualizes this mask in Figure 3 (Left) using a concrete example with block length 3. For a sequence A, B, C, D, the doubled input is [A, B, C, D, M, M, M]. The causal region (positions 0-3) has a standard lower-triangular attention matrix. The diffusion region (positions 4-6) has attention to all positions in the causal region (full columns for positions 0-3) and full bidirectional attention within the diffusion block (a dense 3×3 submatrix for positions 4-6). The upper-right quadrant (causal tokens attending to mask tokens) is entirely zeroed out.

This design is a modification of Block Diffusion's attention mask [12]. Block Diffusion uses intra-block bidirectional and inter-block causal attention across all blocks — the entire sequence is partitioned into blocks, each block is bidirectional internally, and blocks are causally related to each other. TiDAR modifies this by making only the last block (the diffusion block) bidirectional, while keeping the rest (the prefix) purely causal. The paper explains the motivation:

"The benefits of doing so are two-folds. First, it allows us to compute the chain-factorized joint distribution just like in AR models. This, as we will show in Section 3.2, will allow us to conduct rejection sampling using the joint distribution with high quality guarantee and evaluate likelihood the same way as AR in terms of efficiency. Second, computing next token prediction (NTP) loss on the prefix becomes possible during model pre-training and finetuning."

The second benefit requires explanation. In Block Diffusion's architecture, because tokens within a block attend bidirectionally, a token in the middle of a block can see tokens that come after it in the sequence. If you tried to compute standard next-token prediction loss on position $i$ — which requires predicting token $x_{i+1}$ given $x_{\leq i}$ — the model would have already seen $x_{i+1}$ through the bidirectional attention, creating label leakage. TiDAR avoids this by making the prefix fully causal: token at position $i$ in the prefix can only attend to positions $0$ through $i$, so predicting $x_{i+1}$ from $x_{\leq i}$ is a valid autoregressive task with no information leakage. This means TiDAR can compute next-token prediction loss on every position in the causal prefix, providing a dense training signal that Block Diffusion cannot access.

Sequence Doubling and Label Alignment

To implement this dual-mode computation, the input sequence length is effectively doubled: the model takes an input of length $2S$ where $S$ is the original sequence length. The first $S$ positions contain clean tokens; the second $S$ positions contain mask tokens.

The label alignment differs between the two segments:

• For the causal (AR) segment: The target labels are shifted by one position, matching the standard next-token prediction objective. The model predicts token $x_{i+1}$ from the hidden state at position $i$ (which has causally attended to $x_0, \ldots, x_i$). The loss is computed for positions $0$ through $S-2$.

• For the diffusion segment: The target labels remain aligned with their input positions — there is no shift. The model predicts token $x_i$ from the hidden state at mask position $i$ (which has attended to the entire prefix bidirectionally and to all other masks bidirectionally). The loss is computed for all $S$ mask positions.

This dual alignment is necessary because the two modes answer different questions: the AR mode asks "what comes next given what we've seen so far?" while the diffusion mode asks "what token belongs here given the full context and the fact that this position is currently masked?"

The Full-Mask Strategy and Why It Matters

A critical design choice is that, during training, all tokens in the diffusion section are set to mask tokens — there is no stochastic masking schedule as in traditional diffusion language models (where a random subset of positions are masked according to a noise schedule). The paper states:

"We propose a simpler and more effective training strategy by setting all tokens in the diffusion section to mask tokens. This eliminates the hassle of deciding the optimal masking strategy."

This choice is not merely a simplification — it is motivated by three specific benefits that the paper enumerates:

Benefit 1: Denser diffusion loss signal. In traditional diffusion training with random masking, the loss is computed only on the subset of positions that are actually masked (since those are the only positions where the model needs to perform denoising). With full masking, the loss is computed on every position in the diffusion section — all $S$ mask tokens contribute to the diffusion objective. This provides a richer training signal per example.

Benefit 2: Easier loss balancing. The paper explains a subtle training dynamic: in randomized masking, the number of masked positions varies across training samples, which means the scale of the diffusion loss varies (fewer masked positions → fewer loss terms → smaller total loss). Meanwhile, the AR next-token prediction loss always has $S-1$ terms. This mismatch makes it difficult to balance the two losses with a fixed weighting factor. With full masking, both losses have a consistent number of terms (approximately $S$), allowing straightforward balancing:

"By masking all tokens, the number of loss terms is consistent across both types of losses (equal to the sequence length), allowing straightforward balancing using a user-defined weighting factor."

Benefit 3: Train-test consistency for one-step diffusion. At inference time, TiDAR uses one-step diffusion drafting: the diffusion block contains only mask tokens, and the model predicts all tokens in a single forward pass (no iterative denoising). Training with full masks matches this inference condition exactly — the model never sees partially denoised sequences during training, so there is no train-test distribution mismatch. This is explicitly stated:

"It allows us to do one-step diffusion during inference, which makes the drafting process more efficient than multi-step denoising."

The paper validates this choice in an ablation (Table 5), comparing "Random" masking (traditional stochastic masking) against "Full" masking. For TiDAR with 4 drafts, full masking improves HumanEval average from 32.62% to 38.42% (+5.8 percentage points) and MBPP average from 48.63% to 50.96% (+2.3 points), while also improving tokens per NFE from 3.37 to 3.46. The efficiency gain is attributed to the model being better calibrated to the one-step denoising condition it encounters at inference.

The Training Objective

The training loss combines the two objectives with a balancing factor $\alpha$:

$\mathcal{L}_{\text{TiDAR}}(\theta) = \frac{1}{1 + \alpha} \left( \sum_{i=1}^{S-1} \frac{\alpha}{S-1} \cdot \mathcal{L}_{\text{AR}}(x_i, x_{i+1}; \theta) + \sum_{i=1}^{S-1} \frac{1}{S-1} \cdot \mathcal{L}_{\text{Diff}}([\text{mask}], x_i; \theta) \right)$

where $\alpha \in [0, 1]$ is the loss balancing factor that controls the relative weight of the AR and diffusion objectives, $\{x_i\}^S$ is the input sequence of length $S$, $\mathcal{L}_{\text{AR}}$ is the cross-entropy loss computed on the logits from the causal attention positions, and $\mathcal{L}_{\text{Diff}}$ is the cross-entropy loss computed on the logits from the bidirectional (masked) attention positions.

What it computes: For each position $i$ from $1$ to $S-1$ in the original sequence, the model produces two predictions — one from the causal path (predicting $x_{i+1}$ from $x_{\leq i}$) and one from the diffusion path (predicting $x_i$ from the full prefix and the mask context). The AR loss term at position $i$ penalizes the model if the causal prediction of token $x_{i+1}$ is incorrect. The diffusion loss term at position $i$ penalizes the model if the bidirectional prediction of token $x_i$ (from the corresponding mask position) is incorrect. Both are standard cross-entropy losses. The two terms are normalized by the sequence length $S-1$ and weighted by $\alpha$ and $1$ respectively, then the sum is divided by $1 + \alpha$ to keep the total loss scale independent of $\alpha$.

Why this form: The normalization by $\frac{1}{1+\alpha}$ ensures that when $\alpha = 1$ (equal weighting), the total loss is the arithmetic mean of the two average per-position losses. If this normalization were absent, increasing $\alpha$ would increase the total loss magnitude, which would be equivalent to changing the learning rate — the normalization decouples loss weighting from effective learning rate. The per-position normalization by $\frac{1}{S-1}$ ensures that the contribution of each objective is independent of sequence length, which matters because the diffusion loss and AR loss would otherwise have scales that depend on $S$. The paper sets $\alpha = 1$ for most experiments, meaning both objectives contribute equally to the gradient, and ablates this choice (Figure 6 shows that varying the effective AR-to-diffusion ratio from 0.8:1.0 to 1.2:1.0 produces consistent results, indicating the model is robust to this hyperparameter).

What alternatives would have been wrong: A naive summation $\mathcal{L}_{\text{AR}} + \mathcal{L}_{\text{Diff}}$ without per-position normalization would make the loss magnitude dependent on sequence length, which complicates learning rate scheduling across different training configurations. Using a fixed ratio without the $\frac{1}{1+\alpha}$ denominator would change the effective learning rate as $\alpha$ changes. Computing only the diffusion loss (as in pure dLMs) would lose the dense NTP signal from the causal prefix, which the paper argues is valuable for language modeling quality. Computing only AR loss with a separate diffusion head would not train the model to handle the bidirectional attention pattern, which is essential for parallel drafting.

---

Fully Parallelizable Self-Speculative Generation: How One Forward Pass Both Drafts and Verifies

The inference procedure is where TiDAR's architectural innovations translate into throughput gains. The key claim is that drafting (generating candidate future tokens) and verification (checking drafted tokens against the AR distribution) happen in the same forward pass, in parallel. This subsection walks through the exact mechanics, which are illustrated in Figure 2.

The Three-Part Sequence Structure During Decoding

At each decoding step, the input to the model is partitioned into three contiguous sections:

Section 1: Prefix tokens. These are the clean, already-generated-and-accepted tokens from all previous steps. They form the established context. In the first decoding step, this is just the input prompt. In subsequent steps, this grows by however many tokens were accepted in the previous step's rejection sampling.

Section 2: Tokens drafted from the last step. These are the candidate tokens that the model proposed (via diffusion) in the previous forward pass. They have not yet been accepted — they are hypotheses that will be verified in the current forward pass. If the previous step drafted $K$ tokens, this section contains $K$ token positions.

Section 3: Tokens pre-drafted for the next step. These are mask tokens that will be filled in (via diffusion) during the current forward pass, producing candidate tokens for the next step's verification. The number of mask tokens equals the draft length $K$.

The attention pattern applied to these three sections is:

• Section 1 → Section 1: Causal self-attention (standard AR).
• Section 1 → Section 2: Causal (tokens drafted from last step can attend to all prefix tokens).
• Section 2 → Section 2: Causal (later drafted tokens can attend to earlier drafted tokens — this is the autoregressive verification condition).
• Section 2 → Section 1: Blocked (prefix tokens cannot see the yet-unverified drafted tokens — this prevents information leakage from unverified hypotheses into the established context).
• Sections 1+2 → Section 3: Full bidirectional attention from the mask tokens to both the prefix and the drafted tokens. The mask tokens can see the entire context (prefix + all drafted hypotheses), which is what enables them to produce good predictions regardless of which subset of drafted tokens ultimately gets accepted.
• Section 3 → Section 3: Bidirectional within the mask block.
• Section 3 → Sections 1+2: Blocked (the prefix and drafted tokens cannot see the mask tokens).

The paper visualizes the exact decoding mask in Figure 3 (Right), showing a concrete example where the prefix length is 3 (tokens A, B, C), and the sampling-draft region contains tokens D, E, F (drafted from last step) followed by mask tokens M, M, M (for next step pre-drafting). The attention matrix shows a lower-triangular structure for the [A, B, C, D, E, F] submatrix (causal), and dense attention from the mask positions to all clean positions plus dense intra-mask attention.

The Rejection Sampling Mechanism

Once the model forward pass completes, the system has two sets of logits for each position in Section 2 (the drafted tokens):

• AR logits: Produced by the causal attention path at each position $i$ in the "drafted from last step" section. These represent $p_{\text{AR}}(x_i \mid x_{<i})$ — the probability distribution over the vocabulary at position $i$ given all preceding tokens (both the established prefix and the earlier drafted tokens, since attention is causal within Section 2).

• Draft token: The actual token that was placed at position $i$ during the previous step's diffusion drafting (when it was a mask token in Section 3 of the previous forward pass).

The rejection sampling procedure, following the standard speculative decoding algorithm [11], operates token by token from left to right within Section 2:

1. At position $i$, draw a random sample from the AR distribution $p_{\text{AR}}(x_i \mid x_{<i})$.
2. If the sampled token equals the drafted token at position $i$, accept the draft token and move to position $i+1$.
3. If the sampled token differs from the drafted token, reject the draft token at position $i$ (and implicitly all subsequent drafted tokens). The AR-sampled token becomes the accepted token at position $i$, and the verification stops — positions $i+1$ through $K$ are discarded regardless of what they contain.

This procedure guarantees that the sequence of accepted tokens is distributed according to $p_{\text{AR}}$ — exactly the same distribution that standard autoregressive decoding would produce. The paper states this implicitly through its connection to speculative decoding, which has the distribution preservation property proven in [11]. The draft tokens only affect speed (how many tokens are accepted per step), not the output distribution.

What the draft quality determines: The acceptance rate — the expected number of tokens accepted per verification step — depends on how well the diffusion-drafted tokens match the AR distribution. If the diffusion mode perfectly predicted the AR mode's output at every position, every draft token would be accepted and the speedup would be proportional to the draft length $K$. If the diffusion mode predicted poorly, most tokens would be rejected and the speedup would be minimal (you'd pay the cost of the forward pass but only get one or two tokens out of it). The paper's results (Table 2, with tokens per NFE ranging from 5.07 to 10.13 depending on the task and draft length) demonstrate that the diffusion mode, even with one-step denoising, achieves high enough acceptance rates to realize substantial throughput gains.

Pre-Drafting for the Next Step

The acceptance length — how many draft tokens survive rejection sampling — varies from step to step and is not known before the forward pass completes. Yet the model needs to produce draft tokens for the next step during the current forward pass (when Section 3's mask tokens are being denoised). The paper solves this with a technique inspired by Apple's MTP work [20]:

The model pre-drafts multiple candidate continuations, one for each possible acceptance length from 0 to $K$. Concretely, among the $K$ mask tokens in Section 3, the model produces predictions conditioned on different prefix outcomes:

• Proposal 0: If no tokens are accepted (length 0), the next step's context is just the current prefix.
• Proposal 1: If 1 token is accepted, the next step's context is the current prefix plus the first drafted token.
• Proposal 2: If 2 tokens are accepted, the context is the current prefix plus the first two drafted tokens.
• And so on, up to $K$.

For each hypothesis, the model predicts what the first token of the next draft block should be — this token becomes the $F', F'', F'''$ in Figure 2 (different predictions for different acceptance outcomes). The remaining mask positions in each hypothesis attend bidirectionally within their block to produce the full draft of $K$ tokens for that particular acceptance outcome.

After rejection sampling determines the actual acceptance length $L$, the system selects Proposal $L$ — the pre-drafted block conditioned on exactly $L$ accepted tokens — and these become the "tokens drafted from last step" (Section 2) for the next forward pass.

Why this works without label shifting: The paper notes a critical design choice: the diffusion predictions do not use label shifting. That is, the first mask token in Section 3 predicts the token that should appear at that position given the accepted context, not the token that appears one position later. This means the mask token immediately after the last accepted token predicts that position directly, rather than predicting the subsequent position (which would require shifting and would create a mismatch with which acceptance hypothesis is selected). Without label shifting, the first mask token in Proposal 0 predicts the token at position $|\text{prefix}|$, the first mask token in Proposal 1 predicts the token at position $|\text{prefix}|+1$, and so on — each proposal's first token predicts the next token after its respective prefix. This alignment ensures that no matter which proposal is selected, the drafted tokens correctly extend the chosen prefix.

The paper acknowledges a secondary consequence of this design:

"However, this comes at the cost that the AR outputs and every first position in the block will be predicting the same position (e.g. $E$ to $E''$, $F'$ to $F''$, $G'$ to $G''$), leading to a potential decision of choosing what to use for verification."

In other words, the AR prediction at the last accepted position and the diffusion prediction at the first draft position of the selected proposal are predicting the same token. The paper resolves this by introducing a mixing parameter $\beta$ (discussed in the next subsection).

KV Cache Management

TiDAR supports exact KV caching in a way that pure diffusion models do not. The procedure is:

1. During each forward pass, all tokens in Section 1 (prefix) and Section 2 (drafted from last step) are processed with causal attention, and their key-value pairs are stored in the cache.
2. After rejection sampling determines the acceptance length $L$ (where $L \leq K$):
   • The KV entries for the accepted drafted tokens (the first $L$ positions in Section 2) are retained — they become part of the prefix for the next step.
   • The KV entries for the rejected drafted tokens (positions $L+1$ through $K$ in Section 2) are evicted from the cache, since those tokens are discarded and will not appear in future contexts.
3. The mask tokens in Section 3 do not produce cached KV entries (they are re-generated with new masks in each step).

This approach means that TiDAR never recomputes the KV representations of any token. The paper explicitly contrasts this with Block Diffusion and other diffusion caching methods:

"It is worth mentioning that we do not waste any computation by recomputing the KV cache of any token, which makes our method extremely efficient compared to Block Diffusion, SBD, and the cache methods used in pure diffusion (e.g. Fast-dLLMs [13, 24] and d-KV Cache [27])."

The efficiency comes from the fact that once a token is verified and accepted, its KV representation is final — no subsequent denoising step will change it, so there is no need for recomputation. In contrast, pure diffusion models that iteratively denoise blocks must recompute attention over tokens whose representations change as they are progressively "unmasked."

Trusting AR vs. Diffusion Predictions

The paper addresses a subtlety: at the boundary between accepted tokens and the first draft token, the AR prediction and the diffusion prediction are estimating the same token (the next token after the accepted prefix). In principle, these should be identical if the model is perfectly trained. In practice, they may differ slightly due to the different attention patterns (causal vs. bidirectional) and training dynamics.

The paper introduces a logit mixing parameter $\beta$ to combine these predictions before sampling:

$\arg\max_i \{\beta \cdot \text{logits}^{\text{ar}}_i + (1 - \beta) \cdot \text{logits}^{\text{diff}}_i, \ i \in |V|\}$

where $\beta \in [0, 1]$ controls how much to trust the AR prediction versus the diffusion prediction, $\text{logits}^{\text{ar}}_i$ is the unnormalized log-probability from the causal attention path for token $i$, $\text{logits}^{\text{diff}}_i$ is the unnormalized log-probability from the bidirectional attention path for token $i$, and $|V|$ is the vocabulary size.

What it computes: A weighted average of the AR and diffusion logits at the shared prediction position, followed by argmax to select the token that will be compared against the draft. When $\beta = 1$, the model relies entirely on the AR prediction (which has access to the full causal context). When $\beta = 0$, it relies entirely on the diffusion prediction (which has bidirectional context over the draft block).

Why this form: Averaging in logit space (rather than probability space) preserves the relative scale of the two predictions — a token that both paths strongly prefer will have its logit boosted, while a token that one path strongly disprefers will have its logit suppressed. The paper shows in Figure 6 that the model is robust to this choice — performance is nearly flat as $\beta$ varies from 0 to 1 across different training configurations — indicating that the AR and diffusion predictions are well-calibrated to each other. This robustness is presented as evidence of successful dual-mode training:

"This shows that our model is well trained so that no matter what logits we choose to use for verification, the quality is preserved because in the ideal case (i.e. well-trained), these two outputs will be strictly the same. This also indicates that it is the autoregressive rejection sampling that guarantees the quality-speedup trade-offs rather than the AR knowledge."

For the main results, the paper reports two modes: "Trust AR" (setting $\beta$ closer to 1) and "Trust Diffusion" (setting $\beta$ closer to 0). At the 8B scale, "Trust Diffusion" slightly outperforms on average (65.31% vs. 63.90%, Table 2), particularly on math tasks, suggesting that the diffusion predictions benefit from the additional bidirectional context for certain reasoning problems.

The Full Step-by-Step Generation Cycle

To make the procedure completely concrete, here is exactly what happens in one decoding step, assuming a draft length of $K = 3$ and starting from a prefix of length $P$:

Step $t$ (not the first step):

1. Input construction: The input has length $P + 2K$. Positions $0$ to $P-1$ contain the established prefix (clean tokens, KV cached). Positions $P$ to $P+K-1$ contain the draft tokens from step $t-1$ (the pre-draft proposal that was selected). Positions $P+K$ to $P+2K-1$ contain mask tokens [M].
2. Attention mask application: Positions $0$ to $P+K-1$ use causal attention (lower triangular). Positions $P+K$ to $P+2K-1$ use causal attention to positions $0$ to $P+K-1$ (they can see the full prefix plus all drafts) and bidirectional attention within their own block.
3. Forward pass: The model produces logits at every position. The logits at positions $P$ to $P+K-1$ are the AR predictions for verifying the draft tokens. The logits at positions $P+K$ to $P+2K-1$ are the diffusion predictions for next-step drafting.
4. Rejection sampling: Starting from position $P$, for $i = 0, 1, 2$: sample from the AR distribution at position $P+i$. If it matches the draft token, accept and continue. If not, reject all remaining drafts, use the AR sample as the token at $P+i$, set acceptance length $L = i$, and stop. If all $K$ tokens match, $L = K$.
5. Proposal selection: Among the pre-drafted mask tokens, select Proposal $L$ — the predictions from the mask positions that were conditioned on exactly the first $L$ draft tokens being accepted.
6. KV cache update: Retain KV entries for positions $0$ through $P+L-1$ (prefix + accepted drafts). Evict KV entries for positions $P+L$ through $P+K-1$ (rejected drafts). Mask token KV entries are not cached.
7. State transition: The new prefix length becomes $P + L$. The selected proposal tokens become the "drafted from last step" for step $t+1$. A new block of $K$ mask tokens is appended for pre-drafting.

First decoding step (prefill): The input prompt is encoded causally (standard AR prefill). A block of $K$ mask tokens is appended. The mask tokens attend to the full prompt bidirectionally. The forward pass produces draft tokens at the mask positions, which become the "drafted from last step" for the first decoding step. The decoding cycle then proceeds as above.

---

Training and Inference Optimization Details

Loss Balancing with $\alpha = 1$

The paper states that $\alpha = 1$ is used for most cases, meaning the AR and diffusion losses contribute equally to the gradient. Equal weighting is motivated by the observation that both losses have approximately equal variance after full masking:

"our corrupted sequence is fully masked, which makes loss balancing easier due to the equal variance of the loss scale calculated from the AR and diffusion logits."

The ablation in Figure 6 tests three configurations with AR-to-diffusion weight ratios of 0.8:1.0, 1.0:1.0, and 1.2:1.0, finding consistent performance across all ratios — evidence that the training objective is not highly sensitive to $\alpha$.

Attention Mask Initialization and Reuse

A practical optimization for inference: because every forward pass uses the same attention pattern with the same sequence length (the prefix grows but the drafting + mask regions are fixed), the attention mask can be initialized once and sliced for each step:

"We reorder the draft part and prefix so that we can initialize one block attention mask of size (q_len, q_len + max_sequence_len) and slice the cached mask for each step without recomputing it for Flex Attention."

In Figure 3 (Right), the paper illustrates this: a large mask is pre-computed at model initialization with dimensions (max_seq_len + block_size, max_seq_len + block_size). For each forward pass, a submatrix is sliced based on the current prefix length. The reordering (moving the draft tokens before the mask tokens in the input layout) ensures that the same mask structure applies regardless of prefix length, enabling the slicing optimization. This avoids the overhead of constructing attention masks on-the-fly during generation.

For the prefill step (first forward pass with only the prompt), a separate mask is used (Figure 7 in Appendix B): the prompt tokens attend to themselves causally, and the appended mask tokens attend to all prompt tokens bidirectionally. This prefill mask is also initialized once and sliced per sample.

Training Hyperparameters and Scale

The paper provides specific training configurations:

• TiDAR 1.5B: Continual pretraining from Qwen2.5-1.5B for 50B tokens on NVIDIA H100 GPUs with a global batch size of 2M tokens using standard DDP (Distributed Data Parallel). Trained under block sizes (draft lengths) of 4, 8, and 16.
• TiDAR 8B: Continual pretraining from Qwen3-8B for 150B tokens, block size 16 only. Gradient checkpointing is enabled for memory efficiency.
• Shared settings: Cosine learning rate schedule with max_lr = 1e-5, min_lr = 3e-6, warmup fraction of 1%, max sequence length of 4096 (doubled to 8192 with mask tokens), distributed Adam optimizer [41], BFloat16 precision throughout. The training framework is a modified Megatron-LM [42] with Torchtitan [43] support.

The paper notes that the 50B tokens for 1.5B and 150B tokens for 8B represent "continual pretraining" — the models start from strong AR checkpoints and only need to learn the diffusion capability, not language modeling from scratch. This explains the data efficiency relative to training a model from scratch.

No Hyperparameters During Inference

An important practical property that the paper emphasizes:

"Unlike traditional diffusion models, TiDAR has no hyperparameters to tune during inference."

Pure diffusion models require choosing a noise schedule, a number of denoising steps, and often a confidence or entropy threshold for adaptive decoding. Block Diffusion requires a threshold for when to stop decoding within a block. TiDAR's inference procedure is parameter-free: the draft length $K$ is set at training time (the block size used during training), and the rejection sampling algorithm operates automatically without any tunable knobs. The only flexibility discussed is the $\beta$ logit mixing parameter (trusting AR vs. diffusion), which the paper shows is insensitive and can be fixed at $\beta = 1$ or $\beta = 0$.

Flexibility for Different Scenarios

Despite the "no hyperparameters" claim, the paper notes that the block (draft) length $K$ can be adjusted at inference time in a "zero-shot manner" to accommodate different compute profiles. Training with block size 16 produces a model that can draft 16 tokens per step; if the serving scenario has a different "free token slot" budget (e.g., a GPU with more or less memory bandwidth headroom), the effective draft length can be reduced without retraining. The paper's results in Figure 4 and Table 2 show models with block sizes 4, 8, and 16, each producing different quality-throughput tradeoffs — larger blocks yield higher throughput (more T/NFE) with marginally lower quality due to slightly lower per-token acceptance rates.

---

Connecting the Architecture to the "Free Token Slots" Concept

To understand why TiDAR achieves its speedup, it is essential to connect the architectural description back to the hardware motivation from Figure 1. The key cost accounting is:

In standard AR decoding, one forward pass processes $P+1$ tokens (the prefix of length $P$ plus one new token position) and produces one output token. The latency $T$ of this forward pass is dominated by loading model weights from GPU memory, not by the FLOPs of computing attention over $P+1$ tokens.

In TiDAR decoding, one forward pass processes $P + 2K$ tokens (prefix of length $P$, $K$ draft tokens from last step, $K$ mask tokens for next step) and produces up to $K+1$ output tokens (if all $K$ drafts are accepted, plus the first token of the next draft block). Because the forward pass is still memory-bound for these additional $2K-1$ token slots — they fit within the "free token slots" plateau in Figure 1 — the latency remains approximately $T$. So you get roughly $K$ times the output tokens for the same wall-clock time.

The "free" qualifier is not literal — the additional tokens do consume some compute — but the compute is essentially wasted anyway in the memory-bound regime because the GPU's arithmetic units would otherwise be idle waiting for memory transfers. By filling the idle compute capacity with useful drafting work, TiDAR converts wasted FLOPs into throughput improvement.

The paper's measured speedups (4.71× for 1.5B, 5.91× for 8B) are lower than the theoretical $K+1$ factor (which would be 5×, 9×, or 17× for block sizes 4, 8, 16) because the acceptance rate is not 100% — some draft tokens are rejected. The average tokens per NFE values in Table 2 (e.g., 6.50 to 9.43 for 1.5B across different tasks, 7.30 to 10.13 for 8B) represent the realized efficiency after accounting for rejection sampling. The "relative AR throughput speedup" in Figure 4 further accounts for any small latency differences between TiDAR and AR forward passes, yielding the final 4.71× and 5.91× figures.

4. Key Insights and Innovations

Innovation 1: The "Single Forward Pass" Architecture as a Resolution to the Drafting-Verification Sequential Bottleneck

The dominant structure in speculative decoding — from the original Leviathan et al. [11] formulation through EAGLE-3 [18] and DeepSeek-V3 MTP [19] — is fundamentally two-phase: draft, then verify. Even when the draft model shares the base model's hidden states (as in EAGLE and Medusa [28]), the drafting step is sequential to the verification step because you must complete the base model's forward pass before the drafter can begin producing candidate tokens. This imposes a hard latency floor: the total time is T_verify + T_draft, where T_draft cannot be hidden inside T_verify.

TiDAR's key conceptual move is to collapse drafting and verification into a single forward pass by exploiting the fact that, in the memory-bound regime, adding token positions costs negligible latency. This is not an implementation optimization — it is an architectural principle that was not articulated in prior work. Previous methods asked, "how can we make the draft model faster or more accurate?" TiDAR asks, "why should the draft model run in a separate forward pass at all?" The answer derives from the hardware profiling in Figure 1: if the GPU's compute units are idle waiting for weight loads during AR decoding, then computing draft tokens on those idle units within the same forward pass is effectively free.

Table 1 makes this structural difference explicit. Every prior speculative method has at least one "✗" in the "Parallel to Verification" column — meaning drafting is sequential to verification, adding latency. TiDAR is the first method to get a "✓" in that column, and it gets this not by making the drafter faster but by eliminating the sequential dependency entirely. This is a fundamental architectural shift rather than an incremental refinement: it changes the topology of the computation graph from verify → draft → verify → draft to (verify + draft simultaneously) → select → (verify + draft simultaneously).

The significance extends beyond the measured speedup. By demonstrating that drafting and verification can coexist in one forward pass, TiDAR establishes a new design space for language model inference architectures — one where the boundary between "the model" and "the speculative mechanism" dissolves. The model is the speculative mechanism. A future system designer does not need to choose a draft model, manage separate draft-model weights, or pay the latency cost of sequential drafting phases. They only need to ensure that the combined drafting and verification token budget fits within the memory-bound plateau for their hardware.

Innovation 2: Rejection Sampling as the Quality Guarantee, Not Just a Speedup Mechanism

Speculative decoding is typically understood as a throughput optimization: use a small draft model to guess tokens, verify with the large model, gain speedup from the acceptance rate. The quality guarantee — that the output distribution is identical to the target model's — is a theoretical property but is not typically the reason the method is used. Practitioners adopt speculative decoding for speed, not for its distribution-preservation proof.

TiDAR repurposes rejection sampling into a fundamentally different role: it is the mechanism that makes parallel diffusion generation viable as a drafting strategy. Without rejection sampling, TiDAR's diffusion drafts would produce output from p_Diff — the marginal distribution under the token independence assumption — which is known to degrade quality as the number of parallel tokens increases. The APD [17] results cited in the paper show Dream-7B's GSM8K accuracy dropping 10% when going from one to two tokens per step. TiDAR achieves 5-10 tokens per forward pass (Table 2) with no quality degradation relative to AR at 1.5B, and minimal loss at 8B. The acceptance of this many tokens per step would be impossible if the diffusion drafts were evaluated on their own quality; it is only because rejection sampling filters them against p_AR at every position that the final output maintains AR-level quality.

This is a reconceptualization of what the diffusion mode is for. In prior diffusion language models, the diffusion process produces the output. The quality of the output is therefore directly limited by the quality of the diffusion model's predictions under the independence assumption. In TiDAR, the diffusion process proposes candidates that are filtered by an AR verifier. The quality of the output is determined by the AR verifier, not the diffusion drafter. The diffusion mode only needs to be good enough to achieve a high acceptance rate — a strictly weaker requirement than generating high-quality output directly.

This framing is visible in the paper's ablation on trusting AR vs. diffusion predictions (Figure 6). As β varies from 0 (fully trust diffusion) to 1 (fully trust AR), the quality remains essentially flat. The paper interprets this as evidence that "the autoregressive rejection sampling that guarantees the quality-speedup trade-offs rather than the AR knowledge." In other words, even if the diffusion predictions were used directly for sampling (without AR mixing), the rejection sampling loop would still filter them to match p_AR, preserving quality. The robustness to β is not a trivial observation — it demonstrates that the architecture is quality-robust by construction, not by careful tuning of the drafting quality.

This idea — that a weak-but-parallel proposal distribution can be composed with a strong-but-sequential verification distribution to achieve the speed of the former and the quality of the latter — is a conceptual template that generalizes beyond the specific attention-mask design of TiDAR. Any architecture that can compute both a marginal proposal distribution and a conditional verification distribution within shared parameters could adopt this pattern.

Innovation 3: The "Drafting Capacity = Base Model Capacity" Principle as an Alternative to Separate Draft Models

Prior speculative decoding approaches face a fundamental tradeoff: the draft model must be smaller (or at least cheaper per token) than the target model to provide speedup, but smaller models produce lower-quality drafts, reducing acceptance rates and thus limiting achievable speedup. EAGLE and Medusa attempt to mitigate this by giving the drafter access to the base model's hidden states, increasing drafting capacity without a fully separate model — but the drafter is still a distinct set of parameters (extra layers or heads) with lower representational capacity than the full base model. As the paper notes in Table 1, these methods achieve "Mid" drafting capacity.

TiDAR makes a clean break from this paradigm: the drafting model and the verification model are literally the same model operating in different attention modes. There is no separate draft model, no extra layers, no additional decoding heads. The diffusion draft at each position is produced by the same transformer layers, the same attention weights, and the same feedforward networks that produce the AR verification logits. The only difference is the attention mask — causal for verification, block-causal bidirectional for drafting.

This has a specific consequence for acceptance rates that the paper highlights: "our raw acceptance rate (T/NFE) is higher than those of EAGLE-3 open weights, and more importantly, our conversion rate (from T/NFE to T/s) is higher, thanks to the parallel drafting and sampling with a single model forward." The higher raw acceptance rate follows directly from the higher drafting capacity — a full 1.5B or 8B model, even operating with the cruder bidirectional attention pattern, produces better token predictions than a shallower draft model with access to hidden states. The higher conversion rate from T/NFE to T/s follows from the single-forward-pass architecture (Innovation 1).

What makes this a conceptual innovation rather than just an engineering choice is that it identifies shared-weight dual-mode operation as an alternative axis for improving speculative decoding beyond the standard draft-model-quality-vs-cost tradeoff. The field's default assumption has been that the draft and target models must be different — if they were the same, there would be no speedup because each draft token would cost as much as a target token. TiDAR breaks this assumption by noting that in the memory-bound regime, draft tokens don't cost as much as target tokens because they share the weight-loading cost of the same forward pass. The draft model can be the base model because the base model has idle compute capacity.

Innovation 4: Full-Mask Training as a Principle for Train-Test Consistency in One-Step Diffusion Drafting

Traditional diffusion language model training uses a stochastic masking schedule: during training, a random subset of token positions are replaced with mask tokens according to a noise schedule (e.g., a cosine schedule that determines what fraction of tokens are masked), and the model learns to recover the clean tokens from the partially-masked sequence. At inference time, the model iteratively denoises — starting from all masks, predicting some tokens, remasking others, and repeating.

TiDAR does something different and counterintuitive: it sets all positions in the diffusion section to mask tokens during training, eliminating the masking schedule entirely. The paper presents this initially as a simplification ("This eliminates the hassle of deciding the optimal masking strategy"), but the deeper justification is a principle of train-test consistency for one-step diffusion. At inference time, TiDAR's diffusion block contains only mask tokens and generates all draft tokens in a single forward pass. Training with any form of partial masking would create a distribution mismatch: the model would be trained to denoise partially-clean sequences but tested on fully-masked sequences. By training with full masks, the model's training distribution exactly matches its inference condition.

This principle has three downstream effects that the paper documents in Table 5:

1. Quality improvement: Full masking improves HumanEval average from 32.62% to 38.42% at 4 drafts (+5.8 points), representing a substantial gain from a training strategy change alone.
2. Efficiency improvement: Tokens per NFE increases from 3.37 to 3.46, suggesting the model produces higher-quality drafts (and thus higher acceptance rates) when trained with full masks.
3. Loss balancing simplification: With full masks, both the AR and diffusion losses always have consistent numbers of terms (approximately S), eliminating the need to account for variable numbers of masked positions when setting loss weights.

The conceptual contribution here is the recognition that when diffusion is used as a drafting mechanism (not an output mechanism), multi-step denoising is unnecessary. The paper explicitly notes: "We found one step is sufficient to produce draft tokens whose quality is good enough to secure high acceptance rate." This is a strong claim that runs counter to the intuition from continuous diffusion (images, audio) where more denoising steps generally improve quality. In the discrete text domain with rejection sampling, the marginal quality gain from additional denoising steps does not justify the additional forward passes — a single step already produces drafts that the AR verifier accepts at a high rate, and rejected tokens cost only the wasted draft slot, not output quality.

This insight simplifies the diffusion training pipeline dramatically. There is no noise schedule to design, no number of inference steps to tune, and no train-test gap to manage. The training objective collapses to: teach the model to predict every token from a fully-masked context, which is essentially a cloze-filling task with bidirectional context. This is a significantly simpler training problem than learning a full denoising trajectory.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper evaluates on a broad suite of downstream benchmarks spanning coding, math, factual knowledge, and commonsense reasoning. Coding tasks include HumanEval [44], HumanEval+ [45], MBPP [46], and MBPP+ [45]; math tasks include GSM8K [23] and Minerva Math [47]; factual knowledge is assessed via MMLU [48]; and commonsense reasoning via ARC-Easy, ARC-Challenge [49], HellaSwag [50], PIQA [51], and Winogrande [52]. The specific evaluation configurations (number of few-shot examples, generation length, metric type) are detailed in Appendix A (Table 6). All evaluations use the lm_eval_harness framework [38], version 0.4.8.

• Base model(s). The paper uses continual pretraining from existing AR checkpoints rather than training from scratch. At the 1.5B scale, TiDAR is initialized from Qwen2.5-1.5B [36]. At the 8B scale, TiDAR is initialized from Qwen3-8B [37]. An intermediate 4B variant initialized from Qwen3-4B is also mentioned for likelihood evaluation in Table 3. The paper argues that initializing from strong AR models allows the training to focus on learning the diffusion capability rather than language modeling from scratch, enabling data efficiency (50B tokens for 1.5B, 150B tokens for 8B). The choice of Qwen2.5 and Qwen3 is motivated by their strong performance relative to other open-source models at comparable sizes, as shown in the baseline comparisons in Tables 2 and 3.

• Metrics. The primary metrics are twofold:

  • Quality metrics: Task-specific scores — Pass@1 for coding tasks (HumanEval, HumanEval+, MBPP, MBPP+), strict-match accuracy for GSM8K, exact-match accuracy for Minerva Math, and standard accuracy (or accuracy normalized) for MMLU and commonsense reasoning tasks. For generative tasks (Tables 2 and 4-5, Figures 4-5), these measure the quality of sampled outputs. For likelihood tasks (Table 3), these measure performance when evaluating the model's probability estimates on multiple-choice questions.
  • Efficiency metrics: Tokens per NFE (network function evaluation, i.e., tokens produced per model forward pass) and relative AR throughput speedup measured in tokens per wall-clock second on a single NVIDIA H100 GPU at batch size 1. The throughput speedup uses the AR model within the same size group as the baseline (Qwen2.5-1.5B for 1.5B comparisons, Qwen3-8B for 8B comparisons). The paper emphasizes that tokens-per-NFE captures algorithmic efficiency (how many tokens the model attempts per step, accounting for rejection sampling acceptance rate), while tokens-per-second captures real-world speedup including any latency differences between TiDAR and AR forward passes.

• Baselines. The paper compares against several categories of models:

  • Standard AR models: Llama3.2-1B [39], SmolLM2-1.7B [40], Qwen2.5-0.5B and Qwen2.5-1.5B [36], Qwen3-1.7B, Qwen3-4B, and Qwen3-8B [37]. These represent the quality ceiling that TiDAR aims to match while providing higher throughput.
  • Diffusion language models: Dream-7B [9], LLaDA-8B [8] — the leading open-source diffusion LLMs at comparable scales.
  • Block Diffusion [12]: Trained by the authors under the same training recipe as TiDAR (same base initialization, data, and hyperparameters) to enable fair architectural comparison. Block Diffusion is evaluated at 1.5B (initialized from Qwen2.5-1.5B) and at 4B (initialized from Qwen3-4B). For likelihood evaluation, Block Diffusion uses Monte Carlo sampling with 128 steps (indicated by italicized names in Table 3).
  • Speculative decoding: EAGLE-3 [18] with Qwen3-8B-Instruct, using publicly available weights from AngelSlim and Tengyunw. The paper notes that EAGLE-3 is tested with the instruct model because corresponding weights for the base model were not available. EAGLE-3 represents the state-of-the-art in speculative decoding throughput at comparable model scales.

• Generation budget / compute accounting. The paper does not use a FLOPs-based budget in the style of the reference example. Instead, efficiency is measured directly via:

  • T/NFE (tokens per network function evaluation): The average number of output tokens produced per model forward pass, accounting for rejection sampling acceptance rates. This is reported for TiDAR in parentheses in Table 2 (e.g., 6.50 for TiDAR 1.5B on HumanEval with block size 4, 7.30 for TiDAR 8B on HumanEval with trust-diffusion mode).
  • Wall-clock throughput: Measured in tokens per second on a single H100 GPU at batch size 1. The paper benchmarks TiDAR against AR baselines, Block Diffusion (with different decoding thresholds), and EAGLE-3 using native PyTorch with Flash Attention 2 [15] and Flex Attention [35]. All comparisons use the same hardware, batch size, and prompts from downstream generative tasks.
  • For diffusion model baselines (Dream, LLaDA): The paper decodes one token per NFE because "this very often guarantees the best achievable quality for most tasks" (Section 4.2.1). This means the diffusion baselines are evaluated at their quality ceiling, not at a throughput-optimized setting — a conservative comparison that favors the baselines on quality but gives them no throughput advantage.
  • For Block Diffusion: Two decoding settings are used in Figure 4 — threshold = max (most conservative, highest quality) and threshold = 0.8 (more aggressive parallel decoding, higher throughput but lower quality), shown as two points per task connected by a Pareto frontier.

• Cross-validation / statistical protocol. The paper does not employ cross-validation or statistical significance testing. The main results (Tables 2-3, Figures 4-5) report single-run evaluation scores on standard benchmarks with fixed few-shot configurations. For the Pareto frontier analysis (Figure 5), multiple points per method represent different decoding configurations (TiDAR with block sizes 4, 8, 16; Block Diffusion with different thresholds; AR with and without fine-tuning), but each point represents a single evaluation run. For the ablation on trusting AR vs. diffusion (Figure 6), three different training configurations (varying loss weight ratio) are presented, and performance is shown as average score across tasks — the paper interprets the flatness of the curves as evidence of robustness but does not provide confidence intervals.

Main Quantitative Results

Generative Task Quality and Efficiency

The headline result (Table 2): TiDAR 1.5B achieves 44.03% average score across coding and math tasks while producing 7.45 tokens per forward pass, representing competitive quality with the AR baseline Qwen2.5-1.5B (41.64%) while generating ~7.5× more tokens per step. At the 8B scale, TiDAR achieves 65.31% average (trust-diffusion mode) with 8.25 T/NFE, closing most of the gap with Qwen3-8B's 68.09% while generating over 8× more tokens per forward pass.

Breaking this down by task category at 1.5B (Table 2):

• HumanEval: TiDAR reaches 43.29% vs. Qwen2.5-1.5B's 35.98% and Block Diffusion's 39.02%. TiDAR exceeds the AR baseline by 7.3 percentage points while producing 6.50 T/NFE. Qwen3-1.7B, a newer and slightly larger AR model, reaches 48.17%.
• HumanEval+: TiDAR (39.02%) similarly exceeds Qwen2.5-1.5B (29.88%) and Block Diffusion (34.76%), though Qwen3-1.7B (41.46%) remains ahead.
• MBPP: TiDAR (41.40%) is competitive with Qwen2.5-1.5B (43.60%) at 9.43 T/NFE, but trails Block Diffusion's 34.00% and significantly trails Qwen3-1.7B's 55.80%.
• GSM8K strict-match: TiDAR (53.90%) nearly matches Qwen2.5-1.5B (54.74%) and Block Diffusion (52.99%) at 5.07 T/NFE. Qwen3-1.7B achieves 66.72%.
• Minerva Math: TiDAR (25.48%) is slightly below Qwen2.5-1.5B (26.40%) at 7.92 T/NFE, but above Block Diffusion (21.56%).

The pattern shows TiDAR achieving lossless or nearly-lossless quality relative to its initialization base model (Qwen2.5-1.5B) while providing 5-9× more tokens per forward pass. The quality gap to the stronger Qwen3-1.7B (which benefits from architectural and training improvements in the Qwen3 family) is real but should be attributed to base model quality differences, not TiDAR's architectural overhead — TiDAR inherits the capability ceiling of its initialization checkpoint.

At the 8B scale (Table 2):

• TiDAR (trust-diffusion) achieves 65.31% average vs. Qwen3-8B's 68.09% — a 2.78 percentage point gap across 6 tasks.
• On HumanEval: 57.93% vs. 64.63% (Δ = -6.7 points).
• On HumanEval+: 55.49% vs. 56.71% (Δ = -1.2 points) — the smallest gap.
• On MBPP: 65.40% vs. 69.40% (Δ = -4.0 points).
• On GSM8K: 80.44% vs. 81.80% (Δ = -1.4 points) — near-lossless.
• On Minerva Math: 51.64% vs. 52.94% (Δ = -1.3 points) — near-lossless.

The trust-AR mode (63.90%) slightly underperforms trust-diffusion (65.31%), with the gap largest on math tasks (GSM8K: 79.83% vs. 80.44%; Minerva Math: 50.58% vs. 51.64%). The paper attributes this to the diffusion mode benefiting from bidirectional context for reasoning.

Compared against prior diffusion LLMs (Dream-7B, LLaDA-8B), TiDAR-8B substantially outperforms both: Dream-7B achieves 58.74% average, LLaDA-8B achieves 41.78% — gaps of 6.6 and 23.5 percentage points respectively. This is despite Dream and LLaDA being evaluated at one token per forward pass (their best-quality setting), while TiDAR achieves higher quality with 7-10× more tokens per step.

Compared against Block Diffusion under the same training recipe: at 1.5B, Block Diffusion achieves 38.41% vs. TiDAR's 44.03% — a 5.6 percentage point gap. At the 4B/8B scale, Block Diffusion achieves 60.27% vs. TiDAR's 65.31% — a 5.0 point gap. These gaps directly measure TiDAR's architectural advantage, holding training data, initialization, and compute budget constant.

Likelihood Task Evaluation

The paper evaluates models on factual knowledge (MMLU) and commonsense reasoning (ARC, HellaSwag, PIQA, Winogrande) using likelihood-based scoring rather than generation (Table 3). For TiDAR, this is straightforward — the model's AR mode (causal attention over the clean prefix only) computes exact sequence likelihoods in a single forward pass, identical to standard AR evaluation. For diffusion models (LLaDA, Dream, Block Diffusion), likelihood evaluation requires Monte Carlo sampling over multiple denoising steps (128 steps used for the results marked in italics in Table 3).

At the 1.5B scale: TiDAR achieves 64.43% average, slightly below Qwen2.5-1.5B (65.16%) and comparable to Qwen3-1.7B (64.58%). Block Diffusion achieves 61.05%, a gap of 3.4 points. The largest gaps favor TiDAR on HellaSwag (65.26% vs. Block Diffusion's 56.26%, Δ = +9.0 points) and PIQA (75.52% vs. 70.13%, Δ = +5.4 points).

At the 8B scale: TiDAR achieves 75.40% average, exceeding Qwen3-8B's 74.25% by 1.15 percentage points. This is notable — TiDAR's likelihood scores match or exceed the AR baseline even as its generative scores show a small gap (Tables 2 vs. 3). On individual tasks, TiDAR exceeds Qwen3-8B on ARC-Easy (84.18% vs. 81.90%), ARC-Challenge (58.53% vs. 53.16%), PIQA (80.25% vs. 79.22%), and Winogrande (76.48% vs. 75.69%), while slightly trailing on MMLU (76.57% vs. 76.93%) and HellaSwag (76.36% vs. 78.59%). Against Dream-7B (71.86%) and LLaDA-8B (68.06%), TiDAR's advantage is substantial (3.5 and 7.3 points respectively).

The paper emphasizes three properties of TiDAR's likelihood evaluation that differentiate it from diffusion models: (1) it is directly comparable to AR models because the computation is identical (causal mask, single forward pass), (2) it is faithful to generative quality because the AR mode is what the model uses for sampling, and (3) it is extremely efficient — a single NFE vs. 128-step Monte Carlo for diffusion baselines.

Efficiency Benchmarking: Throughput vs. Quality Tradeoffs

Figure 4 presents the central efficiency-quality scatter plots for coding and math tasks at both model scales. Each point represents a model evaluated on a specific task (y-axis: task score; x-axis: relative AR throughput speedup in tokens per second). The paper reports T/NFE values on top of each point.

At 1.5B scale (Figure 4, top row):

• TiDAR with block size 4, 8, and 16 (three left-to-right points per task) achieves relative throughput speedups ranging from approximately 2.8× to 6.5× over Qwen2.5-1.5B.
• TiDAR's three configurations form a relatively flat quality curve as throughput increases — HumanEval score ranges from ~43% (block size 4, ~3.5× speedup) to ~41% (block size 16, ~6.5× speedup), a loss of only ~2 percentage points for nearly doubling throughput.
• Block Diffusion with threshold = max (most conservative) achieves throughput close to 1× (essentially no speedup, since it decodes one token per forward pass for best quality) and quality scores below TiDAR across all tasks.
• Block Diffusion with threshold = 0.8 achieves higher throughput (~2.3-3.5×) but sees substantial quality degradation: on HumanEval, quality drops from ~39% (threshold = max) to ~26% (threshold = 0.8) — a 13-point drop for only modest throughput gain. On GSM8K, the drop is from ~53% to ~47%.
• TiDAR achieves an average relative throughput of 4.71× when matching AR quality (the paper states this as the headline number). The highest throughput points (block size 16) reach 5.51-6.50× relative speedup.

At 8B scale (Figure 4, bottom row):

• TiDAR-8B (single configuration shown: block size 16, trust-diffusion and trust-AR modes) achieves relative throughput speedups of 7.07-10.13× over Qwen3-8B.
• On HumanEval: TiDAR achieves ~58% score at ~7.3× throughput (trust-diffusion) and ~55% score at ~7.5× (trust-AR), compared to Qwen3-8B at 1× throughput and ~65% score.
• On GSM8K: TiDAR achieves ~80% score at ~7.1× throughput (trust-diffusion, the rightmost point) vs. Qwen3-8B at ~82% score. The throughput-quality tradeoff is extremely favorable — losing ~2 points of accuracy for 7× throughput.
• EAGLE-3 (with Qwen3-8B-Instruct) achieves throughput speedups of 3.32-4.52× depending on the open-weight variant. This is substantially below TiDAR's 7-10×, and EAGLE-3 quality is also slightly lower (Qwen3-8B-Instruct scores are used as the reference point).
• TiDAR achieves an average relative throughput of 5.91× (the headline number). The higher per-task numbers (7-10×) represent task-specific throughput while the average accounts for variation across tasks.

A critical detail in interpreting these results: the relative throughput speedup is computed against the base AR model within the same size group (Qwen2.5-1.5B or Qwen3-8B). This means the speedup measures how much faster TiDAR generates tokens compared to the standard AR model at the same parameter count and on the same hardware. The EAGLE-3 comparison uses Qwen3-8B-Instruct as the base model because corresponding EAGLE-3 weights for the base model were not available — the paper notes this limitation.

The paper explicitly highlights two observations about the TiDAR vs. EAGLE-3 comparison:

• "our raw acceptance rate (T/NFE) is higher than those of EAGLE-3 open weights" — directly visible from the T/NFE numbers annotated on Figure 4 points (TiDAR: 7.30-10.13; EAGLE-3: 3.32-4.52).
• "our conversion rate (from T/NFE to T/s) is higher, thanks to the parallel drafting and sampling with a single model forward" — meaning TiDAR's tokens-per-second speedup is closer to its tokens-per-NFE ratio than EAGLE-3's is, because EAGLE-3 pays the latency cost of sequential drafting and verification phases while TiDAR amortizes them in one pass.

The paper is careful to note that all throughput measurements are in native PyTorch with Flex Attention and Flash Attention 2, without custom kernels or scheduling optimizations. They state that "all of these methods can significantly benefit from further system optimizations such as custom kernels, more efficient KV cache management, and request scheduling," so the absolute throughput numbers should be interpreted as a first-order comparison rather than production-ready performance.

Pareto Frontier Under Controlled Training

Figure 5 provides the cleanest architectural comparison by training all models (AR, fine-tuned AR, Block Diffusion, TiDAR) under the same recipe starting from Qwen2.5-1.5B with 50B tokens. The x-axis is T/NFE (algorithmic efficiency), the y-axis is task score. Each method produces multiple points through different decoding configurations.

Key observations from Figure 5:

• AR (single point): Sits at T/NFE = 1 with scores around 35-55% depending on task. This is the quality ceiling for standard decoding.
• Fine-tuned AR (single point): Also at T/NFE = 1, with scores 5-10 points higher than the base AR across tasks — representing the best achievable quality with additional training, but no throughput gain.
• Block Diffusion (multiple points across varying thresholds): Forms a curve that moves down and to the right as the threshold is lowered. At threshold = max (leftmost point), T/NFE ≈ 1-2 and quality is below AR. As threshold decreases to 0.6-0.8, T/NFE increases to 3-7 but quality drops substantially — on HumanEval, from ~39% at T/NFE ≈ 1 to ~26% at T/NFE ≈ 4.5; on MBPP, from ~34% at T/NFE ≈ 2 to ~26% at T/NFE ≈ 5.5.
• TiDAR (multiple points across block sizes 4, 8, 16 and training variants): Forms a curve that is strictly above and to the right of Block Diffusion on every task — meaning TiDAR achieves both higher quality and higher throughput at every operating point. On HumanEval, TiDAR maintains ~40-43% scores at T/NFE = 3-7, compared to Block Diffusion's 26-39% over the same T/NFE range. On MBPP+, TiDAR achieves ~51-61% at T/NFE = 4-10, compared to Block Diffusion's ~35-49%.
• The full-mask TiDAR variants (block sizes 4, 8, 16) consistently outperform the random-mask TiDAR variant — visible on HumanEval where the random-mask point falls below the full-mask curve.
• The AR fine-tuned model remains above TiDAR on most tasks in absolute quality, but at T/NFE = 1 — TiDAR achieves comparable quality at 3-7× higher T/NFE. The paper frames this as TiDAR "approaching the quality of fine-tuned AR with 7× more tokens per NFE."

This Pareto analysis directly supports the claim that TiDAR's hybrid architecture fundamentally shifts the quality-efficiency tradeoff compared to pure diffusion (Block Diffusion). The gap between the two curves is TiDAR's architectural contribution — both use the same training data, same initialization, and same compute budget, but TiDAR achieves higher quality at every throughput level because rejection sampling corrects the errors that Block Diffusion's independence assumption introduces.

Comparison of Decoding Strategies

Table 4 provides a systematic comparison of TiDAR against standard diffusion decoding strategies using the full-mask trained model. The key finding: TiDAR's parallel draft-and-sample procedure achieves substantially higher quality at comparable or higher throughput than any confidence-based or left-to-right decoding strategy.

For context, the baseline strategies are:

• Confidence Max / Left-to-right AR: Decode one token per forward pass (T/NFE = 1.00). These represent the quality ceiling for diffusion-based decoding but provide no throughput advantage.
• Confidence > threshold (0.6-0.9): Generate multiple tokens per step by accepting any token whose predicted probability exceeds the threshold, then iteratively denoise the remaining positions. Higher thresholds (0.9) produce conservative decoding (T/NFE ≈ 2.63), while lower thresholds (0.6) produce aggressive decoding (T/NFE ≈ 3.81) but degraded quality.

The results (Table 4):

• Confidence Max (T/NFE = 1.00): HumanEval 34.45%, MBPP 43.92%, GSM8K 53.07%. This is the best achievable quality with pure diffusion decoding one token at a time.
• Left-to-right AR (T/NFE = 1.00): HumanEval 36.28%, MBPP 46.51%, GSM8K 53.37%. Slightly better than confidence max because it follows the natural left-to-right generation order.
• Confidence > 0.9 (T/NFE = 2.63): Quality drops to HumanEval 32.01%, MBPP 42.50%, GSM8K 51.40%. A 2-4 point quality loss for ~2.6× throughput — a poor tradeoff.
• Confidence > 0.8 (T/NFE = 3.06): Quality drops further: HumanEval 28.96%, MBPP 39.28%, GSM8K 47.54%.
• Confidence > 0.7 (T/NFE = 3.42): HumanEval 27.74%, MBPP 33.90%, GSM8K 43.44%.
• Confidence > 0.6 (T/NFE = 3.81): HumanEval 22.56%, MBPP 26.47%, GSM8K 37.60%.
• TiDAR 4 drafts (T/NFE = 3.47): HumanEval 38.42% (+15.9 points above confidence > 0.7 at comparable T/NFE), MBPP 50.96% (+17.1 points), GSM8K 55.87% (+12.4 points).
• TiDAR 8 drafts (T/NFE = 5.49): HumanEval 39.94%, MBPP 52.13%, GSM8K 54.74% — higher throughput than any confidence-based method with higher quality than even the T/NFE = 1.00 baselines.
• TiDAR 16 drafts (T/NFE = 6.97): HumanEval 41.16% — nearly 2× the quality of confidence > 0.6 at 1.8× the throughput.

The takeaway is stark: confidence-based decoding strategies (the standard approach in diffusion LMs) see quality degrade sharply as throughput increases, while TiDAR achieves higher quality at higher throughput because the quality is guaranteed by AR rejection sampling, not by the diffusion predictions themselves. The left-to-right AR scheme achieves reasonable quality at T/NFE = 1.00 but cannot provide throughput gains; extending it to T/NFE = 2.00 causes quality to collapse (HumanEval drops from 36.28% to 21.95%, MBPP from 46.51% to 18.61%).

Ablation Studies and Robustness Checks

Full mask vs. random masking in training (Table 5): The paper ablates the masking strategy used during training. Under the "Random" strategy (traditional diffusion training with stochastic masking), TiDAR with 4 drafts achieves HumanEval 32.62%, MBPP 48.63%, GSM8K 55.11%, and T/NFE = 3.42. With "Full" masking, TiDAR achieves HumanEval 38.42% (+5.8 points), MBPP 50.96% (+2.3 points), GSM8K 55.87% (+0.8 points), and T/NFE = 3.47 (+0.05). At 8 drafts, random masking achieves HumanEval 33.85%, MBPP 48.77%, GSM8K 54.43%, T/NFE = 5.22; full masking achieves HumanEval 39.94% (+6.1 points), MBPP 52.13% (+3.4 points), GSM8K 54.74% (+0.3 points), T/NFE = 5.49 (+0.27). The largest gains are on coding tasks; math tasks see smaller improvements. The efficiency gain (higher T/NFE) is attributed to higher acceptance rates from better draft quality under the full-mask training regime, which eliminates train-test distribution mismatch.

Trusting AR vs. diffusion predictions (Figure 6): The paper varies both the logit mixing parameter β (x-axis: 0.0 = fully trust diffusion, 1.0 = fully trust AR) and the training loss weight ratio across three configurations (AR:Diff ratios of 0.8:1.0, 1.0:1.0, 1.2:1.0). All three training configurations produce nearly flat curves across the full range of β, with average scores varying by less than ~2 percentage points from β = 0 to β = 1. The 1.0:1.0 loss ratio produces the highest scores at most β values (approximately 48-49% average), while the 0.8:1.0 and 1.2:1.0 ratios are slightly lower (46-48%). The flatness of all curves demonstrates that the model's AR and diffusion predictions are well-calibrated to each other — they produce similar logit distributions at shared prediction positions — and that the rejection sampling mechanism, not the choice of which logits to trust, drives the quality guarantee. The paper interprets this as: "it is the autoregressive rejection sampling that guarantees the quality-speedup trade-offs rather than the AR knowledge."

Block size (draft length) variation: Across Tables 2, 4, and Figure 4, the paper evaluates TiDAR at block sizes 4, 8, and 16. At 1.5B (Table 4): block size 4 → T/NFE = 3.47, average score 48.42%; block size 8 → T/NFE = 5.49, average 48.94%; block size 16 → T/NFE = 6.97, average 48.44% (computed from the three-task averages). The quality is essentially flat as block size increases (less than 0.5 points variation), while throughput increases nearly proportionally — demonstrating that the AR rejection sampling effectively filters diffusion drafts even at high parallelism levels. At 8B (Table 2, trust-diff mode): block size 16 achieves T/NFE = 8.25 with 65.31% average across 6 tasks. The paper does not report 8B results at smaller block sizes, limiting direct comparison of block size scaling at the larger model scale.

Loss balancing factor α ablation (Figure 6, implicitly): The three configurations shown in Figure 6 — (AR × 0.8 + Diff × 1.0) / 1.8, (AR × 1.0 + Diff × 1.0) / 2.0, (AR × 1.2 + Diff × 1.0) / 2.2 — represent effective AR-to-diffusion weight ratios of 0.8:1.0, 1.0:1.0, and 1.2:1.0. The performance is relatively consistent across these ratios, with the 1.0:1.0 configuration showing a slight advantage. The paper does not test more extreme ratios (e.g., 0.2:1.0 or 5.0:1.0), so the claim of robustness is established only within a narrow range around equal weighting.

Training data scale: The paper does not conduct systematic data ablation — all 1.5B models are trained on 50B tokens and all 8B models on 150B tokens, with no intermediate checkpoints evaluated. The paper notes qualitatively that "TiDAR might require a bit more knowledge due to the initial adaptation phase" when discussing the remaining quality gap to fine-tuned AR, implying that additional training data could close the gap, but no evidence is provided.

Critical Assessment

Claim 1: TiDAR achieves lossless quality compared to its AR counterpart at 1.5B while delivering 4.71× relative throughput speedup, and at 8B delivers 5.91× throughput speedup with minimal quality loss.

This claim is the paper's central and most important result. The evidence supporting it, when examined closely, is nuanced:

At 1.5B, the claim of "lossless quality" rests on the comparison between TiDAR-1.5B (initialized from Qwen2.5-1.5B) and Qwen2.5-1.5B itself. Table 2 shows TiDAR achieving 44.03% average vs. Qwen2.5-1.5B's 41.64% — TiDAR actually exceeds the base AR model on average. On individual tasks, TiDAR substantially exceeds the AR baseline on coding (HumanEval: 43.29% vs. 35.98%, HumanEval+: 39.02% vs. 29.88%) while slightly trailing on MBPP (41.40% vs. 43.60%) and GSM8K (53.90% vs. 54.74%). The throughput speedup of 4.71× is supported by Figure 4, which shows TiDAR-1.5B achieving 4.61-6.50× relative throughput across tasks. However, three caveats weaken the "lossless" claim:

1. The comparison is against the initialization model, not the best AR model at that scale. Qwen3-1.7B achieves 52.22% average — 8.2 points above TiDAR. While TiDAR inherited Qwen2.5-1.5B's weights and thus its capability ceiling, a user choosing between models cares about absolute quality, not improvement over initialization. The claim "lossless quality compared to its AR counterpart" is true only if "its AR counterpart" means the specific checkpoint used for initialization, not the best available AR model at that parameter count.

2. Likelihood tasks show a small but consistent gap. In Table 3, TiDAR-1.5B achieves 64.43% vs. Qwen2.5-1.5B's 65.16% (-0.73 points) and Qwen3-1.7B's 64.58% (-0.15 points). While these differences are small for a single benchmark, across 6 tasks the pattern suggests a slight likelihood degradation that may indicate the AR mode's predictions are subtly affected by the dual-mode training, even though the causal attention path is architecturally identical to standard AR models.

3. The 4.71× speedup is an average over tasks with different generation lengths. The per-task T/NFE values in Table 2 range from 5.07 (GSM8K) to 9.43 (MBPP+), and the throughput speedup in Figure 4 similarly varies. For short-generation tasks where the prompt processing (prefill) dominates latency, the relative speedup from faster decoding matters less in absolute terms. The paper does not report end-to-end latency (prefill + decode) for complete benchmark runs — only decoding throughput — so the wall-clock time to complete a full evaluation is not characterized.

At 8B, the claim of "minimal quality loss" is better supported quantitatively but faces different issues. TiDAR-8B (trust-diffusion) achieves 65.31% vs. Qwen3-8B's 68.09% — a 2.78-point gap, which is indeed "minimal" in practical terms. On math tasks specifically, the gap is under 2 points (GSM8K: 80.44% vs. 81.80%; Minerva Math: 51.64% vs. 52.94%). The 5.91× throughput speedup is supported by Figure 4. The concerns are:

1. The base model comparison should include the same amount of continual training. TiDAR-8B benefited from 150B tokens of additional training beyond the Qwen3-8B checkpoint. It is unclear how much of TiDAR's strong performance comes from the architecture vs. simply having seen 150B more tokens. A proper ablation would compare TiDAR-8B against Qwen3-8B further trained for 150B tokens with the standard AR objective (a "fine-tuned AR" baseline like the one shown for 1.5B in Figure 5). The paper includes fine-tuned AR for 1.5B in Figure 5 but does not report an 8B fine-tuned AR baseline in Table 2 — this is a significant missing comparison.

2. The 8B results omit block size variation. Only block size 16 is reported for the 8B model. It is unknown whether smaller block sizes would achieve higher quality (by reducing the number of simultaneously drafted tokens and thus increasing per-token acceptance rate) or whether the quality loss at block size 16 is already at the limit of what rejection sampling can correct. The 1.5B results in Table 4 show nearly flat quality from block size 4 to 16, suggesting robustness, but this pattern may not hold at larger scales where the model's diffusion predictions could be differently calibrated.

3. The EAGLE-3 comparison uses an instruct model, while TiDAR uses a base model. EAGLE-3 with Qwen3-8B-Instruct is compared against TiDAR-8B (initialized from Qwen3-8B base). The instruct model may have different (possibly worse) raw benchmark performance than the base model because instruction tuning can degrade few-shot prompting ability. The paper acknowledges this mismatch: "We use the EAGLE-3 with Qwen3-8B instruct model due to lack of corresponding EAGLE-3 weights for the base model." The 5.91× vs. 3.5-4.5× comparison should be interpreted with this caveat.

Claim 2: TiDAR is the first architecture to close the quality gap with AR models while delivering substantial throughput.

This claim is best evaluated against the diffusion model baselines (Dream, LLaDA, Block Diffusion). Table 2 shows TiDAR-8B achieving 65.31% vs. Dream-7B's 58.74% and LLaDA-8B's 41.78% — substantial gaps that clearly establish TiDAR's superiority over prior diffusion architectures. Against Block Diffusion under the same training recipe, TiDAR leads by 5-6 points at both 1.5B and 8B scales. These comparisons are fair because all diffusion models are evaluated at their best-quality setting (one token per forward pass for Dream/LLaDA, threshold = max for Block Diffusion).

The claim of being "the first" is harder to verify from the evidence presented. The paper cites APD [17], a concurrent work that also attempts to improve diffusion LM throughput, but does not provide direct APD comparisons. Whether another unpublished architecture achieves similar results cannot be determined from the paper alone. The claim should be understood as applying to the specific comparison set evaluated (Dream, LLaDA, Block Diffusion, EAGLE-3).

Claim 3: The quality guarantee comes from autoregressive rejection sampling, not from the diffusion predictions themselves (Figure 6, Table 5, Table 4).

This claim is well-supported by the evidence. Figure 6 shows that varying β from 0 (fully trust diffusion) to 1 (fully trust AR) produces nearly flat performance — if the quality depended on the diffusion predictions being accurate, trusting them completely (β = 0) should produce worse results than trusting AR (β = 1). The flatness indicates that rejection sampling corrects the diffusion errors regardless of which logits are used for the sampling step. Table 4 provides converging evidence: confidence-based decoding (which directly uses the diffusion predictions as output without AR verification) sees quality collapse as throughput increases, while TiDAR (which subjects the same diffusion predictions to AR rejection sampling) maintains or improves quality at higher throughput.

The counterargument would be that rejection sampling only guarantees distributional equivalence to AR, not instance-level quality — a sequence that has high probability under p_AR can still be wrong if the AR model itself is wrong. But this is a property of the base AR model, not of TiDAR's architecture, and the paper explicitly acknowledges this by comparing against the AR baseline's quality.

Weakening considerations for all claims:

• Single hardware configuration: All throughput measurements are on a single NVIDIA H100 at batch size 1. The "free token slots" concept is hardware-dependent — different GPUs (A100, L40S, H200) have different memory bandwidth and compute characteristics, which would shift the boundary between memory-bound and compute-bound regimes and thus change the optimal block size and achievable speedup. The paper does not profile TiDAR on any hardware other than the H100.

• Single evaluation framework and fixed few-shot prompts: All evaluations use lm_eval_harness with the specific configurations in Table 6. While this is standard practice, different few-shot prompt formats or evaluation protocols can shift absolute scores, potentially affecting the quality-gap narrative if the AR baselines are more or less sensitive to prompt format than TiDAR.

• No long-context evaluation: The paper acknowledges that long-context generation is untested (Section 5, Limitations): "Since our current implementation requires doubling the sequence length with appended mask tokens during training, we defer the exploration of efficient long context extension methods for future work." On tasks requiring long outputs (multi-step reasoning, long-form generation), the doubling of the memory footprint from mask tokens could become a bottleneck, and the acceptance rate of draft tokens may differ from the short-generation tasks evaluated.

• Batch size 1 only: Throughput benchmarking is exclusively at batch size 1, which is appropriate for interactive latency-sensitive applications but does not characterize TiDAR's behavior in batch inference pipelines. The paper claims that TiDAR "can achieve competitive performance in terms of FLOPs / token" at larger batch sizes, but provides no evidence. At large batch sizes, the memory-bound assumption weakens (the forward pass may become compute-bound), and the "free token slots" justification for TiDAR's speedup would no longer apply in the same way. The paper does not report speedup at batch sizes other than 1.

• No statistical significance or variance reporting: All results are single evaluation runs. For benchmarks like HumanEval (164 problems) and GSM8K (~1300 test examples), the variance of Pass@1 estimates can be non-trivial. A ±2-3 percentage point confidence interval would overlap several of the claimed quality advantages, particularly the TiDAR-vs-AR comparisons where gaps are small (e.g., GSM8K: 80.44% vs. 81.80%).

• Missing ablation: the contribution of additional training tokens. The paper does not control for the effect of the 50B and 150B additional training tokens when comparing TiDAR against the base AR model. In Figure 5, the "AR FT" (fine-tuned AR) point provides some control — it represents the AR model trained for the same 50B tokens but with the standard AR objective only. TiDAR substantially exceeds this baseline in T/NFE while approaching its quality, which is strong evidence for the architecture's contribution. However, this ablation is only shown for 1.5B in the Pareto plot, not for 8B in Table 2, where it would be most informative for the headline 8B claims.

• The definition of "lossless" could be contested. At 8B, TiDAR trails Qwen3-8B by 2.78 percentage points on average — a gap that is small but consistently negative across most tasks. Whether this qualifies as "minimal loss" or represents a real quality ceiling imposed by the dual-mode training is a matter of interpretation. The paper does not provide evidence that additional training data would close this gap (it is speculated but not demonstrated), and the 150B tokens already represent substantial continued pretraining. If the gap is fundamental — arising from the capacity split between AR and diffusion objectives — then the claim of "closing the quality gap" overstates the result for the 8B scale.

Experiments that would have strengthened the paper:

1. TiDAR vs. fine-tuned AR at 8B scale. This would isolate the architectural contribution from the effect of additional training tokens at the larger model size that generates the headline 5.91× speedup.

2. Middle-checkpoint evaluations during training. By evaluating TiDAR at 10B, 25B, 50B, 100B, and 150B tokens of continual pretraining, the paper could show whether the quality gap to AR is closing or has plateaued, directly testing the speculation that "more data could potentially close" the gap.

3. Hardware diversity. Profiling on A100, L40S, or H200 would characterize the "free token slots" concept's generality and provide practical guidance for practitioners on different hardware.

4. Batch size scaling. Measuring throughput at batch sizes 2, 4, 8, 16 would reveal whether TiDAR's speedup persists or diminishes in higher-throughput serving scenarios.

5. Ablation on the contribution of pre-drafting across multiple hypotheses. TiDAR pre-drafts tokens conditioned on all possible acceptance outcomes (0 to K accepted). An ablation that only conditions on a single hypothesis (e.g., assuming all drafts are accepted) would isolate the value of this multi-hypothesis pre-drafting vs. the simpler approach of just re-drafting after rejection sampling. Such an ablation is not present.

6. Direct comparison against more speculative decoding variants. While EAGLE-3 is a strong baseline, comparing against Medusa [28] or against standard speculative decoding with a smaller draft model (e.g., a 0.5B model drafting for the 8B) would provide a more complete picture of where TiDAR stands in the speculative decoding landscape — particularly for scenarios where exact output distribution preservation is required (standard speculative decoding guarantees this; TiDAR does as well through rejection sampling, but the paper does not emphasize this property).

6. Limitations and Trade-offs

6.1 The "Free Token Slots" Justification Is Demonstrated on Only One GPU at Batch Size 1

The assumption or constraint. The entire throughput advantage of TiDAR rests on the hardware property demonstrated in Figure 1: that in the memory-bound regime (small batch sizes, modest sequence lengths), adding extra token positions to a forward pass incurs negligible additional latency. The paper profiles this property on a single NVIDIA H100 GPU with Qwen3-32B at batch size 1, observing that latency stays nearly flat up to a certain number of token slots before transitioning to the compute-bound regime. From this observation, the paper generalizes that "if for a given $k$ such that both computations are still memory-bound, the forward time... should be similar" and that "we leverage this characteristic to achieve almost free parallelled drafting and sampling for TiDAR" (Section 1). All throughput benchmarking (Figure 4) is conducted on this same hardware configuration.

The consequence. The "free token slots" concept is not a universal property — it depends intimately on the GPU's memory bandwidth, compute throughput (FLOPS), and the specific model architecture (which determines the ratio of weight-loading to compute per token). Different hardware — an A100 (lower memory bandwidth), an H200 (higher bandwidth, more memory), or an L40S (different compute/bandwidth ratio) — will have different boundaries between the memory-bound and compute-bound regimes, and thus different optimal block sizes and achievable speedups. On hardware where the memory-bound plateau is narrower (more of the latency curve is in the cheap-token or compute-bound regions), the additional token slots for drafting would increase per-step latency, reducing or eliminating the throughput advantage. The paper acknowledges this dependency implicitly when it notes that "all of these methods can significantly benefit from further system optimizations" (Section 4.3), but it provides no evidence that the 4.71× and 5.91× speedups generalize beyond the specific GPU tested.

The batch size 1 limitation is equally significant. At batch size 1, the memory-bound assumption holds strongly because there is minimal compute to amortize weight loading across. At larger batch sizes (common in batch inference pipelines, offline evaluation, or high-throughput serving), the forward pass may become compute-bound, where adding token positions does increase latency proportionally. The paper claims that TiDAR "can achieve competitive performance in terms of FLOPs / token" at larger batch sizes (Section 5), but this claim is unsubstantiated. In the compute-bound regime, the speedup would be bounded by the tokens-per-NFE acceptance rate rather than being amplified by the "free" latency of additional token slots — the effective throughput gain could drop from ~5-6× to ~2-3× or less, depending on how the compute cost of the additional mask token positions scales.

What evidence exists in the paper. Figure 1 profiles only the H100 at batch size 1, with one model (Qwen3-32B). The boundary between free, cheap, and expensive token slots is shown for this single configuration. All throughput measurements in Figure 4 and the headline speedup numbers (4.71×, 5.91×) are benchmarked on a single H100 at batch size 1 (Section 4.3). No profiling data or throughput measurements are provided for any other GPU, any other batch size, or any other hardware generation.

Mitigation status. The paper acknowledges this as a limitation only implicitly. Section 5 (Limitations) mentions batch size and states that "Not only can we adjust the block (draft) length during decoding in a zero-shot manner to accommodate different compute profile, but also can achieve competitive performance in terms of FLOPs / token." This is aspirational — the "zero-shot" block length adjustment is mentioned as a capability but not demonstrated in any experiment, and the "competitive performance in FLOPs / token" is not measured. The paper does not profile TiDAR at any batch size other than 1, nor on any hardware other than the H100. For a practitioner evaluating whether to deploy TiDAR on A100-based infrastructure (still widely deployed) or in a batch inference setting, the paper provides no guidance.

---

6.2 The 8B Baseline Comparison Confounds Architectural Gains with Additional Training Data

The assumption or constraint. TiDAR models are trained via continual pretraining from existing AR checkpoints: 50B tokens for the 1.5B model (initialized from Qwen2.5-1.5B) and 150B tokens for the 8B model (initialized from Qwen3-8B). The headline claim — "TiDAR 8B... achieved an impressive 5.91× relative throughput speedup with minimal loss" (Section 1) — compares TiDAR-8B after 150B tokens of additional training against the base Qwen3-8B checkpoint without any additional training. This confounds two effects: (a) the architectural advantage of TiDAR's dual-mode drafting-and-verification, and (b) the quality improvement from seeing 150B additional training tokens with any objective.

The consequence. If the base Qwen3-8B model were further trained for 150B tokens with the standard AR next-token prediction objective (producing a "fine-tuned AR 8B" baseline), its benchmark scores would likely improve — the 1.5B-scale results in Figure 5 show that fine-tuned AR gains 5-10 points across tasks from 50B tokens of additional AR training. At the 8B scale, with 150B tokens (3× the 1.5B training budget), the quality improvement from further AR training could be substantial. If a fine-tuned AR 8B achieves, say, 72% average instead of Qwen3-8B's 68%, then TiDAR's 65.31% represents a ~7-point gap rather than a ~3-point gap — a "minimal loss" that is actually a meaningful degradation attributable specifically to the dual-mode training objective (which splits model capacity between AR and diffusion tasks).

The consequence for deployment decisions is direct: a practitioner choosing between TiDAR-8B and a hypothetical fine-tuned AR 8B would need to weigh a ~7× throughput gain against a ~7-point quality loss, which is a different calculus than the ~3-point loss presented in Table 2 against the untuned base model. The paper's framing of "minimal quality loss" may overstate TiDAR's competitiveness if the proper baseline is an AR model given the same additional training budget.

What evidence exists in the paper. The Pareto frontier analysis in Figure 5 provides the key evidence — but only at the 1.5B scale. At 1.5B, the "AR FT" point (fine-tuned AR with 50B additional tokens) achieves substantially higher quality than both the base AR and TiDAR-1.5B on most tasks (e.g., HumanEval: ~45% vs. ~43% for TiDAR; MBPP: ~55% vs. ~51% for TiDAR). This demonstrates that additional AR training alone produces quality gains, and that TiDAR-1.5B does not close the gap to a comparably-trained AR model — it approaches but does not match it, while providing 3-7× higher throughput. At the 8B scale (Table 2, Figure 4), this "AR FT" baseline is entirely absent. The paper compares TiDAR-8B only against the untuned Qwen3-8B, Qwen3-8B-Instruct (for EAGLE-3), Dream-7B, LLaDA-8B, and Block Diffusion-4B — none of which received 150B tokens of additional training under the same recipe. The missing fine-tuned AR 8B baseline is a significant gap in the experimental design for the 8B results.

Mitigation status. The paper does not acknowledge this as a limitation. The 1.5B-scale Figure 5 demonstrates awareness of the issue (the AR FT baseline is included there), but no explanation is given for its absence at 8B. The paper speculates in Section 4.4.1 that "the remaining quality gap from the fine-tuned AR models... could be potentially closed with more data due to the fact that TiDAR might require a bit more knowledge due to the initial adaptation phase." This speculation is made in the context of 1.5B results but is not tested at either scale — no intermediate training checkpoints are evaluated to show whether the quality gap is shrinking or stable. A practitioner cannot determine from the paper whether TiDAR-8B with 300B or 500B tokens would close the gap to fine-tuned AR or whether the gap is fundamental (arising from the capacity split between objectives).

---

6.3 Long-Context Generation Is Untested and the Sequence-Doubling Architecture Imposes a Memory Penalty

The assumption or constraint. TiDAR's training requires doubling the input sequence length: the original sequence of length $S$ is appended with $S$ mask tokens, resulting in an effective training sequence length of $2S$. The paper states this explicitly: "Similar to Block Diffusion [12] and Set Block Decoding (SBD) [31], we also need to double the sequence length, as a result of appending original input sequence with corrupted tokens" (Section 3.1). During inference, each forward pass processes $P + 2K$ tokens, where $P$ is the prefix length and $K$ is the block size — the $2K$ term represents $K$ draft tokens from the last step plus $K$ mask tokens for the next step.

The consequence. For long-context generation, the sequence-doubling architecture imposes two problems:

Training memory footprint: Training on sequences of length 2S means that for a given GPU memory budget, the maximum original sequence length S is halved compared to standard AR training. If the training infrastructure supports a maximum of 4096 tokens per sequence for AR models, TiDAR can only train on original sequences of 2048 tokens (since the doubled sequence reaches 4096 positions). This directly limits the model's exposure to long-range dependencies during training, which is critical for tasks like long-document summarization, multi-turn dialogue, or repository-level code generation.

Inference memory scaling: During decoding, the KV cache must store entries for the entire prefix $P$ plus accepted draft tokens. Unlike standard AR decoding where the KV cache grows linearly with the number of generated tokens, TiDAR temporarily stores KV entries for $K$ drafted tokens that may be rejected and evicted. While the paper notes that rejected tokens' KV entries are evicted (Section 3.2), the peak KV cache memory during each forward pass is higher than AR decoding by the $K$ draft token entries. For very long generations (thousands of tokens), this additional memory overhead could be the difference between fitting in GPU memory and requiring offloading or recomputation strategies. Additionally, the mask tokens in Section 3 consume memory for their hidden states during the forward pass (even though their KV entries are not cached), further increasing peak memory usage versus AR decoding of the same prefix length.

The paper's evaluation uses prompts from the benchmark tasks (coding, math, reasoning), which have relatively short expected outputs (typically under 512 tokens). None of the evaluated tasks stress long-context generation, so the paper provides no evidence about how TiDAR's throughput or quality behaves when generating thousands of tokens — where the acceptance rate may degrade (long-range dependencies are harder to draft accurately with one-step diffusion) and the memory overhead of the doubled architecture becomes more acute.

What evidence exists in the paper. The training configuration (Section 4.1) sets max sequence length to 4096 — but this is the doubled length, so the effective original sequence length is only 2048 tokens. All evaluated benchmarks (Appendix A, Table 6) use generation lengths of 256-512 tokens, well within this range. No long-context benchmark (e.g., LongBench, L-Eval, or needle-in-haystack retrieval) is evaluated. The paper does not measure peak GPU memory usage during TiDAR decoding relative to AR decoding at equivalent prefix lengths. No experiment varies the generation length to characterize how TiDAR's throughput and quality scale with output length.

Mitigation status. The paper explicitly acknowledges this limitation (Section 5, Limitations): "Since our current implementation requires doubling the sequence length with appended mask tokens during training, we defer the exploration of efficient long context extension methods (e.g. context parallelism specifically designed for TiDAR) for future work." This is a clear and honest acknowledgment, but it means that for a practitioner whose use case involves long-form generation, TiDAR's applicability is unvalidated. The proposed mitigation — context parallelism — is mentioned only as a future direction with no design details or preliminary results. The paper offers no short-term workaround (e.g., truncating context, using smaller block sizes for long sequences, or applying TiDAR only to the early tokens of a long generation).

---

6.4 The Difficulty Estimation Problem and the Absence of Adaptive Drafting

The assumption or constraint. TiDAR's inference procedure operates with a fixed block (draft) size that is set at training time. The model always drafts and verifies $K$ tokens per forward pass, regardless of the difficulty of the current generation context — whether the next tokens are highly predictable (where a larger block could be accepted) or highly uncertain (where many drafts will be rejected). The paper emphasizes that "Unlike traditional diffusion models, TiDAR has no hyperparameters to tune during inference" (Section 3.3) as a benefit, but this also means TiDAR cannot adapt its drafting behavior to the local difficulty of the generation.

The consequence. The throughput of TiDAR is directly determined by the acceptance rate — the expected number of draft tokens accepted per verification step. This acceptance rate depends on how well the diffusion predictions match the AR predictions, which in turn depends on how predictable the next tokens are given the context. In regions of high predictability (e.g., boilerplate code, common phrases, structured output formats), the acceptance rate could be very high, and a larger block size would yield proportionally more throughput. In regions of high uncertainty (e.g., the first token of a novel idea, branching points in reasoning, rare named entities), the acceptance rate could be very low — potentially near zero — meaning the model pays the cost of drafting and verifying $K$ tokens but accepts only one or two. In the limit, if acceptance rates are consistently low on certain types of content, TiDAR's throughput advantage could erode significantly compared to standard AR decoding, because the forward pass still processes $2K$ additional token slots (drafts + masks) even when most drafts are rejected.

This is the difficulty estimation problem familiar from the compute-optimal test-time scaling literature (Snell et al., 2024): the optimal allocation of compute between drafting and verification depends on how hard the current prediction is, and a fixed strategy is necessarily suboptimal across the full difficulty distribution. TiDAR's fixed block size is analogous to using a fixed beam width in search or a fixed number of parallel samples in best-of-N — it cannot allocate more drafting capacity to hard tokens and less to easy ones.

Additionally, because TiDAR uses one-step diffusion drafting, the draft quality is what it is — there is no mechanism to improve draft quality for hard tokens by investing additional denoising steps. If the one-step diffusion prediction for a particular position is poor, that draft token will be rejected, and the slot is wasted. A multi-step diffusion drafter could (in principle) spend more compute on difficult positions, but the paper explicitly argues that one-step is sufficient and does not explore adaptive multi-step drafting.

What evidence exists in the paper. The paper does not directly measure or report how the acceptance rate varies across different types of content or across positions within a single generation. The T/NFE values reported in Table 2 (e.g., 5.07 for GSM8K, 9.43 for MBPP+ at 1.5B) are averages over entire benchmark runs — they obscure any positional variation in acceptance rates. The per-task variation in T/NFE is noted but not analyzed: why does TiDAR achieve 9.43 T/NFE on MBPP+ but only 5.07 on GSM8K? Is it because GSM8K requires more precise reasoning tokens that are harder to draft, or because of differences in output length, or because of the strict-match evaluation requiring exact final answers? The paper offers no analysis. No experiment varies the block size during a single generation based on local confidence or entropy. The paper does not measure the distribution of acceptance lengths per forward pass (e.g., what fraction of steps accept 0, 1, 2, ..., K tokens).

Mitigation status. The paper acknowledges flexibility in the block size ("we can adjust the block (draft) length during decoding in a zero-shot manner to accommodate different compute profile," Section 5), but this refers to changing $K$ between deployments, not dynamically during a single generation. There is no mechanism proposed for adaptive drafting, no ablation testing dynamic block size schedules, and no analysis of how acceptance rates vary across difficulty levels. This is a genuine open problem — adaptive drafting would require some difficulty signal (e.g., the entropy of the AR distribution, or the PRM-like confidence of the diffusion predictions) and a policy for adjusting $K$ based on that signal. TiDAR provides no infrastructure for either.

---

6.5 The Method Has Not Been Demonstrated on Non-Benchmark, Open-Ended, or Multi-Turn Tasks

The assumption or constraint. All of TiDAR's evaluations are on standard few-shot benchmarks with fixed answer formats: coding benchmarks (HumanEval, MBPP) where the output is a complete function, math benchmarks (GSM8K, Minerva Math) where the output is a chain-of-thought leading to a final numeric answer, and multiple-choice likelihood tasks (MMLU, ARC, HellaSwag, PIQA, Winogrande). These tasks share a narrow structural property: each prompt has a single, well-defined correct answer that can be scored with exact-match or pass@1 metrics. The paper does not evaluate TiDAR on open-ended generation (creative writing, long-form QA, summarization, dialogue), multi-turn interaction, instruction-following, or any task where output quality is assessed by human judgment or learned reward models rather than exact match.

The consequence. The quality guarantee that TiDAR relies on — that autoregressive rejection sampling preserves the output distribution of the AR mode — applies to the distribution of generated sequences, not to the quality of those sequences as judged by downstream metrics. For benchmark tasks with exact-match scoring, preserving the AR distribution is sufficient to preserve benchmark scores (modulo finite-sample noise). But for open-ended tasks where quality is multidimensional (relevance, coherence, factual accuracy, stylistic appropriateness) and evaluation is approximate (LLM-as-judge, human preference ratings), it is unknown whether TiDAR's generation maintains the same quality characteristics as pure AR decoding from the same base model. Specifically:

• The rejection sampling guarantee is distributional, not instance-level. For any given prompt, TiDAR produces a different sequence than AR greedy decoding would (even with the same random seed, because the rejection sampling introduces stochasticity). This means that on tasks where the base model's output distribution has high variance in quality — as is common in open-ended generation — TiDAR could produce lower-quality outputs on some fraction of prompts even though the distribution of outputs matches. This is not a violation of the theoretical guarantee, but it matters for user-facing applications where each individual generation is judged.

• The acceptance rate on diverse, unpredictable content may be much lower. Benchmarks like HumanEval and GSM8K have highly structured output formats (function definitions, step-by-step reasoning). The next token in such formats is often predictable from local context (keywords, syntax patterns), which favors high diffusion draft acceptance rates. In open-ended generation — storytelling, explanation, argumentation — the content is less formulaic, and token-level predictions may be harder for the one-step diffusion drafter, leading to lower acceptance rates and reduced throughput. The paper provides no evidence about whether the 5-7× throughput advantage persists on less structured text.

What evidence exists in the paper. None. The evaluation suite (Table 6, Appendix A) consists exclusively of few-shot benchmarks with fixed-format outputs and exact-match or multiple-choice scoring. There is no instruction-tuned variant of TiDAR evaluated (all models are base models, not chat/instruct models), no open-ended generation task, and no human evaluation or LLM-as-judge evaluation. The paper's claim that "TiDAR strikes a nice balance between generative quality and efficiency... making it a highly appealing choice for many critical application scenarios with a stringent latency requirement" (Section 4.2.1) implicitly generalizes beyond the evaluated task types, but the evidence does not support this generalization.

Mitigation status. The paper does not acknowledge this as a limitation. The focus on base models and standard benchmarks is understandable — it enables clean comparison with prior work and isolates the architectural contribution — but the absence of instruction-tuned models and open-ended task evaluation means a practitioner considering TiDAR for a chatbot or assistant application has no data on which to base their decision. The paper does not suggest future work on instruction tuning or open-ended evaluation. In principle, TiDAR could be instruction-tuned by applying the same dual-mode training objective to instruction-following data, but this is neither discussed nor attempted.

---

6.6 The EAGLE-3 Baseline Comparison Is Not Matched for Model Type or Training Budget

The assumption or constraint. The paper's comparison against speculative decoding — specifically against EAGLE-3 [18], the state-of-the-art open-weight speculative decoding method — is presented as a key result: "we show for the first time that diffusion models can surpass the efficiency gains over speculative decoding" (Section 4.3). Figure 4 shows TiDAR-8B achieving 5.91× average relative throughput vs. EAGLE-3's 3.5-4.5×. However, this comparison is confounded by several mismatches that the paper acknowledges partially.

The consequence. The comparison is not like-for-like in three ways:

Model type mismatch: EAGLE-3 uses the Qwen3-8B-Instruct model as its base, while TiDAR-8B uses Qwen3-8B-Base. Instruction-tuned models often have different (sometimes worse) few-shot benchmark performance than base models because instruction tuning can interfere with in-context learning ability. The EAGLE-3 configurations achieve quality scores that appear lower than the Qwen3-8B-Instruct baseline on some tasks in Figure 4 (the reference point is Qwen3-8B-Instruct rather than Qwen3-8B-Base), making TiDAR's quality advantage potentially inflated. If EAGLE-3 were benchmarked with a base model (which would have higher raw scores on few-shot evaluations), the quality-throughput tradeoff comparison might shift.

Training budget mismatch: TiDAR-8B received 150B tokens of additional training via dual-mode continual pretraining. The EAGLE-3 draft heads were trained on a separate dataset with a separate training procedure and budget. These training budgets are not controlled or compared. TiDAR's architecture is the speculative mechanism — training the architecture and training the speculative capability are the same process — while EAGLE-3's draft heads are an add-on trained post-hoc. This makes a fair comparison of "architectural efficiency" difficult: is TiDAR's advantage due to a better architectural concept, or due to more effective training (150B tokens of end-to-end dual-mode training vs. whatever budget was used for EAGLE-3's draft head training)?

Guarantee mismatch: EAGLE-3, as a speculative decoding method, provides an exact output distribution preservation guarantee — the sequence of generated tokens is guaranteed to be identically distributed to what the base model would produce under the same sampling parameters. TiDAR, via rejection sampling against the same model's AR distribution, also provides this guarantee (the paper connects TiDAR to speculative decoding in Section 2.2 but does not explicitly claim distribution preservation). However, any quality comparison between a method that preserves the base model's exact output distribution and one that does not is comparing different things: EAGLE-3's quality is, by definition, identical to its base model's quality (modulo random seed), while TiDAR's quality can differ from its base AR model's quality because the model itself has changed through dual-mode training. The "quality" axis in Figure 4 means different things for the two methods.

The practical consequence: a user who needs exact output preservation from a specific base model (e.g., for reproducibility, safety certification, or regulatory compliance) cannot use TiDAR — they would need to use the TiDAR architecture trained from that specific base model, and the quality would differ from the base model due to dual-mode training. Speculative decoding with EAGLE-3 preserves the base model's outputs exactly. TiDAR trades this property for higher throughput, but the paper's comparison does not account for this tradeoff.

What evidence exists in the paper. The paper acknowledges the model type mismatch explicitly (Section 4.3): "We use the EAGLE-3 with Qwen3-8B instruct model due to lack of corresponding EAGLE-3 weights for the base model." However, this acknowledgment appears as a footnote (Footnote 2 in Section 4.3), not in the main text where the comparison is discussed. The training budget mismatch and guarantee mismatch are not acknowledged. The paper provides no comparison against other speculative decoding methods (Medusa, standard draft-model-based speculative decoding with a smaller model) that would contextualize the EAGLE-3 result.

Mitigation status. Partial acknowledgment (the model type issue), but the structural confounds in the comparison are not addressed. The paper does not train an EAGLE-3 variant from the Qwen3-8B-Base model to enable a matched comparison, nor does it control for training budget by reporting TiDAR performance at intermediate training checkpoints (which would show how much of the quality comes from the architecture vs. the 150B additional tokens). The claim of "surpassing speculative decoding" would be more defensible if the comparison were against standard speculative decoding with a smaller draft model (e.g., a 0.5B model drafting for the 8B) — which is the more common deployment scenario — and if training budgets were controlled. As it stands, the comparison establishes that TiDAR's particular training recipe produces better quality-throughput tradeoffs than the specific EAGLE-3 checkpoints available, but it does not isolate the architectural contribution from the effects of model type, training data, and training budget.

7. Implications and Future Directions

How This Work Changes the Landscape

TiDAR is best understood not as an incremental refinement of either diffusion language models or speculative decoding, but as a category-spanning architectural demonstration that dissolves the boundary between "the model" and "the inference acceleration mechanism." Prior to this work, the field operated with an implicit assumption that the drafting and verification roles in speculative decoding must be performed by different models or at least different parameter subsets — the drafter is smaller, shallower, or an add-on head. TiDAR shows that the same model, using the same weights, operating in two different attention modes within the same forward pass, can be its own drafter. This is a genuinely new point in the design space, and the 4.71× to 5.91× measured throughput improvements with minimal quality loss demonstrate that this point is practically significant, not just theoretically interesting.

The conceptual contribution is the reframing of "free compute" in the memory-bound regime as a resource to be algorithmically exploited, not just a hardware fact to be tolerated. Figure 1's profiling — showing that latency stays flat as token slots increase within the memory-bound plateau — is not itself novel (hardware-aware practitioners have long understood this property). But TiDAR is the first architecture to use those free token slots to run a complete speculative decoding loop — drafting and verification — within a single model and single forward pass. Prior speculative methods used the free compute only partially: they might share hidden states between the base model and drafter (EAGLE, Medusa), but the drafting step was still sequential to verification. TiDAR's key move is recognizing that once you accept the memory-bound premise, there is no reason drafting and verification cannot be parallel — they process different parts of the sequence with different attention masks, the computation for both is bottlenecked on the same weight loads, and the results can be reconciled via rejection sampling after the forward pass completes. This insight upends the two-phase mental model (draft → verify → draft → verify) that has structured LLM inference research since Leviathan et al. [11].

The paper also provides a unifying explanation for a set of contradictory findings in the diffusion language model literature. Prior work showed that diffusion models could achieve competitive perplexity and benchmark scores (MDLM, Llada, Dream) but also that parallel token generation caused substantial quality degradation (the 10% GSM8K drop cited from APD). TiDAR's formalization in Section 1 — decomposing the problem into the chain-factorized $p_{\text{AR}}$ versus the conditionally-independent $p_{\text{Diff\_Independent\_K}}$ — provides a clean theoretical basis for understanding why diffusion quality degrades with parallelism, and why rejection sampling fixes it. This is not an ad-hoc engineering fix but a principled resolution: AR rejection sampling filters the independence-assuming drafts to match the chain-factorized distribution. The empirical demonstration that this works at draft lengths of 8-16 tokens (far beyond the 1-2 tokens where pure diffusion quality collapses) validates the theoretical framing and reconciles the conflicting narratives: diffusion can be fast, and AR can provide quality, and the two can be combined without compromise because the drafting quality only needs to be high enough for acceptance — it does not need to be perfect.

For the LLM inference research community, this work redirects attention from draft-model design to attention-mask co-design. Prior speculative decoding research asked: How do we make a better draft model? (Smaller? Shared hidden states? Tree-structured drafts?) TiDAR asks a different question: How do we structure the attention mask so that a single forward pass computes both the draft and the verification? This is a fundamentally different optimization space — it is about how information flows between token positions within a single computation graph, not about how separate models coordinate across sequential steps. The paper's structured attention mask (Figure 3) — causal for the prefix and draft verification, block-causal bidirectional for the diffusion pre-drafting — is not just a training trick but the core architectural innovation. Future work on inference acceleration can now consider attention patterns as a first-class design dimension alongside model size, draft model capacity, and verification strategies.

The work also shifts the burden of proof for future inference acceleration methods. TiDAR achieves 4.71-5.91× throughput speedup while matching or approaching AR quality — all in native PyTorch without custom kernels. This sets a high empirical bar. Any new method claiming to accelerate LLM inference should compare against TiDAR (or a TiDAR-style architecture) in addition to standard AR baselines and speculative decoding. Methods that only marginally improve on EAGLE-3 but do not approach TiDAR's throughput-quality frontier will be harder to justify.

Finally, TiDAR weakens the case for pure diffusion language models as standalone generation architectures. If the best quality from diffusion comes from one-token-per-step decoding (which eliminates the throughput advantage), and parallel decoding introduces a quality penalty from the independence assumption, then the standalone diffusion paradigm faces a ceiling. TiDAR demonstrates that the productive use of diffusion in language modeling is as a drafting mechanism paired with AR verification — a hybrid role rather than a replacement. This does not mean research on pure diffusion LMs should stop, but it does suggest that the most impactful near-term applications of diffusion to LLM inference will come through hybridization rather than pure-diffusion generation. The paper's Pareto frontier analysis (Figure 5), showing TiDAR strictly dominating Block Diffusion at every point in the quality-throughput space, makes this case quantitatively.

Follow-Up Research This Work Enables

Instruction-tuned TiDAR and open-ended generation evaluation. The current paper evaluates only base models on few-shot benchmarks with exact-match scoring. The most urgent follow-up is to apply the TiDAR training recipe to an instruction-tuned model (e.g., Qwen3-8B-Instruct) and evaluate on open-ended tasks with LLM-as-judge or human evaluation. The specific experiment: take Qwen3-8B-Instruct, continue training with the TiDAR dual-mode objective for 100-150B tokens on instruction-following data, then benchmark on AlpacaEval, MT-Bench, and Chatbot Arena-style comparisons against the base instruct model. The key question is whether the 5.91× throughput advantage survives the transition to open-ended generation where token-level predictability is lower and output quality is multi-dimensional. If the acceptance rate drops substantially (e.g., from 8 T/NFE on benchmarks to 3-4 T/NFE on open-ended chat), TiDAR's practical value for chatbot deployment would be substantially lower than the headline numbers suggest. Conversely, if acceptance rates remain high — which would indicate that even diverse conversational text is locally predictable enough for one-step diffusion drafting — TiDAR would become immediately relevant to production LLM serving.

Dynamic block size scheduling based on local difficulty. TiDAR currently uses a fixed block size $K$ throughout generation. This is analogous to using a fixed beam width in search or a fixed number of samples in best-of-N — it cannot allocate more drafting effort to difficult tokens and less to easy ones. A natural extension is to make the block size adaptive: start with a small block (e.g., $K = 4$) and increase to larger blocks ($K = 8, 16$) when the acceptance rate over recent steps is high, or decrease when acceptance drops. The difficulty signal could be the AR mode's per-token entropy (high entropy → uncertain → reduce block size), the observed acceptance rate over a sliding window, or a learned meta-controller trained via RL to maximize throughput subject to a quality constraint. A concrete experiment: train a TiDAR model with block size 16, then at inference, after each forward pass, measure the average entropy of the AR distribution over the accepted tokens. If entropy exceeds a threshold, reduce the effective block size for the next step by truncating the draft (using only the first $K' < K$ pre-drafted tokens). Compare the throughput-quality curve of this adaptive scheme against fixed block sizes. The hypothesis is that adaptive scheduling would maintain higher average T/NFE on easy content while avoiding wasted draft slots on hard content, shifting the Pareto frontier further upward. The paper's observation that T/NFE varies substantially across tasks (5.07 on GSM8K vs. 9.43 on MBPP+, Table 2) already provides evidence that task-level difficulty affects acceptance rates — within-task adaptation should yield further gains.

Multi-step diffusion drafting for hard tokens. TiDAR uses one-step diffusion drafting — all mask tokens are predicted in a single forward pass. The paper justifies this by arguing that one-step quality is sufficient for high acceptance rates, and the evidence (Table 5, Table 4) supports this for the evaluated tasks. However, on harder content — novel reasoning chains, rare terminology, long-range dependencies in extended generation — one-step diffusion predictions may be substantially worse than multi-step predictions, reducing acceptance rates and throughput. A follow-up could explore two-step drafting: in regions where the AR distribution has high entropy (indicating uncertainty), spend an additional forward pass to refine the draft tokens using a standard denoising step (re-mask low-confidence draft tokens, re-predict them). The key question is whether the improved acceptance rate from better drafts justifies the cost of the extra forward pass. The specific experiment: during decoding, after the standard TiDAR forward pass, measure the confidence of the diffusion predictions (e.g., the softmax probability of the predicted token). If the average confidence over the $K$ draft tokens is below a threshold, perform a second forward pass where the lowest-confidence positions are re-masked and re-predicted (with the higher-confidence predictions kept as clean tokens, providing additional context). Measure the net T/NFE and T/s accounting for the extra forward passes. This would establish whether there exists a regime where multi-step drafting is net beneficial, or whether (as TiDAR's authors implicitly claim) the marginal gain from additional denoising steps never justifies their cost in the rejection-sampling framework.

TiDAR pretraining from scratch vs. continual pretraining. The current paper uses continual pretraining from strong AR checkpoints (Qwen2.5, Qwen3). This leaves open the question: can a TiDAR model be trained from scratch, and if so, does the dual-mode objective improve or harm pretraining efficiency compared to pure AR? The specific experiment: train two 1.5B models from random initialization on the same corpus for the same number of tokens — one with standard AR next-token prediction, one with the TiDAR dual-mode objective (with $\alpha = 1$). Compare not just downstream benchmark scores but also pretraining perplexity (the AR mode's NTP loss) as a function of training tokens. The hypothesis is that TiDAR's diffusion loss acts as a form of auxiliary task regularization — predicting tokens from bidirectional context (cloze-filling) is a harder task than left-to-right prediction, and training on both jointly might produce better representations that improve AR quality as well. If TiDAR from scratch matches or exceeds pure AR pretraining efficiency, the architecture would become attractive not just for inference acceleration but for training efficiency as well. Conversely, if the dual-mode objective slows down pretraining (because model capacity is split between two tasks), then TiDAR's value is purely as a post-hoc architecture modification applied to already-trained models.

Cross-architecture and cross-hardware profiling of the "free token slots" assumption. TiDAR's speedup relies on the memory-bound plateau shown in Figure 1, but this plateau's width depends on both the model architecture (which determines the FLOPs-per-token) and the GPU hardware (which determines the memory bandwidth and compute throughput). A systematic profiling study would measure the latency scaling over token slots for TiDAR at different model sizes (0.5B, 1.5B, 8B, 30B+) and on different GPUs (A100, L40S, H200, B200). For each (model, GPU) pair, identify the maximum block size $K$ that still fits within the free/cheap token slot regime, and then measure TiDAR's throughput at that optimal $K$. The output would be a hardware-dependent scaling law for TiDAR throughput — a function throughput_gain(model_size, GPU_type) that tells practitioners what speedup to expect on their specific infrastructure. This is essential for adoption: a practitioner with A100s needs to know whether TiDAR's 5.91× speedup on H100s translates to 3× or 2× or 0.8× on their hardware before they can make deployment decisions. The paper's current single-hardware evaluation leaves this question unanswered for the vast majority of deployed infrastructure.

TiDAR as a training-data generation accelerator for self-improvement loops. A distinct use case from interactive serving is batch inference for synthetic data generation — using an LLM to generate training data for itself or for smaller models (as in STaR, ReST^EM, or distillation pipelines). In this setting, throughput matters enormously because millions of examples must be generated, and batch sizes can be large. TiDAR's current evaluation at batch size 1 does not characterize its behavior in this regime. The specific experiment: benchmark TiDAR-8B at batch sizes 4, 8, 16, 32 on GSM8K and HumanEval generation, measuring both tokens-per-second and quality (pass@1). Compare against AR decoding at the same batch sizes. The hypothesis is that at larger batch sizes, the memory-bound advantage diminishes (the forward pass becomes compute-bound), and TiDAR's throughput advantage may shrink from 5.91× to something smaller — but even a 2-3× advantage with maintained quality would be highly valuable for data generation pipelines. Additionally, measure whether TiDAR-generated solutions have different diversity characteristics than AR-generated solutions — the rejection sampling procedure might subtly affect which modes of the AR distribution are represented in the output, which could impact the quality of downstream models trained on TiDAR-generated data.

Practical Applications and Downstream Use Cases

Latency-critical interactive applications with quality requirements. The paper's 1.5B results — lossless quality relative to the AR baseline with 4.71× throughput — directly enable deployment scenarios where a small model must respond quickly without sacrificing accuracy. Consider on-device code completion in an IDE: the model needs to suggest the next few lines of code within tens of milliseconds to feel responsive, and incorrect suggestions frustrate users. A TiDAR-1.5B model (or smaller) running locally, benefiting from the "free token slots" on an edge GPU or even a powerful CPU with sufficient memory bandwidth, could generate code suggestions at 4-5× the tokens per second of an equivalent AR model, reducing perceived latency while maintaining the AR model's suggestion quality. The key numbers: TiDAR-1.5B achieves 43.29% on HumanEval (vs. 35.98% for the base Qwen2.5-1.5B) while producing 6.50 tokens per forward pass. This means the model can draft multiple lines of code in the time an AR model would draft one, with no quality penalty.

High-throughput batch evaluation and synthetic data pipelines. Organizations that run LLMs over large datasets — evaluating thousands of math problems, generating training data for distillation, scoring candidate solutions — care primarily about total wall-clock time and cost. TiDAR's 5.91× throughput at 8B with a 2.8 percentage point average quality gap (65.31% vs. 68.09%) represents a compute-cost reduction of approximately 83% for a small quality tradeoff. For use cases where the quality difference is acceptable (e.g., generating candidate solutions that will be filtered by a verifier, or evaluating on tasks where TiDAR's gap is even smaller — 1.4 points on GSM8K, 1.3 points on Minerva Math), switching from AR to TiDAR decoding would directly reduce GPU-hours by a factor of 5-6. The paper's native PyTorch implementation — without custom kernels — means that even greater efficiency is possible with engineering investment, making TiDAR a strong candidate for cost-sensitive batch workloads.

Model serving with dynamic quality-throughput tradeoffs. TiDAR's robustness to the block size (quality is nearly flat from block size 4 to 16 at 1.5B, Table 4) and the absence of inference hyperparameters suggest a deployment architecture where the block size is selected per-request based on the user's latency tolerance or priority tier. A premium-tier API customer might receive TiDAR decoding with a conservative block size (4-8 drafts, highest per-token acceptance rate, minimal quality loss), while a free-tier or batch-processing customer might receive aggressive drafting (block size 16, slightly higher rejection rate but more tokens per second). Because the same trained model supports all block sizes (the paper notes "zero-shot" adjustment of draft length), this requires no model swapping or weight reloading — just a parameter change in the inference loop. The quality-throughput curve in Figure 4 (the three TiDAR points per task at 1.5B) already characterizes this tradeoff space, giving service operators a data-driven basis for tiered offerings.

When to Prefer This Method

The paper explicitly positions TiDAR against three named alternatives — pure AR decoding, Block Diffusion, and speculative decoding (EAGLE-3) — and the experimental results in Figures 4-5 and Tables 2-4 support specific decision rules:

• Prefer TiDAR over pure AR decoding when you are deploying at batch size 1 on a GPU where memory bandwidth is the bottleneck (most modern GPUs at interactive batch sizes), and you can tolerate a small quality gap (0-3 percentage points on average benchmark scores) in exchange for 4-6× higher throughput. The case is strongest for coding and math tasks where TiDAR's quality gap is smallest or even negative (TiDAR-1.5B exceeds its AR base on HumanEval). The case is weaker for tasks requiring maximum accuracy where even a 2-3 point gap is unacceptable, or when deploying at large batch sizes where the memory-bound assumption may not hold (the paper provides no large-batch data).

• Prefer TiDAR over Block Diffusion when you need both quality and throughput from a diffusion-style architecture. Figure 5 demonstrates that TiDAR strictly dominates Block Diffusion at every quality-throughput operating point under matched training — there is no regime where Block Diffusion is preferable if TiDAR is available. The only reason to use Block Diffusion would be if the TiDAR training recipe cannot be applied (e.g., you have a model that cannot be fine-tuned, or you need to work with an existing Block Diffusion checkpoint without retraining).

• Prefer TiDAR over EAGLE-3 speculative decoding when you are willing to modify the base model through additional training (continual pretraining with the dual-mode objective) and you prioritize throughput over exact output preservation. TiDAR achieves substantially higher throughput than EAGLE-3 (5.91× vs. 3.5-4.5× in Figure 4), but at the cost of (a) requiring 150B tokens of architecture-specific training, (b) producing a model whose output distribution differs from the original base model, and (c) not having been validated on instruction-tuned or chat models. The paper's EAGLE-3 comparison has caveats (different base model type, uncontrolled training budgets), so this preference should be treated as provisional until a matched comparison is conducted.

• Prefer standard AR decoding when you require exact reproducibility from a specific pretrained checkpoint, cannot afford additional training, or operate in a regime (very large batch sizes, very long contexts, hardware without a wide memory-bound plateau) where TiDAR's speedup has not been demonstrated. The paper's limitations (Section 5) explicitly flag long-context and non-batch-size-1 scenarios as unexplored.