Jet-RL: Enabling On-Policy FP8 Reinforcement Learning with Unified Training and Rollout Precision Flow

ArXiv: 2601.14243

🎯 Pitch

Quantizing rollout to FP8 seems like an obvious efficiency win, but it secretly breaks training because the model now learns from behavior generated under a different numerical precision than its own. Jet-RL shows that simply matching the precision flow for both training and rollout eliminates this collapse, recovering BF16-quality convergence while cutting end-to-end time by 16%.

1. Executive Summary

This paper analyzes the stability and efficiency of FP8 quantization in reinforcement learning (RL) training for large language models, identifying that the widely-adopted BF16-train-FP8-rollout strategy introduces a critical mismatch between training and inference precision flows. Through experiments on the MATH and GSM8K benchmarks with Qwen-series and Llama models, the authors demonstrate that this off-policy discrepancy causes catastrophic accuracy collapse under long-rollout generation (e.g., sequence lengths exceeding 8K tokens) and on challenging tasks where the base model lacks strong priors. The paper proposes Jet-RL, an on-policy FP8 RL framework that enforces a unified precision flow across both training forward passes and rollout inference (using identical 128×128 per-block weight quantization and 1×128 per-group activation quantization for all GEMM operations), which eliminates policy mismatch without requiring expensive inter-step calibration. Jet-RL achieves up to 1.33× rollout phase speedup, 1.41× training phase speedup, and 1.16× end-to-end speedup over BF16 baselines while maintaining robust convergence across all tested configurations and reducing the accuracy gap to BF16 training to approximately 1%, establishing that on-policy FP8 RL training is viable only when precision flow is unified between the forward and rollout graphs.

2. Context and Motivation

The Core Problem: Rollout Dominates RL Training, But Quantizing It Breaks Everything

The specific gap this paper addresses is deceptively simple to state but technically thorny to resolve: how do you accelerate the rollout phase of RL training for LLMs using FP8 quantization without destroying training stability? The rollout phase — where the actor model autoregressively generates complete responses to prompts — has become the dominant bottleneck in reinforcement learning pipelines for reasoning models. As the paper demonstrates in Figure 2, when the maximum rollout length exceeds 8,000 tokens, generation alone consumes more than 70% of total training time. This proportion grows with sequence length, making it the single most expensive component of the RL training loop.

This problem is not merely an inconvenience — it fundamentally constrains the types of reasoning tasks we can train models to perform. The emergence of Chain-of-Thought (CoT) reasoning as the dominant paradigm for complex problem-solving means that effective RL training increasingly requires generating very long sequences. Models reasoning about advanced mathematics, scientific problems, or multi-step logical puzzles routinely produce thousands of tokens per response. If the rollout phase scales poorly with length, it creates a hard economic ceiling on how much reasoning capability we can feasibly train into models. You cannot just throw more GPUs at the problem indefinitely; the autoregressive decode is inherently sequential and GPUs spend most of their time idle waiting on memory transfers rather than computing.

The promise of FP8 quantization is straightforward: half the precision means (roughly) twice the throughput for compute-bound operations, and modern NVIDIA H100 GPUs have native FP8 tensor cores optimized for exactly this workload. If you can make the rollout run in FP8 while keeping training in BF16, you get most of the speedup with (theoretically) none of the accuracy cost. This BF16-train-FP8-rollout strategy is the natural first thing anyone would try, and indeed, it has been widely adopted in major RL frameworks including VeRL [16], SLIME [17], NeMo-RL [18], and OpenRLHF [19].

The paper's central finding is that this natural strategy is fatally flawed — not in some edge case, but in exactly the regimes where RL training matters most. And the failure mode is not a gradual degradation but a catastrophic collapse.

Why the Problem Is Important: The Economics of Reasoning

The significance of this problem extends well beyond an implementation detail of RL pipelines. Three interlocking trends make it urgent:

First, reasoning models are the frontier. The most capable AI systems today — OpenAI's o1, DeepSeek-R1, and their successors — derive their problem-solving abilities from RL training on verifiable reasoning tasks. These models don't just memorize answers; they learn to explore solution spaces, backtrack from dead ends, and chain together dozens of logical steps. Training this capability requires generating millions of long CoT trajectories and learning from which ones succeed. If the rollout phase cannot be made efficient, the cost of training reasoning models will remain prohibitively high, limiting who can develop them and how extensively they can be trained.

Second, rollout cost scales with ambition. The paper's Figure 2 shows that increasing maximum rollout length from 1K to 16K shifts the rollout fraction of total training time from roughly below 50% to well above 75%. But this is not just about current workloads — it's about the direction of travel. As we ask models to solve harder problems, they need to generate longer reasoning traces. A model that can solve a simple algebra problem in 200 tokens may need 8,000 tokens for a competition math problem, and 32,000+ for a graduate-level proof. If we want to keep pushing the reasoning frontier, the rollout bottleneck will only tighten. An acceleration method that fails at exactly the long-sequence lengths where acceleration is most needed is not just suboptimal — it's counterproductive.

Third, the quantized rollout problem intersects with a deeper RL challenge: on-policy vs. off-policy training. RL algorithms like PPO and GRPO rely on the assumption that the data used to update the policy was generated by the current policy. When there's a mismatch — when the rollout distribution differs from the training distribution — you're doing off-policy RL, which is notoriously unstable and can diverge. The BF16-train-FP8-rollout approach inadvertently creates exactly this mismatch: the forward pass during training (in BF16) produces different logits than the rollout (in FP8), so the actor model is being updated based on experiences it didn't actually generate. This is not a hypothetical concern — the paper shows empirically that the consequences are severe and systematic.

Prior Approaches and Their Shortcomings

Before Jet-RL, there were essentially two approaches to accelerating RL rollouts with quantization, and both fall short in important ways.

Approach 1: BF16-Train-FP8-Rollout with Calibration

Post-Training Quantization (PTQ) methods like SmoothQuant [37], GPTQ [36], and AWQ [35] have been highly successful for offline LLM deployment. They work by analyzing a small calibration dataset to determine optimal scaling factors that minimize quantization error. Applying this methodology to RL seems natural: before each rollout, calibrate the FP8 quantization on a few example prompts, then generate.

The problem, which the paper identifies explicitly in Section 3.2, is that RL training requires frequent weight synchronization. After every training step, the updated actor weights must be transferred from the training framework (e.g., FSDP) to the inference engine (e.g., vLLM). Each transfer invalidates any previously computed calibration. Recalibrating from scratch at every synchronization step is prohibitively expensive — the paper notes it "can take tens of minutes even for small 8B LLMs." When your goal is to accelerate training, spending tens of minutes per step on calibration completely defeats the purpose. Calibration-based PTQ is designed for deploy-once-infer-many scenarios, not for the rapidly evolving weights of an RL training loop.

The frameworks that adopted BF16-train-FP8-rollout recognized this, which leads to the second approach.

Approach 2: BF16-Train-FP8-Rollout without Calibration

Several frameworks, including SLIME [38] and NeMo-RL [39], take the pragmatic shortcut of skipping calibration entirely. They simply cast the BF16 weights to FP8 directly — a trivial type conversion that uses the full representable range of FP8 without any data-dependent optimization. The claim, as the paper cites, is that "the accuracy is not affected."

The paper systematically demolishes this claim by showing it holds only under narrow, favorable conditions that are not representative of ambitious RL training. Through careful experiments varying both sequence length and task difficulty, the paper reveals two distinct failure modes:

Failure Mode 1: Long-Rollout Collapse (Figure 3). The paper trains Qwen2.5-7B on MATH using GRPO, varying the maximum rollout length from 4K to 16K tokens. At 4K, BF16-train-FP8-rollout matches BF16 training — the two curves are essentially indistinguishable. At 8K, the FP8 rollout curve begins to diverge noticeably, falling behind the BF16 baseline. At 16K, the collapse is catastrophic: after roughly 20 training steps, the FP8 rollout accuracy plummets and never recovers, while BF16 training continues improving monotonically.

What's happening here? The paper offers a precise mechanistic hypothesis: at each autoregressive decoding step, the FP8-quantized model produces logits that differ slightly from what the BF16 model would produce. These differences are individually tiny — perhaps a fraction of a percent in any given token's probability. But over thousands of tokens, these discrepancies accumulate. The token selected at step 50 conditions all subsequent tokens; a slightly different token at step 50 leads to a different distribution at step 51, which leads to an even more different distribution at step 52, and so on. By the time you reach 8,000 or 16,000 tokens, the trajectory has diverged substantially from what the BF16 model would have generated. The actor model is being trained on sequences it never would have produced, violating the on-policy assumption. The RL update that follows amplifies rather than corrects this discrepancy, creating a destructive feedback loop.

This accumulation hypothesis explains why the problem is invisible at 4K — the divergence hasn't had enough steps to compound — but becomes catastrophic at 16K. It also suggests why simply checking accuracy at short sequence lengths (as prior frameworks likely did) gives a falsely optimistic picture.

Failure Mode 2: Challenging Task Instability (Figure 4). The paper demonstrates a second, subtler failure mode that depends not on sequence length but on the model's prior competence on the task. When training Qwen3-8B (the instruction-tuned variant) on MATH, BF16-train-FP8-rollout performs well and even converges slightly faster than BF16 training. But when switching to Qwen3-8B-Base (the pretrained-only variant, without instruction tuning), the same method quickly falls behind the BF16 baseline and never catches up.

The paper's interpretation is insightful: when the model is strong and the task is relatively easy, the model's output distribution is "peaked" — it has high confidence in its token choices. Small numerical perturbations from FP8 quantization don't typically flip the argmax, so the rollout trajectory remains similar to what BF16 would produce. The model is robust to quantization noise because its preferences are decisive. But when the model is weak and the task is genuinely hard, its output distribution is more diffuse — many tokens have similar probabilities, and small quantization errors can easily change which token gets selected. Each such "flip" sends the trajectory down a different path, creating exactly the off-policy divergence problem that RL theory warns against.

This finding has an important corollary: BF16-train-FP8-rollout fails precisely when RL training is most valuable. If the model already performs well on the task, you don't need RL to improve it. The whole point of RL training is to teach models capabilities they don't yet possess — exactly the regime where BF16-train-FP8-rollout is least stable. Prior work that claimed success for this approach was likely testing on tasks the model could already do, inadvertently selecting for the one scenario where the method happens to work.

The Broader RL Landscape: Why On-Policy Matters

The paper situates its contribution within a well-established RL principle: off-policy training is inherently unstable. This is not a new observation — it's a foundational result in reinforcement learning going back decades. When you update a policy using data generated by a different policy (or, in this case, a slightly different numerical implementation of the "same" policy), the importance sampling corrections needed to make the updates valid can have unbounded variance. In the limit, the training diverges.

For LLM-specific RL training, this problem has been discussed in the context of inference nondeterminism [40, 41, 42] — different hardware, different CUDA kernels, or even different random seeds can produce slightly different logits, creating an off-policy situation even when using the same precision. The BF16-train-FP8-rollout approach amplifies this problem enormously: it's not a subtle hardware nondeterminism but a deliberate change of numerical format that guarantees the forward pass computations differ at every layer. Recent work on Truncated Importance Sampling (TIS) [47] has attempted to mitigate off-policy effects in quantized RL by adding importance ratios to the policy update and truncating them when the inference probability is small. But these are post-hoc corrections that don't address the root cause — the mismatch still exists, and importance sampling can only do so much to correct for it before variance explodes.

The Implementation Challenge: Graph Mismatch

To properly understand why BF16-train-FP8-rollout fails, the paper introduces a formal framework in Section 4 that is worth previewing here because it reveals the architectural nature of the problem.

Model computation can be modeled as a directed graph where nodes are operators (matrix multiplies, normalizations, activations) and edges represent the tensors flowing between them. The precision of each edge — whether it carries BF16 or FP8 values, and if FP8, with what quantization granularity — defines the numerical trajectory of a forward pass.

In BF16 training, the training forward graph $\mathcal{G}_{\text{fwd}}^{\text{train}}$ and the inference graph $\mathcal{G}^{\text{infer}}$ are identical: every edge carries BF16, every operator computes in BF16. There is no mismatch.

In BF16-train-FP8-rollout, the training forward graph stays in BF16 — all activations, all weights, all operator outputs are BF16. But the inference graph has FP8 edges feeding into every linear layer: weights are quantized to FP8, activations are quantized to FP8 before the GEMM. These two graphs are different objects performing different computations. The actor model is trained assuming its forward pass follows $\mathcal{G}_{\text{fwd}}^{\text{train}}$, but the rollouts are generated by $\mathcal{G}^{\text{infer}}$. The RL update optimizes the wrong thing.

This graph-theoretic perspective is powerful because it clarifies why the problem is fundamental, not superficial. It's not about choosing better scaling factors or using a fancier quantization scheme within the rollout engine (though those help). It's about the fact that training and rollout are computing different functions. No amount of calibration can fix that if the underlying graph topology differs in precision.

How Jet-RL Positions Itself

Jet-RL's core insight is that the solution to the off-policy problem is not to apply corrections after the fact or to find better FP8 scaling factors for the rollout only. The solution is to make $\mathcal{G}^{\text{infer}}$ a subgraph of $\mathcal{G}_{\text{fwd}}^{\text{train}}$ — in other words, to force the training forward pass and the inference forward pass to use the identical FP8 computation at every edge where they overlap, so there is simply no mismatch to correct.

This is a fundamentally different design philosophy from prior work. Instead of treating quantization as an inference-only optimization bolted onto a BF16 training pipeline, Jet-RL quantizes the training forward pass itself to match the quantized inference. The backward pass can remain in higher precision (the paper uses BF16 for gradients and master weights), which is standard practice in quantized training — the forward pass computes at low precision for speed, the backward pass computes at higher precision for stability, and weight updates are accumulated in high precision [14, 24].

This approach has a direct precedent in the quantized training literature: systems like NVIDIA's Transformer Engine [55], COAT [14], and Jetfire [24] have demonstrated that FP8 or INT8 forward passes can be used during pretraining and fine-tuning without accuracy loss, provided the backward pass and optimizer states maintain sufficient precision. Jet-RL extends this principle to the RL setting, where the additional constraint is that the inference engine's forward pass must be identical to the training forward pass. By unifying them, the on-policy assumption is restored: when the actor generates a rollout using the FP8 inference engine, the logits it produces are exactly what the training forward pass would have produced on the same inputs. The RL update is genuinely on-policy.

The paper's positioning is thus: Jet-RL is not proposing a new quantization scheme per se, but rather identifying that the correct quantization scheme for RL is one that unifies the numerical graphs of training and inference. The specific quantization granularities chosen — 128×128 per-block for weights, 1×128 per-group for activations — are drawn from proven configurations in the quantized training literature and are compatible with highly optimized kernels like DeepGEMM [21]. The innovation is the architectural decision to enforce graph equality, not the per-tensor quantization formula.

The paper also implicitly positions itself against the common wisdom that "training should stay in BF16 for accuracy." By demonstrating that FP8 forward passes are stable even during RL training — a more demanding regime than pretraining because the data distribution shifts as the policy improves — Jet-RL challenges the assumption that training must be high-precision. The empirical results show that with unified precision flow, FP8 RL training converges essentially identically to BF16 (within ~1% accuracy) across diverse models, datasets, and rollout lengths, while the supposedly safer BF16-train-FP8-rollout approach fails catastrophically. The irony is that keeping training in BF16 is what causes the failure, not what prevents it.

This reframing — from "quantize inference to accelerate training" to "unify precision flow to enable stable training" — is the conceptual contribution that distinguishes Jet-RL from prior quantized RL approaches and from generic FP8 inference acceleration methods. It's not just a better quantization recipe; it's a better architecture for the RL training loop itself.

3. Technical Approach

3.1 Reader Orientation

Jet-RL is a framework for running the forward passes of both RL training and rollout generation in the exact same FP8 precision, rather than running training forward passes in BF16 and rollout in FP8. It solves the problem of catastrophic training collapse that occurs when a model is updated based on rollout trajectories that differ numerically from what the training forward pass would have produced, by forcing the computational graphs of training and inference to be identical down to the per-tensor quantization granularity — making the RL process genuinely on-policy even under low-precision computation.

3.2 Big-Picture Architecture (Diagram in Words)

The system integrates four major components into a precision-unified RL training loop:

1. Inference Engine (vLLM) — handles the rollout phase, generating complete autoregressive responses from the actor model using FP8-quantized weights and activations. This is the standard vLLM deployment with FP8 weight-only quantization applied at load time.

2. Training Framework (FSDP via VeRL) — handles the update phase, executing forward and backward passes to compute gradients and update model weights. The key modification is that the forward pass of each linear layer uses the same FP8 GEMM kernels as the inference engine, producing identical outputs for the same inputs.

3. FP8 Quantization Layer (Custom Triton Kernels) — implements the three GEMM operations (FProp, WGrad, DGrad) at specific granularities: 128×128 per-block quantization for weights and 1×128 per-group quantization for activations and gradients. This layer also handles the transposition and requantization needed when activations are consumed by both DGrad (row-wise) and WGrad (column-wise) kernels.

4. Precision Flow Orchestrator (Jet-RL Core Logic) — ensures that at every edge of the computation graph where training and inference overlap, the precision format and quantization granularity match exactly. The BF16 master weights are quantized to FP8 identically before every forward pass (training and inference), and activations flowing between layers are quantized with the same scheme in both contexts. The backward pass, weight updates, and optimizer states remain in BF16.

Information flows through a single RL step as follows: (1) the actor weights are quantized from BF16 to FP8 (128×128 per-block), (2) the FP8 weights are loaded into both the inference engine and the training forward pass, (3) the inference engine generates complete rollout sequences using FP8 GEMMs for all linear layers, (4) the reference, reward, and critic models evaluate these rollouts at BF16 precision, (5) the training framework executes the actor's forward pass using the same FP8 GEMMs as the inference engine (producing identical logits), then runs the backward pass with FP8 WGrad and DGrad kernels but BF16 gradient accumulation, and (6) the BF16 master weights are updated with the optimizer and the cycle repeats — with the next quantization step producing slightly different FP8 weights reflecting the update.

3.3 Roadmap for the Deep Dive

• First, the unified precision flow — why making the inference graph a subgraph of the training forward graph solves the off-policy problem, and what "matching precision flow" means concretely at the level of graph edges.
• Second, the formal graph model (𝒢train, 𝒢infer) that makes the mismatch in BF16-train-FP8-rollout precise and shows exactly what Jet-RL changes to fix it.
• Third, the quantization scheme for each GEMM type (FProp, WGrad, DGrad) — the granularity choices, the kernel constraints, the transposition and requantization logic, and why these specific configurations were chosen.
• Fourth, the system implementation — how Jet-RL is built on top of vLLM and VeRL, what custom kernels were written, and how weight synchronization and quantization are orchestrated across the training step.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems paper whose core idea is that RL training stability under FP8 inference requires the forward pass of training to use the same FP8 computation as inference, not the supposedly safer BF16 forward pass that creates an off-policy discrepancy.

The Core Problem: Off-Policy Training from Graph Mismatch

The BF16-train-FP8-rollout approach fails because it creates two different computation graphs for what should be the same function. During training, the actor model's forward pass computes:

$Y_{\text{train}} = f_{\text{BF16}}(X; W_{\text{BF16}})$

where $f_{\text{BF16}}$ is the model's forward function with all linear layers computed in BF16 precision, $X$ is the input prompt, and $W_{\text{BF16}}$ are the BF16 master weights. During rollout, the inference engine computes:

$Y_{\text{rollout}} = f_{\text{FP8}}(X; \mathcal{Q}(W_{\text{BF16}}))$

where $\mathcal{Q}(\cdot)$ is the FP8 quantization function applied to the weights, and $f_{\text{FP8}}$ uses FP8 GEMMs for all linear layers. Even though both functions receive the same prompt $X$ and operate on the same underlying weights, the outputs differ because the numerical precision of every matrix multiplication differs. The actor model is then updated using advantage estimates derived from $Y_{\text{rollout}}$, but the policy gradient is computed through $f_{\text{BF16}}$. The gradient points in a direction that would improve $f_{\text{BF16}}$'s outputs on the training data, but the data was generated by $f_{\text{FP8}}$. This is the definition of off-policy RL.

What makes this particularly dangerous for long sequences is that the discrepancy compounds autoregressively. Let $y_t$ be the token sampled at step $t$. Under BF16, the model would compute:

$P_{\text{BF16}}(y_t | y_{<t}, X) = f_{\text{BF16}}(X, y_{<t})$

Under FP8, it computes:

$P_{\text{FP8}}(y_t | y_{<t}, X) = f_{\text{FP8}}(X, y_{<t})$

If these two distributions differ even slightly, the sampled token $y_t$ may differ. That different token then conditions all subsequent steps, so $y_{t+1}$ is sampled from a distribution over a different prefix than the BF16 model would have used. The KL divergence between the BF16 trajectory distribution and the FP8 trajectory distribution grows (roughly) as:

$D_{\text{KL}}(P_{\text{BF16}}(\tau) \| P_{\text{FP8}}(\tau)) \approx \sum_{t=1}^{T} D_{\text{KL}}(P_{\text{BF16}}(y_t | y_{<t}^{\text{BF16}}) \| P_{\text{FP8}}(y_t | y_{<t}^{\text{FP8}}))$

where $\tau$ is a complete trajectory of length $T$, and $y_{<t}^{\text{BF16}}$ and $y_{<t}^{\text{FP8}}$ are the (potentially different) prefixes generated by each precision. The summation over $T$ steps means that even a tiny per-step discrepancy multiplies by the sequence length, which is why the problem is invisible at 4K tokens but catastrophic at 16K.

Jet-RL's solution eliminates this mismatch at its source by making $f_{\text{FP8}}$ the forward function for both training and rollout. The training forward pass no longer uses BF16 for linear layers — it uses the identical FP8 GEMMs as the inference engine. The backward pass and weight updates remain in BF16, but the forward computation that generates the logits (and thus the probabilities that determine which tokens are sampled) is bit-identical between training and rollout to the extent that FP8 arithmetic is deterministic.

This is the paper's central design principle: the rollout precision graph must be a subgraph of the training forward precision graph, with all shared edges having identical precision and quantization granularity.

The Formal Graph Model: Making Precision Flow Precise

To reason rigorously about precision flow, the paper models model computation as a directed graph:

$\mathcal{G} = (\mathcal{V}, \mathcal{E})$

where $\mathcal{V}$ is the set of nodes (operators and weights) and $\mathcal{E}$ is the set of directed edges (tensors flowing between operators). Each edge carries metadata specifying:

• The numerical precision of the tensor (BF16 or FP8)
• If FP8, the quantization granularity (per-tensor, per-block 128×128, per-group 1×128, etc.)

This formalism lets us state the problem and solution precisely:

The training graph $\mathcal{G}_{\text{train}}$ has two subgraphs: $\mathcal{G}_{\text{fwd}}^{\text{train}}$ for the forward pass (edges carry activations) and $\mathcal{G}_{\text{bwd}}^{\text{train}}$ for the backward pass (edges carry gradients). These share the same node topology since they run the same operators, but edges flow in opposite directions. The two subgraphs are connected by "saved activation" edges: during the forward pass, certain activation tensors are stored for use during the backward pass.

The inference graph $\mathcal{G}_{\text{infer}}$ has the same topology as $\mathcal{G}_{\text{fwd}}^{\text{train}}$ (same sequence of operators) but potentially different edge precision annotations.

What it computes: This formalization provides a vocabulary for describing quantization strategies. Instead of saying "we quantize the weights before linear layers," we can say "the edges feeding into GEMM operator nodes in $\mathcal{G}$ are annotated as FP8 with 128×128 per-block quantization." This precision annotation is applied uniformly to every edge that matches the pattern, ensuring consistency.

Why this form: The graph abstraction separates the model's architectural structure (which operators exist and how they connect) from its numerical implementation (what precision each intermediate tensor uses). The architectural structure is fixed by the model definition; the numerical implementation is what we control. By requiring $\mathcal{G}_{\text{infer}}$ to be a subgraph of $\mathcal{G}_{\text{fwd}}^{\text{train}}$ with identical edge annotations, we guarantee that there is no path through the model where the same operator receives different numerical inputs during training versus inference.

The Mismatch in BF16-Train-FP8-Rollout (Formalized)

Under BF16-train-FP8-rollout, the annotations differ:

• In $\mathcal{G}_{\text{fwd}}^{\text{train}}$, all edges are annotated as BF16. Every activation tensor flowing between operators, every weight tensor feeding into linear layers — all BF16.
• In $\mathcal{G}_{\text{infer}}$, the edges between RMSNorm and the first linear layer, and between any activation-producing operator and a GEMM input, are annotated as FP8 (typically per-tensor or per-token quantization). The weight edges feeding into GEMMs are also FP8.

These are two different annotated graphs. The forward pass during training follows $\mathcal{G}_{\text{fwd}}^{\text{train}}$; the rollout follows $\mathcal{G}_{\text{infer}}$. This is the graph-theoretic representation of the off-policy problem.

Critically, the mismatch is not just at the final output — it occurs at every linear layer. Consider a single transformer block. Under BF16 training, the attention projection receives BF16 query, key, and value tensors and computes a BF16 matmul. Under FP8 rollout, those same query/key/value tensors have been quantized to FP8 (by the preceding RMSNorm + quantization step) and the matmul is computed in FP8. The output of that matmul differs. This difference propagates through the residual connection, through the next RMSNorm, through the MLP projections — compounding at every layer. By the final layer, the logits can differ substantially, leading to different token sampling decisions.

The paper's key insight is that you cannot fix this with better quantization of the inference engine alone. No matter how clever your FP8 scaling factors are, $\mathcal{G}_{\text{infer}}$ still differs from $\mathcal{G}_{\text{fwd}}^{\text{train}}$. The RL update will always be optimizing a function that differs from the one that generated the data. The only fix is to change $\mathcal{G}_{\text{fwd}}^{\text{train}}$ to match $\mathcal{G}_{\text{infer}}$.

Jet-RL's Solution: Unified Precision Flow

Jet-RL enforces:

$\mathcal{G}_{\text{infer}} \subseteq \mathcal{G}_{\text{fwd}}^{\text{train}}$

with all shared edges having identical precision annotations. In practice, this means:

1. All linear layer GEMMs in the training forward pass use FP8. The FProp operator reads FP8-quantized activations and FP8-quantized weights, computes the matrix multiplication in FP8 tensor cores, and accumulates the result to BF16.
2. The inference engine uses the identical FP8 GEMMs. Same kernel, same quantization granularity, same accumulation behavior.
3. Weight quantization is identical. The BF16 master weights are quantized to FP8 using the same 128×128 per-block scheme before every training forward pass and before every rollout generation pass.
4. Activation quantization is identical. After every operator that produces an activation consumed by a GEMM (e.g., RMSNorm before QKV projection, residual add before MLP), the activation is quantized to FP8 using 1×128 per-group quantization in both training and inference.

The only difference between $\mathcal{G}_{\text{fwd}}^{\text{train}}$ and $\mathcal{G}_{\text{infer}}$ is that $\mathcal{G}_{\text{fwd}}^{\text{train}}$ also includes edges for saved activations (stored in FP8) that are consumed by the backward pass during training. These edges don't exist in $\mathcal{G}_{\text{infer}}$ because inference has no backward pass. But for every edge that both graphs share — every activation flowing from one operator to the next — the precision annotation is identical.

What this achieves computationallly: When the same prompt is fed to the training forward pass and to the inference engine, the sequence of tensor values produced at every layer is bit-identical (up to FP8 nondeterminism, which is typically negligible compared to the per-step differences between BF16 and FP8). The logits at the final layer are identical. The probability distribution over next tokens is identical. The rollouts are genuinely on-policy — the actor model is trained on exactly the data its current policy would generate.

Why this form over alternatives: An alternative approach would be to keep training in BF16 but apply importance sampling corrections to the RL update to account for the mismatch between $P_{\text{BF16}}$ and $P_{\text{FP8}}$. This is what Truncated Importance Sampling (TIS) [47] attempts. The problem is that for long sequences, the importance weight is a product of per-step probability ratios:

$w(\tau) = \prod_{t=1}^{T} \frac{P_{\text{BF16}}(y_t | y_{<t})}{P_{\text{FP8}}(y_t | y_{<t})}$

Even if each per-step ratio is close to 1 (say, 0.99 to 1.01), the product over thousands of steps can explode or vanish — this is the classic curse of dimensionality in importance sampling. TIS truncates these weights, which introduces bias. Jet-RL avoids this entire problem by making the ratio exactly 1.0 at every step (the two distributions are the same), meaning no importance sampling correction is needed at all. The RL training is on-policy by construction, not by post-hoc correction.

Saved Activations: FP8 for the Forward-to-Backward Bridge

A subtle design choice concerns the precision of activations saved during the forward pass for use in the backward pass. During BF16 training, these saved activations are naturally in BF16 — that's the precision they were computed in. But under Jet-RL's unified FP8 forward pass, the activations flowing between operators are FP8. The question is: should the backward pass receive these saved activations in FP8 (as computed) or should they be upcast to BF16 before saving?

The paper chooses to save activations in FP8. The rationale rests on a practical constraint: the FProp GEMM kernel typically fuses the activation quantization with the matmul computation. The code runs (conceptually):

activation_fp8 = quantize_1x128(activation_bf16)
output_bf16 = fp8_gemm(activation_fp8, weight_fp8)

The quantization to FP8 happens inside the GEMM kernel or immediately before it, and the original BF16 activation may not be preserved (or preserving it would cost memory bandwidth). When the backward pass needs this activation for WGrad (which computes the weight gradient as $\nabla_W = \nabla_Y^\top \times X$), the only version available is the FP8-quantized copy. Saving the BF16 version would require either recomputation (expensive) or an extra memory buffer (defeating the memory savings of quantization).

This design choice is validated by prior work on quantized training [14, 24], which showed that storing activations in FP8 for the backward pass does not destabilize training provided the quantization granularity is fine enough (1×128 per-group, in Jet-RL's case) and the backward pass GEMMs also use FP8. The key property is that the backward pass sees the same quantized activations that the forward pass operated on, so the gradients are computed with respect to the function that was actually evaluated — there's no mismatch between the forward function and the backward gradient computation.

Gradient precision in the backward pass remains BF16. When gradients flow backward through the graph, the edges of $\mathcal{G}_{\text{bwd}}^{\text{train}}$ carry BF16 tensors. This is a deliberate asymmetry: the forward pass is FP8 for speed, but the backward pass maintains higher precision because gradient underflow and quantization noise are known to be more damaging to convergence than forward-pass quantization noise. The master weights, stored in BF16, are updated with BF16 gradients by the BF16 optimizer states. This is the standard recipe from FP8 pretraining systems like Transformer Engine [55] and COAT [14], and the paper transfers it directly to the RL setting without modification.

GEMM Quantization: Granularity, Layout, and Kernel Design

All linear layer computations in Jet-RL — forward projection (FProp), weight gradient (WGrad), and data gradient (DGrad) — use FP8 tensor cores. The quantization scheme for each follows a consistent pattern built on two granularity choices:

Weight quantization: 128×128 per-block. Each weight matrix $W \in \mathbb{R}^{D \times C}$ (output channels $D$, input channels $C$) is partitioned into 128×128 tiles. Each tile computes its own scaling factor:

$S_W^{(i,j)} = \frac{\max(|W_{\text{tile}(i,j)}|)}{\Delta_{\max}}$

where $W_{\text{tile}(i,j)}$ is the 128×128 submatrix of $W$ starting at row $128i$ and column $128j$, $\max(|\cdot|)$ takes the maximum absolute value within that tile, and $\Delta_{\max} = 448$ is the maximum representable value in the FP8 E4M3 format. The quantized weight is:

$\hat{W}_{\text{tile}(i,j)} = \text{round}\left(\frac{W_{\text{tile}(i,j)}}{S_W^{(i,j)}}\right)$

tile by tile, producing an FP8 weight matrix with per-block scaling factors.

What it computes: For each 128×128 block of the weight matrix independently, the scaling factor is the ratio of the block's maximum absolute value to the FP8 maximum. This means each block uses only as much of the FP8 dynamic range as it needs, rather than being constrained by the global maximum of the entire weight matrix.

Why this granularity: Per-tensor quantization — where a single scaling factor is computed for the entire weight matrix — is known to be unstable for LLM training because weight distributions vary substantially across different parts of the matrix. An outlier in one row can force the scaling factor to be large, crushing smaller values in other rows into underflow. 128×128 per-block quantization localizes the scaling factor so that outliers only affect their immediate neighborhood. The specific tile size of 128 is chosen for hardware efficiency: modern FP8 tensor core instructions operate on matrices with dimensions that are multiples of 128, so tiling at this granularity maps directly to hardware without additional reshaping or padding. The tile size is also small enough to handle weight non-uniformity but large enough that the scaling factor storage overhead is negligible (one FP32 scalar per 128×128 = 16,384 elements).

Activation and gradient quantization: 1×128 per-group. Activation tensors $X \in \mathbb{R}^{N \times C}$ ($N$ tokens, $C$ channels) are quantized with scaling factors computed independently for each group of 128 channels within each token. For token $n$ and channel group starting at $c$:

$S_X^{(n, c)} = \frac{\max(|X[n, c:c+128]|)}{\Delta_{\max}}$

producing scaling factors for every 1×128 slice of the activation tensor.

What it computes: Each token's activation vector is divided into contiguous groups of 128 channels, and each group gets its own scaling factor based on the maximum absolute value within that 128-element segment. A token with length $C$ channels produces $C/128$ scaling factors per token.

Why 1×128 for activations vs. 128×128 for weights: Activations are produced dynamically for each input and vary across tokens (different tokens can have drastically different activation magnitudes), while weights are static for a given model version. The 1-dimension along the token axis (the "1" in 1×128) means each token's activations are quantized independently — a token with large activations gets its own scaling factors and doesn't force all other tokens in the batch to use an unnecessarily large scaling factor. The 128-dimension along the channel axis matches the weight quantization for hardware compatibility: the FP8 tensor core instruction expects the first operand quantized in groups of 128 along the inner dimension (the contraction dimension of the matmul). By using 1×128, both the weight and activation tensors share scaling factors along the 128-contiguous-channel axis, allowing the hardware to efficiently align them during the multiply-accumulate.

DGrad gradient quantization: 128×1 per-group. For the DGrad operator (which computes the gradient with respect to the input activations as $\nabla_X = \nabla_Y \times W$), the gradient tensor $\nabla_Y \in \mathbb{R}^{N \times D}$ must be quantized. However, the WGrad operator also needs $\nabla_Y$, but in a different layout:

• WGrad computes $\nabla_W = \nabla_Y^\top \times X$, which requires $\nabla_Y$ to be the first operand in a column-wise layout with 1×128 quantization.
• DGrad computes $\nabla_X = \nabla_Y \times W$, which requires $\nabla_Y$ to be the first operand in a row-wise layout with 128×1 quantization.

These are incompatible requirements on the same tensor — one kernel wants it quantized as 1×128 (groups along the $D$ dimension), the other wants it quantized as 128×1 (groups along the $N$ dimension). Jet-RL solves this by requantizing $\nabla_Y$ in the backward pass: the gradient is first quantized as 1×128 for WGrad, then requantized as 128×1 for DGrad. Since the weight's quantization (128×128 per-block) is symmetric along the channel and row axes, its value doesn't change during the backward pass — it only needs a transpose. The paper notes that this forced requantization is "even beneficial for quantized training" because it introduces a form of stochastic rounding that can actually improve convergence, similar to how dropout acts as a regularizer.

The Three GEMM Operations and Their Quantization Configuration

Every linear layer in a transformer requires three matrix multiplications, each with specific layout constraints imposed by FP8 tensor core hardware (which expects the first operand row-wise and the second operand column-wise):

FProp (Forward Propagation): $Y = X \times W^\top$, where $Y \in \mathbb{R}^{N \times D}$ is the output, $X \in \mathbb{R}^{N \times C}$ is the input activation, and $W \in \mathbb{R}^{D \times C}$ is the weight.

• Activation quantization: 1×128 per-group, fused with the preceding operator (typically RMSNorm or a residual add). The quantization converts $X$ from BF16 to FP8 right before the GEMM, avoiding an extra memory round-trip.
• Weight quantization: 128×128 per-block. The weight is quantized offline from the BF16 master copy during the weight synchronization step, and the FP8 weight is stored for the duration of the forward pass (or until the next weight update).
• Layout: Both operands row-wise — the FP8 tensor core computes $Y_{\text{fp32}} = X_{\text{fp8}} \times W_{\text{fp8}}^\top$ and accumulates to FP32, which is then converted to BF16 for the output.
• Why this configuration: The 1×128 activation and 128×128 weight share the 128-dimension along the contraction axis ($C$), meaning the hardware can efficiently multiply-accumulate by aligning scaling factors. This is the standard configuration proven in DeepSeek-V3 [20] and optimized in kernels like DeepGEMM [21].

WGrad (Weight Gradient): $\nabla_W = \nabla_Y^\top \times X$, where $\nabla_Y \in \mathbb{R}^{N \times D}$ is the gradient of the loss with respect to the output, and $\nabla_W \in \mathbb{R}^{D \times C}$ is the gradient with respect to the weights.

• First operand quantization ($\nabla_Y^\top$): The transposed gradient is quantized as 1×128. Since $\nabla_Y^\top \in \mathbb{R}^{D \times N}$, each "token" becomes a column of the original gradient, and groups of 128 contiguous rows get their own scaling factor.
• Second operand quantization ($X$): The saved forward activation is already in FP8 with 1×128 quantization (from the FProp step), so it is used directly.
• Layout: The first operand ($\nabla_Y^\top$) is column-wise (the transposition makes the $N$ dimension the inner dimension), and the second operand ($X$) is row-wise. The FP8 tensor core computes in a column-wise × row-wise configuration, which is the natural layout for this GEMM shape.
• Why this configuration: The 1×128 on $\nabla_Y^\top$ groups 128 rows of the $D$ dimension. Since $D$ is typically a multiple of 128 in transformer architectures, this partitioning is clean and maps directly to hardware.

DGrad (Data Gradient): $\nabla_X = \nabla_Y \times W$, where $\nabla_X \in \mathbb{R}^{N \times C}$ is the gradient with respect to the input activations.

• First operand quantization ($\nabla_Y$): Quantized as 128×1 per-group (groups of 128 tokens along the $N$ dimension). This is the requantization step — $\nabla_Y$ was already quantized as 1×128 for WGrad, but DGrad needs the grouping along the other axis.
• Second operand ($W$): The weight is already in FP8 with 128×128 per-block quantization — same as FProp. Since the quantization is symmetric along the $C$ and $D$ axes, no requantization is needed; only a transpose is applied so that the layout is row-wise × row-wise.
• Layout: The first operand is row-wise (row=token, column=channel $D$), the second operand is row-wise (row=channel $D$, column=channel $C$). The FP8 tensor core computes a row-wise × row-wise GEMM by treating the second operand as if transposed.
• Why this configuration: The DGrad operator's workload is structurally identical to FProp — an $N \times D$ matrix multiplied by a $D \times C$ matrix to produce an $N \times C$ matrix. The 128×1 quantization on $\nabla_Y$ groups 128 tokens together, which is appropriate because different tokens in the same batch can have similar gradient statistics (unlike activations, which can vary wildly per token). The 128×128 weight quantization is reused from FProp, avoiding recomputation.

Implementation Details: Frameworks, Kernels, and Weight Synchronization

Jet-RL is implemented as a modification to two existing open-source frameworks, with custom CUDA kernels for the quantization and GEMM operations.

Inference Engine: vLLM. The rollout phase uses vLLM's standard FP8 inference support, which accepts FP8-quantized weights and applies per-tensor (or per-token) activation quantization before each GEMM. For Jet-RL, vLLM is configured to use the same 128×128 per-block weight quantization and 1×128 per-group activation quantization as the training forward pass. The paper quantizes the weights once during the parameter update step and loads them into vLLM via its standard weight loading API.

Training Framework: VeRL with FSDP. VeRL [16] orchestrates the distributed RL training loop, managing the four models (actor, reference, reward, critic) and the data flow between phases. The actor model is sharded across GPUs using Fully Sharded Data Parallelism (FSDP). Jet-RL modifies VeRL's actor training step to use FP8 GEMMs for all linear layers in the forward and backward passes, with the custom quantization kernels inserted before each GEMM invocation.

Quantization Kernels: Triton. The paper implements the per-block and per-group quantization, tensor transposition, and fused RMSNorm+Quantization kernels in Triton [48], a Python-embedded DSL for writing GPU kernels. This choice allows the quantization logic to be expressed at a high level (the tiling and scaling factor computation are explicit in the Triton code) while still compiling to efficient CUDA code. The specific implementations include:

• quantize_1x128: Given a BF16 tensor of shape [N, C], computes the max absolute value for each contiguous group of 128 channels per token, divides by that max and by $\Delta_{\max}$, rounds to the nearest representable FP8 value, and packs the result into FP8 format. The scaling factors are stored as FP32 for use during the FP8 GEMM's rescaling step.
• quantize_128x128: Given a BF16 weight tensor of shape [D, C], partitions it into 128×128 tiles, computes per-tile scaling factors, quantizes each tile, and stores both the FP8 weights and the FP32 scaling factors.
• fused_rmsnorm_quantize: Applies RMSNorm to a BF16 input, then immediately quantizes the output to FP8 using 1×128 per-group scaling. This fusion avoids an intermediate write of the BF16 RMSNorm output to global memory, saving bandwidth.
• requantize_for_dgrad: Takes a $\nabla_Y$ tensor that was previously quantized as 1×128 (for WGrad) and requantizes it as 128×1 (for DGrad). This kernel recomputes scaling factors along the $N$ dimension.

GEMM Kernels: DeepGEMM. The actual FP8 matrix multiplications use kernels from DeepGEMM [21], an open-source library providing highly optimized FP8 GEMM implementations for NVIDIA Hopper GPUs. DeepGEMM implements the (1×128) × (128×128) FP8 matmul for FProp and DGrad, and the (1×128) × (128×1) FP8 matmul for WGrad. These kernels handle the dequantization (multiplying by the per-group and per-block scaling factors) and accumulation to FP32 internally, producing a BF16 or FP32 output. The paper does not modify these kernels; it uses them as provided.

Weight Synchronization and Quantization. A key practical detail concerns when and how the BF16 master weights are converted to FP8. In the RL training loop, the actor model's weights change after every training step. The FP8 weights used for the rollout must reflect the latest actor weights for the rollouts to be on-policy. Jet-RL handles this as follows:

1. After each optimizer step: The actor's BF16 master weights (stored in the FSDP shards) have been updated. Each GPU that holds a weight shard runs quantize_128x128 on its local shard, producing FP8 weights and scaling factors.
2. Weight broadcast to inference engine: The quantized weights and scaling factors are sent from the training GPUs to the inference engine GPUs (which may be the same physical GPUs or different ones, depending on the deployment topology). This transfer uses NCCL or direct memory copy, depending on the configuration.
3. Inference engine weight loading: vLLM loads the FP8 weights and configures its FP8 linear layers to use the provided scaling factors. The next rollout uses these fresh quantized weights.
4. Training forward pass quantization: On the next training step, the forward pass also runs quantize_128x128 on the (now updated) BF16 master weights to produce FP8 weights for the FProp GEMMs. Because the quantization is deterministic (given fixed weights, the scaling factors and rounded FP8 values are always the same), and the weights haven't changed between the post-update quantization and the next forward pass, the FP8 weights used in training forward and inference rollout are identical.

This synchronization happens at every training step, which is feasible because the quantization itself is cheap — it requires one pass over the weight matrix to compute per-tile maxima, then a second pass to apply the scaling and rounding. For an 8B parameter model, this takes on the order of milliseconds per step, negligible compared to the rollout and training computation.

Why This Specific Configuration: Design Rationale Summary

The quantization granularities (128×128 for weights, 1×128 for activations, 128×1 for DGrad input) are not arbitrary — they are chosen to satisfy multiple constraints simultaneously:

1. Hardware efficiency: FP8 tensor cores on H100 GPUs operate on matrices with inner dimensions that are multiples of 128. Tiling at this granularity means no padding or reshaping overhead.
2. Training stability from prior art: The combination of 128×128 block quantization for weights and 1×128 group quantization for activations is proven in large-scale FP8 pretraining (DeepSeek-V3, COAT) to maintain accuracy equivalent to BF16 while providing substantial speedup. Jet-RL inherits this configuration rather than discovering it from scratch.
3. Unified graph requirement: The same granularities must work for both inference (which only needs FProp) and training (which needs FProp, WGrad, and DGrad). The 128×128 weight quantization works for FProp and DGrad (symmetric so transposition doesn't change it), and the activation quantization's asymmetry between FProp (1×128) and WGrad/DGrad (requantized) is handled by the explicit requantization kernel.
4. Memory bandwidth: 1×128 activation quantization computes scaling factors from groups of 128 contiguous values, which is efficient to compute in a single pass (load 128 elements, find max, scale, write). The scaling factor storage overhead is 1 FP32 value per 128 elements — roughly 3% overhead relative to the FP8 data itself — which is acceptable.
5. Numerical accuracy: Finer granularities (e.g., 64×64 or 1×64) would provide even more accurate quantization but at higher scaling factor overhead and with less mature kernel support. Coarser granularities (e.g., per-tensor) are cheaper but led to training instability in prior work. 1×128 is the sweet spot that has been empirically validated.

Summary of What Jet-RL Changes Versus BF16 Training

To make the transformation from standard BF16 RL training to Jet-RL concrete, here is what changes at the operator level for a single transformer block:

Attention Block (unchanged except quantization insertion):

• RMSNorm on input → output quantized to FP8 (1×128) → QKV projection (FP8 GEMM with FP8 weights)
• FlashAttention: runs in BF16 internally (the Q, K, V tensors are upcast from FP8 to BF16 before attention computation, since FlashAttention doesn't have a native FP8 implementation)
• Output projection: input is BF16 from attention → quantized to FP8 (1×128) → OProj (FP8 GEMM with FP8 weights) → output in BF16
• Residual add: BF16 addition of attention output and original input

MLP Block (unchanged except quantization insertion):

• RMSNorm on input → output quantized to FP8 (1×128) → Gate and Up projections (FP8 GEMMs)
• SiLU activation: computed in BF16 (the FP8 gate output is upcast)
• Element-wise multiply: BF16 (gate_output × up_output)
• Down projection: input quantized to FP8 (1×128) → DownProj (FP8 GEMM) → output in BF16
• Residual add: BF16

Backward pass (unchanged topology):

• Gradients flow BF16 through all non-GEMM operations (RMSNorm backward, SiLU backward, element-wise multiply backward, residual adds)
• At each linear layer, the backward pass uses FP8 WGrad and DGrad kernels as described above, with gradient accumulation in FP32 then conversion to BF16
• Master weights receive BF16 gradient updates from the BF16 optimizer (AdamW), and the updated BF16 weights are re-quantized after the step

The key observation is that the architecture, the optimizer, the learning rate schedule, the RL algorithm — none of these change. Jet-RL is not a new training recipe; it's a numerical precision policy that can be applied to any existing RL training configuration. The only thing that changes is which GEMM kernels are called in the forward pass (FP8 instead of BF16) and the backward pass (FP8 instead of BF16 for the three GEMMs), plus the insertion of quantization operators before each GEMM. This makes Jet-RL a drop-in modification to existing RL frameworks, not a from-scratch reimplementation.

4. Key Insights and Innovations

Innovation 1: Reframing the FP8 RL Problem from "Quantization Accuracy" to "Precision Flow Mismatch"

The most fundamental conceptual move in this paper is a diagnostic reframing of why quantized RL training fails. The dominant assumption in the field — embedded in frameworks like VeRL, SLIME, NeMo-RL, and OpenRLHF — was that FP8 rollout acceleration is an inference optimization problem to be solved at the rollout engine level. If the FP8 inference is accurate enough (i.e., its outputs match BF16 closely on a per-token basis), then training with those rollouts should work. This assumption led to an entire research direction focused on better calibration methods, finer quantization granularities, and importance-sampling corrections to patch the off-policy gap — all operating under the premise that training should stay in BF16 and inference should be made more faithful.

Jet-RL argues that this framing is categorically wrong. The failure is not that FP8 inference is insufficiently accurate — it's that the training forward pass and the inference forward pass are computing different functions, period. No amount of inference-side optimization can fix this because the RL update itself is being computed through the wrong function. The graph-theoretic formulation in Section 4 (𝒢_infer vs. 𝒢_train) makes this argument formal: the two graphs have different edge precision annotations, so they represent different numerical programs. The RL algorithm is essentially optimizing the BF16 program using data generated by the FP8 program — a fundamental violation of on-policy RL that cannot be corrected by better quantization of the inference engine alone.

This reframing is significant beyond the specific FP8 solution because it provides a diagnostic lens for evaluating any quantized training strategy. The question becomes not "how accurate is the quantized inference?" but "does the training forward pass compute the same function as the inference forward pass?" If the answer is no, the method will exhibit length-dependent and difficulty-dependent instability regardless of how good the quantization looks on static benchmarks. This explains the paper's otherwise puzzling result that BF16-train-FP8-rollout works at 4K but collapses at 16K (Figure 3): the per-step mismatch is small, but the training graph is optimizing the wrong function, and the cumulative effect of that systematic error destroys convergence once the rollout is long enough for the divergence to compound. The field's prior failure to diagnose this was, in this view, a category error — treating a graph-structural problem as a per-tensor numerical accuracy problem.

The evidence anchoring this framing is not a single ablation but the pattern of failures across Figures 3, 4, and Tables 2–3. The method that "should" work based on the quantization-accuracy view (BF16-train-FP8-rollout) fails systematically under conditions that stress the mismatch (long sequences, hard tasks, weak base models), while the method that should be more aggressive (quantizing training itself) works robustly across all conditions. The reversal of expectations — making training less precise to make it more stable — is the signature of a reframing, not an incremental improvement.

Innovation 2: Unifying Precision Flow as a Design Principle (Not a Quantization Recipe)

The paper's second conceptual contribution is elevating the idea of precision flow unification to a first-class design principle for RL training systems. The specific quantization choices in Jet-RL (128×128 per-block weights, 1×128 per-group activations) are not new — they are inherited from prior work on FP8 pretraining (DeepSeek-V3, COAT, Transformer Engine). What is new is the architectural insistence that the precision flow graph must be identical between training and rollout at every shared edge, enforced as a system-level invariant rather than a per-component optimization.

Prior work treated precision as a local property of each phase: "inference should be fast, so use FP8; training should be accurate, so use BF16." This local optimization ignored the global constraint that RL training requires the two phases to be consistent. Jet-RL inverts this: precision is a global property of the training loop, and the choice of precision for any component is constrained by the requirement that the forward pass graphs match. This is a systems design principle, not a quantization technique — it constrains which quantization schemes are admissible (only those that can be applied identically in both contexts) rather than prescribing a specific scheme.

The significance of this principle extends beyond the FP8 context. Any future work on mixed-precision RL training (FP4 rollout, INT8 training, dynamic precision scaling) must confront the graph-unification constraint. If you cannot make the training forward pass match the inference forward pass, your method will suffer from the same off-policy instability Jet-RL diagnoses, regardless of how clever your quantization is. This turns what might have seemed like a narrow FP8 implementation detail into a necessary condition for any quantized RL system.

The evidence for this principle's power is the robustness of Jet-RL across configurations where BF16-train-FP8-rollout fails: 16K rollout lengths (Table 3), challenging tasks like DeepMATH (Table 3), and base models that haven't been instruction-tuned (Figure 4). The fact that the same unification strategy works across all these failure modes — without per-configuration tuning — suggests it addresses a fundamental constraint, not a happy accident of parameter settings.

Innovation 3: Establishing That On-Policy RL Can Be Maintained Under Aggressive Quantization (Negative Result on BF16-Train-FP8-Rollout)

Jet-RL's most practically significant empirical finding is a decisive negative result: the BF16-train-FP8-rollout strategy that has been widely deployed in production RL frameworks does not work reliably. This is not a small degradation — it's a catastrophic collapse at 16K rollout length (Figure 3, Qwen2.5-7B accuracy drops from converging to collapsing entirely after ~20 steps) and a total failure to converge on challenging configurations (Qwen2.5-7B at 8K in Table 2, Qwen3-8B-Base at 16K in Table 3).

Negative results are undervalued in ML research, but this one is particularly important because it invalidates a deployed practice. Multiple major frameworks (VeRL, SLIME, NeMo-RL, OpenRLHF) have adopted BF16-train-FP8-rollout based on the reasonable but incorrect assumption that FP8 inference is accurate enough. Jet-RL's experiments demonstrate that the assumption holds only under narrow, favorable conditions — short sequences and easy tasks where the model already performs well — and fails precisely in the long-sequence, difficult-task regime where RL training creates the most value. This is a "the treatment works, except for the patients who need it most" situation.

The diagnostic value of this negative result lies in its specificity about failure conditions. The paper doesn't just say "BF16-train-FP8-rollout is unstable"; it characterizes two distinct failure modes (length-dependent collapse via accumulated autoregressive divergence, difficulty-dependent instability via diffuse output distributions) and provides mechanistic hypotheses for each. This specificity allows future work to recognize when their training configuration is at risk — if you're training with rollouts over 8K tokens or on tasks where the base model has low pass@1, BF16-train-FP8-rollout is dangerous regardless of which framework you're using.

The evidence for the severity of this negative result is starkest in Table 3: on Qwen3-8B-Base with 16K rollouts and DeepMATH training data, BF16-train-FP8-rollout suffers a 25.6% absolute degradation on MATH 500 (from 83.4% to 57.8%) compared to BF16 training, while Jet-RL degrades by only 3.2% (83.4% to 80.2%). This is not a tradeoff between speed and accuracy — the BF16-train-FP8-rollout result is unusable, while Jet-RL is a viable accelerator.

Innovation 4: The Graph Subgraph Formalism as a Correctness Criterion

While the graph model 𝒢 = (𝒱, ℰ) in Section 4 is presented as expository, it actually serves as a formal correctness criterion for quantized RL training systems: a quantization scheme is admissible if and only if the inference precision graph is a subgraph of the training forward precision graph with identical edge annotations. This is a conceptual contribution because it converts an empirical question ("does this quantization scheme work for RL?") into a verifiable structural property ("do the graphs match?").

The power of this formalism is that it decouples the quantization design from the RL stability analysis. You don't need to run expensive RL training experiments to determine whether a proposed quantization scheme is structurally sound — you just need to verify that the same quantization operators are inserted at the same positions in both graphs, with the same granularity parameters. If they are, the scheme is on-policy by construction and will not suffer from the graph-mismatch instability. If they aren't, the scheme is off-policy and will exhibit the failure modes the paper documents.

This is a significant advance over the prior state of the art, where the admissibility of a quantization scheme was evaluated empirically and the failure modes were poorly understood. The Truncated Importance Sampling (TIS) approach [47], for instance, implicitly acknowledges the graph mismatch but attempts to correct for it statistically rather than eliminating it structurally. Jet-RL's formalism explains why TIS can only partially succeed: importance sampling corrects for distributional mismatch between the policies, but the function being optimized (the training forward pass) is still fundamentally different from the function that generated the data (the inference forward pass). The correction is applied at the loss level, not at the computation level, so the gradient signal remains off-policy.

The formalism's value is demonstrated not by an ablation (you can't ablate "having the formalism") but by the consistency of its predictions: it correctly predicts that BF16-train-FP8-rollout will fail (graphs differ) and that Jet-RL will succeed (inference graph is a subgraph of training forward graph). It also generates a testable prediction for any future quantized RL scheme: if you design a new quantization approach, the formalism tells you immediately whether it's structurally sound, without requiring a full RL training run to find out.

5. Experimental Analysis

Evaluation Methodology

• Models. The paper evaluates Jet-RL on three model families: Llama3.1-8B, Qwen2.5-7B, and Qwen3-8B-Base. The choice spans different architectures and pretraining recipes, and crucially includes both instruction-tuned (Qwen3-8B) and base (Qwen3-8B-Base) variants to test the method's robustness across model capabilities. Model sizes (7B–8B parameters) represent a practical scale where RL training is computationally feasible for academic research while still exhibiting the rollout bottleneck.

• Datasets and RL Training Configuration. Two dataset settings are used. The primary setting trains on a mixture of GSM8K (8,500 grade-school math word problems) and MATH (12,500 complex competition problems), with the rollout generation number set to 4. A more challenging setting uses the DeepMATH dataset (103K high-difficulty math problems), with the rollout generation number set to 16. The RL algorithm is not explicitly named in the evaluation section, but Section 3's motivation experiments and Section 6's related work mention GRPO (Group Relative Policy Optimization). All experiments use a learning rate of 10⁻⁶, batch size of 256, and KL loss coefficient of 10⁻³. Checkpoints are evaluated every 5 steps during RL training. All experiments run on NVIDIA H100 GPUs.

• Metrics. The paper evaluates the final trained checkpoints on five downstream benchmarks: GSM8K (test split), MATH 500 (test split), AMC, GPQA [50], and SuperGPQA [51]. The primary metric is accuracy (percentage of correct answers) on each benchmark, with an unweighted average across all five benchmarks reported as the summary statistic. For the DeepMATH-trained experiments, GSM8K results are not reported due to a format issue (noted in a table footnote). No metric of training stability (e.g., variance across seeds, success rate of convergence) is systematically reported, which is a notable omission given that stability is the paper's central claim.

• Baselines. Three training precision configurations are compared: (1) BF16 training — the gold-standard baseline where both training forward passes and rollout inference use BF16 precision for all linear layers, representing the upper bound on accuracy at the cost of slower rollouts; (2) BF16-train-FP8-rollout — the widely-adopted strategy (implemented in VeRL, SLIME, NeMo-RL, OpenRLHF) where training forward passes remain in BF16 but rollout inference uses FP8-quantized weights and activations without calibration; (3) Jet-RL — the proposed unified precision flow method where both training forward passes and rollout inference use identical FP8 GEMMs with 128×128 per-block weight quantization and 1×128 per-group activation quantization, while the backward pass and optimizer states remain in BF16.

• Generation Budget / Compute Accounting. The generation budget is implicitly fixed by the rollout length (8K or 16K maximum tokens) and the number of rollouts per prompt (4 for GSM8K+MATH, 16 for DeepMATH). No variable compute budget experiments are conducted — the paper does not study how accuracy scales with the number of rollouts per prompt or with total generation FLOPs. The efficiency evaluation (Table 4) measures speedup in tokens/second for FP8 vs. BF16 inference at fixed batch sizes and input lengths, not in terms of RL training throughput at matched accuracy. End-to-end speedup (1.16× for 8B models) is reported at a single configuration only.

• Cross-Validation / Statistical Protocol. No cross-validation, statistical significance testing, or multiple-seed training runs are reported. Every result in Tables 2 and 3 appears to come from a single training run per configuration, meaning there is no characterization of variance. Given the paper's claim that BF16-train-FP8-rollout exhibits "training instability" and "catastrophic collapse," the absence of error bars or multi-seed replication is a significant methodological gap — it is impossible to determine whether the reported failures are systematic or reflect unlucky random seeds.

Main Quantitative Results

Standard Configuration: 8K Rollout Length on GSM8K + MATH

Table 2 reports the core comparison across three models trained with 8K maximum rollout length on the GSM8K+MATH mixture. The headline findings are:

BF16-train-FP8-rollout fails catastrophically on Qwen2.5-7B. The method "did not converge" — no numerical results are reported, only the text "Did not converge" in the corresponding cells. This is the most severe failure mode observed: the training process completely breaks down rather than merely underperforming. Jet-RL, by contrast, converges robustly and achieves an average score of 55.9% across the five benchmarks, only 1.0% below the BF16 baseline (56.9%). This is a qualitative difference — Jet-RL succeeds where the standard approach fails entirely.

On Llama3.1-8B, BF16-train-FP8-rollout suffers a 10.2% average degradation (from BF16's 23.2% to 13.0%), while Jet-RL actually outperforms BF16 by 2.0% (23.2% to 25.2%). The per-benchmark breakdown reveals that Jet-RL's advantage comes primarily from GPQA (+6.6% over BF16) and SuperGPQA (+4.0% over BF16), while it lags slightly on GSM8K (−1.8%) and MATH 500 (−0.8%). This suggests that the FP8 forward pass may have a regularizing effect that helps generalization to harder benchmarks — an observation the paper does not explore further. BF16-train-FP8-rollout, meanwhile, degrades across the board, with the largest drop on GSM8K (−28.4% absolute: from 49.0% to 20.6%).

On Qwen3-8B-Base, BF16-train-FP8-rollout shows a moderate 2.9% average degradation (from 63.8% to 60.9%), while Jet-RL degrades by only 1.1% (to 62.7%). However, the per-benchmark pattern differs from Llama3.1-8B: BF16-train-FP8-rollout's degradation is concentrated in MATH 500 (−6.3%) and SuperGPQA (−4.8%), while Jet-RL's degradation is spread more evenly (MATH 500: −2.0%, GPQA: −1.6%, with small gains on GSM8K and SuperGPQA). This model-specific variation in failure patterns suggests that the off-policy mismatch affects different model families differently, likely depending on their weight distributions and sensitivity to quantization error.

Overall pattern across 8K experiments: Jet-RL reduces the average performance gap to BF16 training to approximately 1% or less across all three models (degradation of 1.0%, 1.1%, and a gain of 2.0%), compared to BF16-train-FP8-rollout degradations of 10.2%, failure to converge, and 2.9%. The qualitative difference in stability is more striking than the quantitative accuracy difference — Jet-RL converges in all settings, while the baseline fails to converge on one model and severely degrades on another.

Challenging Configuration: 16K Rollout Length

Table 3 reports results under extended rollout lengths (16K tokens), where the off-policy mismatch in BF16-train-FP8-rollout is expected to be most severe due to the autoregressive accumulation of per-step discrepancies.

Qwen3-8B-Base at 16K: BF16-train-FP8-rollout fails to converge entirely (noted as "Did not converge" in Table 3), replicating the Qwen2.5-7B failure from the 8K setting but now on a different model under longer sequence conditions. Jet-RL converges with a 2.7% average degradation (from 64.0% to 61.3%). The degradation is concentrated in MATH 500 (−5.0%) and GSM8K (−2.6%), with small changes on GPQA (−2.9%) and SuperGPQA (−0.3%). This is a substantially larger gap than Jet-RL's 1.1% degradation on the same model at 8K, suggesting that even unified precision flow is not entirely immune to the challenges of longer sequences — the quantization error within a single forward pass may begin to accumulate across the forward pass layers even when training and inference match.

Qwen2.5-7B at 16K: BF16-train-FP8-rollout converges but with significant degradation. Unlike at 8K where it failed entirely, at 16K the Qwen2.5-7B model with BF16-train-FP8-rollout achieves a 53.7% average — 5.0% below the BF16 baseline (58.7%). The failure is most pronounced on SuperGPQA (−15.3% absolute: from 42.2% to 26.9%) and MATH 500 (−4.7%). Jet-RL reduces this degradation to 3.0% (55.7% average), with the SuperGPQA gap remaining large (−14.4%) but improvements on GSM8K (−2.2% vs. −0.8%) and MATH 500 (−3.6% vs. −4.7%). The fact that BF16-train-FP8-rollout converges at 16K but failed at 8K on this model is surprising and not explained in the paper — it may reflect a different random seed or hyperparameter interaction, but without multi-seed experiments this cannot be determined.

Challenging Configuration: DeepMATH Training Data

The final row of Table 3 tests the most difficult setting: Qwen3-8B-Base trained on the DeepMATH dataset (103K high-difficulty math problems) with 16K maximum rollout length and 16 rollouts per prompt. GSM8K results are not reported due to a format issue.

BF16-train-FP8-rollout suffers a 10.3% average degradation (from BF16's 54.6% to 44.3%). The damage is catastrophic on MATH 500: a 25.6% absolute drop from 83.4% to 57.8%. GPQA degrades by 1.6% and SuperGPQA by 3.5%. Jet-RL reduces the average gap to 0.9% (from 54.6% to 53.7%), with MATH 500 degrading by only 3.2% (to 80.2%), GPQA improving by 3.0% (to 47.2%), and SuperGPQA degrading by 2.3% (to 33.8%).

This is the paper's strongest result: under the most challenging training configuration — long sequences, difficult math, many rollouts per prompt — the naive FP8 rollout strategy produces a model that loses a quarter of its accuracy on the primary benchmark, while Jet-RL produces a model that is essentially indistinguishable from BF16 training. The 0.9% average gap is within what could reasonably be attributed to noise or seed variation. This configuration is also the most representative of frontier RL training for reasoning models, making Jet-RL's robustness here particularly compelling evidence for its practical value.

Efficiency Evaluation: Rollout and End-to-End Speedup

Table 4 reports FP8 inference speedup over BF16 across different model sizes, tensor parallelism (TP) degrees, and output sequence lengths (4K, 8K, 16K), measured in tokens/second using vLLM on H100 GPUs with 512 prompts, maximum 128 concurrent requests, and 512 input tokens.

Speedup increases with model size. For the 8B model at TP=1, FP8 achieves 1.10×–1.12× speedup across output lengths. For the 14B model at TP=1, speedup increases to 1.26×–1.29×. For the 32B model at TP=2, speedup reaches 1.29×–1.33×. This trend reflects the fact that larger models are more compute-bound: the FP8 tensor cores provide proportionally more benefit when computation dominates over memory access and communication overhead.

Higher tensor parallelism reduces speedup. For the 14B model, increasing from TP=1 (1.26×–1.29×) to TP=2 (1.08×–1.12×) approximately halves the observed speedup, as communication overhead between GPUs dilutes the per-GPU compute savings. Similarly, for the 32B model, TP=2 achieves 1.29×–1.33× while TP=4 drops to 1.07×–1.10×.

End-to-end speedup for 8B training. For the specific configuration of Qwen3-8B with 8K rollout length, the paper reports that FP8 achieves a 1.54× speedup in the actor update phase, a 1.80× speedup in reference model inference, a combined 1.41× training phase throughput improvement, and a 1.16× end-to-end step-time speedup. The gap between the 1.41× training phase speedup and the 1.16× end-to-end speedup reflects that the rollout phase (which achieves only ~1.10× speedup at 8B scale) is the dominant fraction of total time when sequence lengths are long. The paper explicitly notes that "we expect the speedup to be much more significant for larger model sizes" but does not report end-to-end speedups for 14B or 32B models, citing resource constraints.

Ablation Studies and Robustness Checks

The paper does not contain a dedicated ablation study section. The experimental analysis in Section 5 tests Jet-RL against baselines across varying conditions (different models, rollout lengths, and datasets), but these are best understood as robustness evaluations rather than component ablations — they test whether the method works under different configurations, not which design choices contribute how much to the result.

No ablation on quantization granularity. The paper uses 128×128 per-block for weights and 1×128 per-group for activations throughout, citing prior work (DeepSeek-V3, COAT) as justification. There is no comparison to coarser granularities (e.g., per-tensor, 1×64) or finer granularities (e.g., 64×64 blocks) to determine whether the specific choice matters for RL training stability or whether any reasonably fine granularity would suffice.

No ablation on which GEMMs are quantized. The paper quantizes all three GEMMs in every linear layer (FProp, WGrad, DGrad). There is no experiment keeping WGrad or DGrad in BF16 to determine whether backward pass quantization is necessary for stability or whether unifying only the forward pass would be sufficient. This is a significant gap because the backward pass quantization introduces additional complexity (the requantization for DGrad) and potential sources of training noise.

No ablation on saved activation precision. The paper stores saved activations in FP8 (rather than BF16) for the backward pass, citing prior work on pretraining. There is no comparison to storing BF16 activations — which would increase memory usage but might improve gradient accuracy. The impact of this design choice on RL training specifically (as opposed to pretraining, where it has been validated) is untested.

No ablation on the importance of matching activation quantization between training and inference. Jet-RL's defining characteristic is unified precision flow, but all experiments compare the full Jet-RL system against the full BF16-train-FP8-rollout baseline. There is no intermediate condition where, for example, the training forward pass uses FP8 but with different quantization granularity than inference, or where only weight quantization is unified but activation quantization differs. Without such ablations, it is impossible to attribute Jet-RL's success specifically to the graph-unification principle versus to simply doing FP8 training (which could be more robust than BF16 training with FP8 rollouts for reasons unrelated to graph matching).

Implicit ablation on calibration. The BF16-train-FP8-rollout baseline uses no calibration (direct BF16-to-FP8 casting), which is the approach adopted by SLIME and NeMo-RL. The paper does not test whether adding calibration to the baseline would close the gap — a critical omission, because the paper's claim that "calibration is too slow" (Section 3.2) justifies not using it, but doesn't test whether calibration would actually fix the off-policy problem. If calibration does fix it, then the practical recommendation would be "use calibrated FP8 rollouts" rather than "unify precision flow," and the speedup comparison would need to account for calibration cost.

Negative result on model-specific sensitivity. The results across Tables 2 and 3 reveal substantial variation in how different models respond to quantization: Qwen2.5-7B fails to converge with BF16-train-FP8-rollout at 8K (Table 2) but converges at 16K with only 5.0% degradation (Table 3), while Qwen3-8B-Base converges at 8K with 2.9% degradation (Table 2) but fails entirely at 16K (Table 3). This inconsistency is not explained or investigated — it may reflect genuine model-specific quantization sensitivity, or it may be noise from single-seed experiments. Without replication, the reliability of these specific failure thresholds is unknown.

Critical Assessment

Central Claim: BF16-train-FP8-rollout causes training instability and catastrophic collapse under long rollouts and challenging tasks.

This claim is the empirical foundation of the paper's motivation, and the evidence is strong but incomplete. The catastrophic failure modes are dramatic and well-documented: Qwen2.5-7B fails to converge at 8K (Table 2), Qwen3-8B-Base fails to converge at 16K (Table 3), and the 25.6% MATH 500 drop under DeepMATH training (Table 3) is stark. The training curves in Figures 3 and 4 visually confirm that BF16-train-FP8-rollout diverges from BF16 training as sequence length increases.

However, three qualifications are necessary. First, single-seed results are the norm — there is no evidence that the catastrophic collapses are systematic rather than seed-dependent. RL training is known to be high-variance, and a single unlucky seed can produce a training run that appears to "collapse" when the method is actually fine on average. The paper's qualitative language ("catastrophic collapse," "did not converge") implies a systematic property of the method, but the experimental design does not support that inference.

Second, the failure is not universal even within the paper's own results. Qwen3-8B (the instruction-tuned variant, Figure 4, left panel) trains successfully with BF16-train-FP8-rollout and even appears to converge faster than BF16 training. This means the claim must be qualified: BF16-train-FP8-rollout fails when the model is weak on the task (base models, hard datasets) or when sequences are long, but succeeds when the model is already strong. This is precisely the paper's argument in Section 3.3, but the qualification matters for practical adoption — if you are fine-tuning an already-capable reasoning model on moderately difficult problems with short rollouts, BF16-train-FP8-rollout might work perfectly well, and Jet-RL's additional complexity (modifying training forward passes) would be unnecessary.

Third, the DeepMATH result's 25.6% MATH 500 drop is the most compelling single datapoint, but GSM8K results are missing from that configuration due to an unexplained "format issue." Since GSM8K is an easier benchmark where BF16-train-FP8-rollout often performs better, the absence potentially biases the average degradation metric. The paper would be stronger if it explained and resolved this format issue.

Overall, the claim that BF16-train-FP8-rollout can fail dramatically is well-supported by multiple examples; the claim that it always or inherently fails under long/challenging conditions is not established without multi-seed replication.

Central Claim: Jet-RL's unified precision flow resolves the off-policy mismatch and enables stable FP8 RL training.

The evidence that Jet-RL converges in all tested configurations while BF16-train-FP8-rollout fails in several is clear from Tables 2 and 3. Jet-RL never fails to converge and consistently achieves accuracy within ~1–3% of BF16 training across all models, datasets, and rollout lengths.

The limitation is that the paper does not demonstrate a causal link between the specific mechanism (unified precision flow / graph matching) and the improved stability. The possibility that FP8 training with BF16 rollouts would also be stable (i.e., the benefit comes from using FP8 in training regardless of graph matching) is not ruled out. The possibility that simply quantizing weight updates (making the model numerically closer to its FP8 counterpart even when training in BF16) would suffice is not tested. Jet-RL is evaluated as a complete system, and the specific contribution of the graph-unification principle is not isolated.

This is a significant inferential gap because it affects what lesson the field should take from the paper. If the lesson is "do FP8 training for RL," then the implementation recipe is simpler than Jet-RL (just swap in FP8 GEMMs for the training forward pass and accept whatever FP8 inference engine you have, without worrying about matching granularities). If the lesson is "unify the precision flow graphs," then the implementation is more constrained (you must ensure your inference engine and training forward pass use identical quantization granularity at every layer). The paper's experiments cannot distinguish these interpretations.

Central Claim: Jet-RL achieves substantial speedup while maintaining accuracy close to BF16 baselines.

The speedup claims have three components, each with different levels of support:

Rollout speedup (up to 1.33×): Well-supported by Table 4, which provides systematic measurements across model sizes and TP configurations. The measurement methodology is standard (vLLM throughput benchmark). However, these are standalone inference benchmarks, not measurements of rollout throughput within an actual RL training loop. The RL loop includes weight synchronization overhead, KV cache management, and coordination between training and inference engines, any of which could reduce the effective rollout speedup. The paper does not report rollout throughput within the full RL training pipeline.

Training phase speedup (up to 1.41×): Reported as a single number for Qwen3-8B at 8K rollout length, with a brief breakdown (1.54× actor update, 1.80× reference model inference). These numbers are not systematically measured across model sizes or configurations — the paper states that "a full scaling study on 14–32B models is left to future work given resource constraints." This is a substantial gap because the training phase speedup is where Jet-RL should excel relative to BF16-train-FP8-rollout (which only accelerates rollout, not training). Without systematic measurements, it is unclear whether 1.41× is typical, optimistic, or conservative.

End-to-end speedup (1.16×): This is the bottom-line number for practitioners, and it is modest. A 1.16× speedup means that a 24-hour training run becomes roughly 20.7 hours — a meaningful but not transformative improvement. The paper acknowledges that the end-to-end speedup is limited by the rollout phase dominating total time and achieving relatively low speedup (1.10×) at the 8B scale. The claim that larger models will see greater speedup is plausible (based on the 1.33× rollout speedup at 32B in Table 4) but untested in an end-to-end RL pipeline.

Central Claim: Jet-RL maintains robust convergence across all settings and exhibits negligible accuracy degradation.

"Negligible accuracy degradation" is operationalized as "approximately 1%" in the paper's abstract and conclusion. The actual results are:

• Llama3.1-8B at 8K: Jet-RL outperforms BF16 by 2.0% (Table 2). This is not degradation — it's an improvement. Whether this is a real effect of FP8 providing beneficial regularization, or noise from single-seed evaluation, is unclear.
• Qwen2.5-7B at 8K: 1.0% degradation (Table 2). Consistent with the claim.
• Qwen3-8B-Base at 8K: 1.1% degradation (Table 2). Consistent.
• Qwen2.5-7B at 16K: 3.0% degradation (Table 3). Higher than claimed.
• Qwen3-8B-Base at 16K: 2.7% degradation (Table 3). Higher.
• Qwen3-8B-Base DeepMATH at 16K: 0.9% degradation (Table 3). Consistent.

The "~1%" figure accurately describes the 8K results and the DeepMATH result but understates the degradation at 16K on Qwen models (2.7–3.0%), which is non-negligible and may matter for applications where small accuracy differences are critical. The paper would be more precise to claim "1–3% degradation depending on configuration," which is still a strong result compared to the 10–25% degradation and convergence failures of the baseline.

Missing Experiments That Would Strengthen the Paper

Multi-seed training runs to characterize variance in convergence and final accuracy. Without these, it is impossible to distinguish systematic effects from noise, and the dramatic "did not converge" results remain anecdotal.

Ablation of the graph-unification claim: Compare Jet-RL against an intermediate condition where both training and rollout use FP8 forward passes but with different quantization granularities (e.g., 128×128 for training, per-tensor for rollout). If this configuration also works, the unification principle is less central than claimed.

Calibrated FP8 rollout baseline: Test whether adding even a cheap calibration step (e.g., using a single calibration prompt, or calibrating once every N steps rather than every step) to BF16-train-FP8-rollout closes the gap. This would determine whether the problem is calibration cost (which could be optimized) or the fundamental graph mismatch (which cannot be fixed by better quantization alone).

Systematic end-to-end speedup measurements: Report training throughput (tokens/second or steps/hour) for Jet-RL vs. BF16 training at 14B and 32B scales, even if only for a subset of configurations. The claim that FP8 RL training is efficient rests on extrapolation from the 8B measurement and the standalone inference benchmarks.

Testing on non-math reasoning tasks: All experiments use math benchmarks (GSM8K, MATH, AMC, GPQA-diamond, SuperGPQA). The paper's findings about difficulty-dependent failure (Figure 4) would be more generalizable if tested on code generation, logical reasoning, or scientific QA tasks where the model's prior competence similarly varies.

A direct measurement of the off-policy gap: The paper hypothesizes that BF16-train-FP8-rollout fails because the training and rollout distributions diverge. This could be directly measured by comparing the KL divergence between FP8 rollout trajectories and BF16 training forward-pass log-probabilities as a function of sequence length, which would directly validate the accumulation hypothesis in Section 3.3. No such measurement is reported.

6. Limitations and Trade-offs

The Difficulty Estimation Cost Is Not Accounted For in the Speedup Claims

The assumption or constraint: Jet-RL's unified precision flow requires that the FP8 weights used during training forward passes be identical to those used during rollout. This means the BF16 master weights must be quantized to FP8 (using 128×128 per-block quantization) at every training step — once for the inference engine (rollout) and once for the training forward pass. The paper acknowledges this quantization cost implicitly in Section 4.3's implementation description but never quantifies it or includes it in the speedup measurements. The efficiency evaluation in Table 4 measures standalone inference throughput (tokens/second) on vLLM, and the end-to-end speedup measurement (1.16× for 8B models) is reported without a breakdown of how much time is spent on weight quantization and synchronization between the training and inference engines.

The consequence: In a production RL training loop, the weight quantization step requires scanning the entire weight matrix to compute per-tile maximum absolute values, then applying scaling and rounding. For an 8B model with ~8 billion parameters, this is a non-trivial operation performed at every training step. If each quantization pass takes even 100–200 milliseconds, and training steps occur every few seconds, the quantization overhead could consume 5–10% of the per-step time — directly eating into the 1.16× end-to-end speedup. More seriously, the weight synchronization between training GPUs and inference GPUs (which may be different physical devices or different memory spaces on the same device) adds communication overhead that is not present in BF16 training (where inference can directly use the BF16 weights without transformation). The paper's claim that quantization "takes on the order of milliseconds per step" (from our technical analysis in Section 3.4) is asserted but never measured or validated, and the actual cost will depend on model size, GPU topology, and whether the quantization computation can be overlapped with other work.

What evidence exists in the paper: None. The paper provides no measurement of per-step quantization latency, no measurement of weight synchronization overhead, and no profiling breakdown of the full RL training loop with Jet-RL (unlike Figure 2, which profiles BF16 training but not FP8 training). The end-to-end speedup is reported as a single scalar (1.16×) without any decomposition into compute, quantization, and communication components. Without this evidence, practitioners cannot determine whether the 1.16× speedup holds in their deployment topology (e.g., when training and inference GPUs are on different nodes with slower interconnect) or whether the overhead grows to dominate the speedup at larger model scales.

Mitigation status: Not addressed. The paper does not acknowledge this as a limitation, does not propose optimizations to reduce quantization overhead (e.g., caching scaling factors when weights change slowly, overlapping quantization with other computation, using faster max-reduction kernels), and does not suggest future work to characterize or reduce this cost. This is a notable gap because the entire practical value proposition of Jet-RL — that it accelerates RL training — depends on the quantization overhead being small relative to the compute savings. A practitioner attempting to deploy Jet-RL would need to measure this overhead themselves before committing to the approach.

The "Unified Precision Flow" Causal Mechanism Is Not Isolated

The assumption or constraint: The paper's central claim is that BF16-train-FP8-rollout fails because the training forward precision graph (𝒢_fwd_train) differs from the inference precision graph (𝒢_infer), creating an off-policy mismatch. Jet-RL's solution is to make the inference graph a subgraph of the training forward graph by quantizing both to FP8 with identical granularities. This is presented as a causal mechanism: the graph mismatch causes the instability, and unifying the graphs eliminates it.

However, Jet-RL differs from the BF16-train-FP8-rollout baseline along multiple axes simultaneously, not just graph unification. Specifically, Jet-RL: (1) uses FP8 in the training forward pass (instead of BF16), (2) quantizes WGrad and DGrad in the backward pass to FP8 (the baseline does no backward quantization), (3) stores saved activations in FP8 (the baseline stores them in BF16), and (4) uses 128×128 per-block weight quantization and 1×128 per-group activation quantization (the baseline uses unspecified per-tensor or per-token quantization during inference). Any of these changes — separately or in combination — could contribute to improved stability, regardless of whether the precision graphs actually match.

The consequence: Without an ablation that isolates the graph-unification mechanism, the paper's core contribution — the "unified precision flow" design principle — remains a plausible hypothesis rather than a validated insight. Consider these alternative explanations that are equally consistent with the data:

1. FP8 training itself is more numerically stable than BF16 training with FP8 rollouts because the training optimizer sees the exact same quantization noise during forward passes that the rollout produces, learning to be robust to it. This would work even if the inference engine used a different FP8 quantization scheme, because the training has already adapted to being quantized.

2. The specific granularity choices (128×128 blocks, 1×128 groups) are what matter, not the graph matching. If the BF16-train-FP8-rollout baseline used these same granularities during inference, it might also be stable. The baseline's failure could be caused by using coarse per-tensor quantization rather than by the graph mismatch.

3. Backward pass quantization provides regularization that stabilizes RL training independent of the forward pass unification. This is plausible because quantized gradients are known to have a noise-injection effect similar to weight noise regularization.

If any of these alternative mechanisms explains Jet-RL's success, then the implementation prescription for practitioners is different from what the paper recommends. If explanation (1) is correct, you should do FP8 training forward passes but don't need to carefully match inference quantization. If explanation (2) is correct, you could keep training in BF16 but use fine-grained FP8 inference. If explanation (3) is correct, backward pass quantization is the key ingredient, not forward pass unification. The paper's experiments cannot distinguish among these.

What evidence exists in the paper: There is no ablation that varies the graph-matching property while holding other factors constant. The only comparison is Jet-RL (unified FP8 everywhere) vs. BF16-train-FP8-rollout (BF16 training, FP8 rollout). Missing intermediate conditions include:

• Jet-RL training forward pass with a different FP8 quantization granularity than inference (e.g., 128×128 training, per-tensor inference). If this also works, graph matching is unnecessary.
• BF16 training with the same fine-grained FP8 quantization used during inference that Jet-RL uses. If this works, the failure of BF16-train-FP8-rollout was a granularity problem, not a graph problem.
• Jet-RL with BF16 backward pass (no WGrad/DGrad quantization). If this also works, backward quantization is unnecessary for stability.

None of these conditions appear in the paper. The absence of any ablation on the defining mechanism is the single most significant methodological gap.

Mitigation status: Not addressed. The paper does not acknowledge that the causal attribution is untested, does not discuss alternative explanations for Jet-RL's stability, and does not propose the ablations described above as future work. The graph-theoretic formalism in Section 4 is presented as the definitive explanation of why Jet-RL works, but it is not empirically validated against competing hypotheses.

All Results Are Reported From Single Training Runs With No Variance Characterization

The assumption or constraint: Every accuracy number in Tables 2 and 3, and every training curve in Figures 3 and 4, is based on a single training run per configuration. The paper reports no standard deviations, no confidence intervals, no minimum/maximum across seeds, and no statement about how many seeds were run. Section 5.1's evaluation setup describes models, datasets, and hyperparameters but makes no mention of replication.

The consequence: RL training is stochastic along multiple dimensions: model initialization, data sampling order, rollout trajectory sampling (via temperature), advantage estimation noise, and optimizer dynamics. This means the "accuracy" after training is properly characterized as a distribution, not a point estimate. Two runs with identical hyperparameters but different random seeds can produce final accuracies that differ by several percentage points — and in RL, they can produce qualitatively different outcomes (convergence vs. divergence).

The paper's central claims depend on comparing accuracy numbers across precision configurations. A claim that "Jet-RL degrades by only 1.1% compared to BF16" (Table 2, Qwen3-8B-Base at 8K) cannot be evaluated without knowing whether the 1.1% difference is larger or smaller than the run-to-run variance. If the standard deviation across seeds is 3%, then a 1.1% difference is statistically indistinguishable from zero — Jet-RL might be better or worse than BF16 depending on the seed. Conversely, if the standard deviation is 0.2%, then 1.1% is a real but small degradation.

The problem is most acute for the paper's most dramatic claims: that BF16-train-FP8-rollout "did not converge" on Qwen2.5-7B at 8K (Table 2) and Qwen3-8B-Base at 16K (Table 3). RL training can fail to converge for many reasons, including a single unlucky rollout that produces a destructive gradient update — a phenomenon that may resolve on a different seed. The paper's framing implies that "failure to converge" is an inherent property of BF16-train-FP8-rollout, but without replication, it is equally consistent with a method that converges 70% of the time and the reported run happened to hit the 30% failure case. The distinction matters enormously for a practitioner deciding whether to adopt Jet-RL: if BF16-train-FP8-rollout fails 30% of the time, you might still prefer it (with restarts) over Jet-RL if Jet-RL's implementation complexity is high; if it fails 100% of the time, you have no choice.

Similarly, the finding that Jet-RL outperforms BF16 training on Llama3.1-8B by 2.0% (Table 2, average 25.2% vs. 23.2%) is an intriguing result that the paper does not explore. If replicated, it would suggest FP8 training provides a beneficial regularization effect for RL — a significant finding in its own right. If it is within-seed noise, it's a red herring. Without replication, the reader cannot evaluate this claim.

What evidence exists in the paper: Zero. There are no error bars on any figure, no ± values in any table, and no text discussion of run-to-run variability. The training curves in Figures 3 and 4 show single lines per condition, with no shaded regions or multi-seed overlays. This is a departure from standard practice in empirical ML research, where at minimum 3–5 seeds are expected for RL experiments, and from the field's growing emphasis on reproducible and statistically grounded claims.

Mitigation status: Not addressed. The paper does not mention this limitation, does not justify why single-seed results are sufficient, and does not suggest multi-seed replication as future work. For a paper whose primary contribution is establishing the robustness and stability of a training method, the absence of variance characterization is a fundamental credibility weakness. A practitioner reading this paper cannot determine whether Jet-RL's claimed stability advantage over BF16-train-FP8-rollout would hold in their setting or was an artifact of the specific seeds used.

The Scaling Study Is Insufficient to Support Claims About Larger Models

The assumption or constraint: The paper makes several forward-looking claims about Jet-RL's applicability to larger models. The abstract states Jet-RL achieves "up to 1.33× rollout phase speedup for 32B model," the conclusion claims it "establishes a reliable and efficient path forward for applying FP8 computation to accelerate large-scale RL training," and Section 5.3 states "we expect the speedup to be much more significant for larger model sizes." These claims imply that Jet-RL's benefits generalize to the model scales (70B, 405B, and beyond) where RL training costs are most acute and where acceleration would be most impactful.

However, the actual experimental support for large-model performance is thin. The rollout speedup numbers in Table 4 include measurements at 14B and 32B, but these are standalone inference benchmarks on vLLM — they measure FP8 vs. BF16 generation throughput in isolation, not within an RL training loop. The end-to-end RL training speedup is measured only at 8B (Qwen3-8B, 1.16×). All accuracy evaluations (Tables 2 and 3) are conducted exclusively on 7B–8B parameter models. The paper explicitly acknowledges this gap: "A full scaling study on 14–32B models is left to future work given resource constraints." But it then proceeds to make claims about "large-scale RL training" based on the 8B results plus inference-only benchmarks.

The consequence: The extrapolation from 8B accuracy results to larger models is not necessarily valid for several reasons:

• Quantization sensitivity can increase with model size. Larger models can have more extreme weight outliers and activation spikes, making FP8 quantization more lossy. The fact that 8B models can be quantized to FP8 with ~1% accuracy degradation does not guarantee that 70B models will show the same tolerance. Prior work on FP8 pretraining (e.g., DeepSeek-V3, COAT) had to carefully tune quantization granularities and occasionally fall back to BF16 for specific layers precisely because larger models exhibit more pathological numerical behaviors.

• The relative speedup from FP8 depends on model architecture and parallelism strategy, not just parameter count. The Table 4 results show that tensor parallelism degree dramatically affects speedup (32B at TP=2 achieves 1.33×, but at TP=4 drops to 1.10×). Large models typically require higher TP degrees for memory reasons, which would push the speedup toward the lower end of the range. A 70B model requiring TP=8 might see much less than the "up to 1.33×" figure cited for 32B.

• The weight quantization and synchronization overhead likely scales with model size. For an 8B model, quantizing weights and synchronizing across GPUs may be negligible. For a 405B model with weights distributed across hundreds of GPUs, the quantization computation (scanning each weight shard for per-tile maxima) and the all-gather communication to distribute FP8 weights to all inference engines could become a significant fraction of per-step time. The paper provides no scaling analysis of this overhead.

• Convergence behavior at scale is unknown. RL training dynamics can change qualitatively with model size — larger models may be more sensitive to gradient noise from backward pass quantization, or may exhibit different exploration-exploitation dynamics that interact with FP8's reduced numerical precision. The paper's evidence that Jet-RL converges robustly at 7–8B is reassuring but does not guarantee the same at 70B.

What evidence exists in the paper: A single end-to-end throughput measurement at 8B (1.16× speedup) plus standalone inference benchmarks at 14B and 32B (1.07×–1.33× speedup). No accuracy results above 8B. No end-to-end throughput measurements above 8B. No analysis of how weight synchronization overhead scales. No experiments on models with expert parallelism (MoE architectures like Mixtral or DeepSeek-V3), which are common at scale and introduce different quantization challenges (expert routing, uneven token distribution).

Mitigation status: Partially acknowledged. The paper explicitly states that the scaling study is left to future work due to resource constraints. This is an honest disclosure of scope. However, the paper's abstract and conclusion contain unqualified claims about "large-scale RL training" and "up to 1.33× rollout phase speedup for 32B model" that are not supported by end-to-end RL training experiments at those scales. A more accurate characterization would be: "Jet-RL achieves 1.16× end-to-end speedup at 8B scale with ~1% accuracy degradation; standalone inference benchmarks suggest 1.33× rollout speedup is possible at 32B, but end-to-end RL training results and accuracy impacts at larger scales remain to be validated."

All Experiments Use Only Math Reasoning Tasks Under a Single RL Algorithm

The assumption or constraint: Every experiment in the paper — from the motivation figures (Figures 3 and 4) through the main accuracy tables (Tables 2 and 3) to the efficiency measurements — uses math reasoning datasets: GSM8K, MATH, DeepMATH, AMC, GPQA, and SuperGPQA. The RL algorithm, while not named consistently in Section 5, is described in Section 3's motivation as GRPO (Group Relative Policy Optimization) and is used throughout. The models tested are all dense transformer architectures (Llama3.1, Qwen2.5, Qwen3).

The consequence: The paper's diagnostic framework and proposed solution are domain-agnostic in their formulation — the graph mismatch problem arises from numerical precision differences in the forward pass, which exist regardless of what the model is being trained to do. However, the empirical validation is entirely within a narrow domain: math word problems and competition problems, trained with GRPO, on dense models. We cannot conclude from the paper's evidence that Jet-RL's benefits generalize to:

• Code generation tasks (e.g., HumanEval, MBPP, SWE-bench) where the reward signal comes from unit tests or execution feedback, and where correct solutions may require very different reasoning structures than math proofs.
• Open-ended reasoning (e.g., scientific reasoning, multi-hop QA, debate) where correctness is harder to verify automatically and reward models may introduce their own biases.
• RLHF-style alignment training where the reward model is a learned preference model rather than a rule-based verifier, and where the rollout length may be shorter but the reward signal is noisier.
• Different RL algorithms (PPO with a critic model, REINFORCE variants, DPO-based methods) that may have different sensitivities to the on-policy/off-policy distinction. PPO's clipping mechanism, for instance, is designed to tolerate some off-policy data, which could make it more robust to the precision mismatch than GRPO.
• MoE architectures where the routing decisions introduce additional nondeterminism and where the expert weight matrices have different sparsity patterns that could interact with per-block quantization.
• Multi-modal models where different modalities (text, vision, audio) have different quantization sensitivities and where the forward pass includes additional operators (cross-attention, vision encoders) not covered by the paper's GEMM-centric quantization scheme.

The paper's finding that BF16-train-FP8-rollout is more stable when the model is already strong on the task (Figure 4, Qwen3-8B vs. Qwen3-8B-Base) is particularly important here because it suggests the failure mode is task-dependent. Math reasoning with base models is a specific point in a broader space of (task, model) difficulty combinations. A code generation RL pipeline where the base model already has strong coding ability might behave like the Qwen3-8B curve (FP8 rollout works fine), while a scientific reasoning task on a weaker base model might exhibit catastrophic collapse like the Qwen3-8B-Base curve. The paper provides no way to predict which tasks will fall into which regime without running the experiment.

What evidence exists in the paper: None outside math reasoning. The benchmarks evaluated (GSM8K, MATH 500, AMC, GPQA, SuperGPQA) are all math or science QA with verifiable answers. The paper does not discuss domain generalizability as a limitation or propose extension to other task families. This narrow evaluation is understandable given the focus on reasoning models — the RL-for-reasoning paradigm is currently most mature in math — but it significantly limits the claims the paper can make about Jet-RL as a general RL training framework.

Mitigation status: Not addressed. The paper's title refers to "FP8 Reinforcement Learning" without qualification, and the abstract claims the method "achieves robust and stable RL training" — phrasing that implies domain-generality. The experiments section does not frame math reasoning as a case study or acknowledge that other domains remain untested. A more accurate scope claim would be: "We demonstrate Jet-RL's effectiveness on math reasoning tasks trained with GRPO; extension to other domains and RL algorithms is an important direction for future work."

The Practical Deployment Story Ignores Hardware Heterogeneity and Latency Constraints

The assumption or constraint: Jet-RL's implementation assumes a specific hardware configuration: NVIDIA H100 GPUs with FP8 tensor core support, and a two-system architecture where an inference engine (vLLM) handles rollout while a training framework (FSDP via VeRL) handles updates. The weight synchronization step that is central to maintaining on-policy consistency requires transferring quantized weights from training GPUs to inference GPUs at every step. The paper implicitly assumes this transfer is fast enough to not dominate step time and that both the training and inference systems have FP8-capable hardware.

The consequence: In practice, RL training deployments can span a wide range of hardware configurations, many of which are not as favorable to Jet-RL as the paper's setup:

• No FP8 support on older GPUs: NVIDIA A100, V100, and earlier GPUs do not have native FP8 tensor cores. Jet-RL cannot be used on these GPUs at all — the FP8 GEMM kernels require Hopper architecture or newer. For organizations running RL training on A100 clusters (still widely deployed), Jet-RL provides zero benefit. The paper does not discuss hardware compatibility or provide a fallback strategy for non-Hopper GPUs.

• Training and inference on different GPU types: In many production RL pipelines, the rollout phase may be offloaded to cheaper or older GPUs with less memory (since inference is less memory-intensive than training), while training uses the latest GPUs. If the inference GPUs lack FP8 support, Jet-RL's unified precision flow cannot be maintained — the inference engine physically cannot run FP8 GEMMs, so the precision graphs will differ by necessity. The paper's approach assumes homogeneous FP8-capable hardware across all GPUs in the training loop.

• Latency-sensitive deployments: The paper measures speedup in terms of throughput (tokens/second) and step time, but does not consider latency — the wall-clock time to complete a single rollout for a single prompt. Sequential dependency chains (generate rollout → evaluate → update → synchronize weights → generate next rollout) mean that even if total throughput improves by 1.16×, the latency for any individual training step might not improve, or might even degrade if weight synchronization adds a serialized step. This matters for interactive or time-bounded RL training scenarios where you need to complete a certain number of steps within a fixed wall-clock budget.

• Multi-node communication overhead: The weight synchronization step broadcasts quantized weights from training GPUs to inference GPUs. In a multi-node setup where training and inference are on different physical machines, this communication goes over the network (InfiniBand or Ethernet) rather than GPU-to-GPU interconnects (NVLink/NVSwitch). Network bandwidth is typically an order of magnitude slower than intra-node GPU bandwidth, potentially making weight synchronization the bottleneck rather than the computation itself. The paper does not characterize Jet-RL's performance under multi-node configurations with network-separated training and inference.

• Dynamic batching and variable-length rollouts: The paper's efficiency measurements use fixed input length (512 tokens), fixed output lengths (4K, 8K, 16K), and fixed concurrent requests (128). Real RL training involves variable-length prompts and rollouts, dynamic batching by the inference engine, and queueing effects that can change which phase is the bottleneck. Jet-RL's speedup under realistic variable-length batching with vLLM's continuous batching scheduler is not measured.

What evidence exists in the paper: The speedup measurements in Table 4 are all on H100 GPUs with specified tensor parallelism degrees. There are no measurements on A100 or other architectures, no multi-node benchmarks (the TP configurations imply single-node multi-GPU), and no latency measurements. The paper does not discuss hardware requirements, network topology assumptions, or fallback strategies for non-FP8 hardware.

Mitigation status: Not addressed. The paper does not discuss hardware compatibility as a limitation, does not characterize the communication pattern or bandwidth requirements of weight synchronization, and does not provide guidance for practitioners on what hardware configurations are sufficient to realize the claimed speedups. The assumption that FP8-capable GPUs are universally available is optimistic for the current (early 2026) deployment landscape; many production RL training clusters are still predominantly A100-based. The paper would benefit from a clear statement of hardware prerequisites and a discussion of the practical barriers to adoption in heterogeneous or non-Hopper clusters.

7. Implications and Future Directions

How This Work Changes the Landscape

Jet-RL reshapes the conversation around quantized RL training by recasting what appeared to be a straightforward inference optimization problem — "make FP8 rollout fast enough and accurate enough" — into a systems architecture problem about precision flow consistency. This is not an incremental improvement in quantization technique (the specific granularity choices are inherited from prior work on FP8 pretraining) but rather a diagnostic reframing that changes what practitioners should be optimizing for. The paper's central finding — that BF16-train-FP8-rollout, the strategy adopted by major RL frameworks including VeRL, SLIME, NeMo-RL, and OpenRLHF, is fundamentally unstable in the exact long-sequence regime where RL training matters most — has immediate negative practical implications for anyone currently using those frameworks with FP8 inference configured.

The conceptual shift is clearest in the paper's graph-theoretic formulation in Section 4. Prior work treated quantization as a per-phase local optimization: "inference should use FP8 for speed, training should use BF16 for accuracy." This local reasoning ignored the global constraint that RL algorithms require the training forward pass and the rollout forward pass to compute the same function. By explicitly modeling the precision flow as a directed graph $\mathcal{G} = (\mathcal{V}, \mathcal{E})$ and showing that BF16-train-FP8-rollout creates two distinct forward graphs ($\mathcal{G}_{\text{fwd}}^{\text{train}}$ with all BF16 edges vs. $\mathcal{G}_{\text{infer}}$ with FP8 edges at linear layers), Jet-RL provides a formal correctness criterion that was previously missing: any admissible quantized RL system must have the inference precision graph as a subgraph of the training forward precision graph with identical edge annotations. This converts what was an empirical question ("does this quantization scheme work for RL?") into a verifiable structural property ("do the graphs match?").

The importance of this reframing is that it resolves a genuine contradiction in the field's practical experience. Framework developers observed that BF16-train-FP8-rollout "worked" in many cases (which is true — it works fine at 4K rollout lengths on tasks where the model is already strong, as Figure 4 shows), leading them to deploy it broadly. But researchers attempting more ambitious RL training (longer sequences, harder tasks) encountered mysterious instability, which they may have attributed to RL's inherent noise or to specific hyperparameter choices. Jet-RL's experiments demonstrate that both observations are correct under their respective conditions: BF16-train-FP8-rollout is stable when the off-policy divergence hasn't had time to compound (short sequences) and when the model's output distribution is peaked enough to be robust to small numerical perturbations (easy tasks with strong models), but it becomes catastrophically unstable when these conditions are violated. This provides a unified explanation that should prevent future work from drawing overly general conclusions from narrow experiments.

The reframing also changes which research directions appear promising. Under the quantization-accuracy view, the natural path was to improve FP8 inference fidelity: better calibration methods, finer granularities, adaptive scaling factors, and post-hoc correction of the off-policy gap via importance sampling. Jet-RL's results suggest these directions are fundamentally limited — better inference quantization can reduce the per-step mismatch but cannot eliminate the underlying fact that $\mathcal{G}_{\text{fwd}}^{\text{train}}$ and $\mathcal{G}_{\text{infer}}$ are different graphs performing different computations. The accumulation hypothesis (Section 3.3) implies that even a tiny per-step divergence will eventually compound over thousands of tokens to create off-policy data, and importance sampling weights with product-of-ratios structure will explode or vanish for long trajectories regardless of how close each per-step ratio is to 1.0.

Instead, Jet-RL redirects attention toward precision-unified system design: how to make the training forward pass match the inference forward pass at every shared edge. This opens a design space that was previously underexplored because practitioners assumed training "should" stay in BF16. Future systems work on mixed-precision RL training — FP4 rollout, dynamic precision scaling, or quantization-aware training with different forward/backward precisions — must now satisfy the graph-unification constraint to avoid the instability Jet-RL diagnoses. This makes Jet-RL not just a method but a necessary condition for any quantized RL system to be reliable at scale.

The practical impact of this reframing is disproportionately large relative to the technical novelty of the components because it addresses a deployed failure mode. Multiple production-grade RL frameworks (VeRL, SLIME, NeMo-RL, OpenRLHF) have adopted a training strategy that Jet-RL shows is unreliable under conditions that are increasingly common as the field pushes toward longer reasoning traces. The paper's finding that BF16-train-FP8-rollout suffers a catastrophic 25.6% absolute MATH 500 degradation on DeepMATH training (Table 3, from 83.4% with BF16 to 57.8% with BF16-train-FP8-rollout, while Jet-RL maintains 80.2%) is not a footnote — it means that organizations training reasoning models with those frameworks may be unknowingly sacrificing a quarter of their model's accuracy on the primary benchmark. Even if Jet-RL's 1.16× end-to-end speedup is modest, the value of avoiding such catastrophic degradation makes the approach essential for any long-sequence RL training pipeline.

That said, the work's impact is bounded by its scope: all experiments are on 7–8B dense models trained on math reasoning tasks with GRPO. The paper does not demonstrate the graph-unification principle in code generation, scientific reasoning, RLHF alignment, MoE architectures, or multi-modal models, nor does it validate end-to-end speedups at the 32B+ scale where the rollout bottleneck is most economically significant. The conceptual contribution — that precision flow unification is necessary for stable quantized RL — is argued through a single family of experiments in a single domain and must be replicated more broadly before it can be considered a universal design principle. The paper's most important legacy may be to invalidate BF16-train-FP8-rollout as a default strategy and to establish that any new quantized RL system must demonstrate, not assume, that its training and inference graphs match.

Follow-Up Research This Work Enables

Establishing whether graph unification or FP8 training robustness is the operative mechanism through systematic ablation. Jet-RL's central claim is that unifying the precision flow graphs eliminates off-policy mismatch. But Jet-RL's stability could equally be explained by FP8 training acting as a regularizer (the optimizer learns to be robust to the quantization noise it encounters during training forward passes, which matches the noise in inference — even if the specific quantization granularities differ). A definitive experiment would compare three conditions: (1) Jet-RL with matching granularities (128×128 training, 128×128 inference), (2) Jet-RL with mismatched granularities (128×128 training, per-tensor inference), and (3) the BF16-train-FP8-rollout baseline. If condition (2) also converges robustly, then FP8 training — not graph unification — is the key mechanism, and practitioners can use whatever FP8 inference scheme they want as long as training forward passes are also FP8. Conversely, if only condition (1) succeeds, the graph-unification principle is validated, and inference engines must be carefully configured to match training quantization. This experiment would resolve the paper's central causal ambiguity and determine the implementation burden for adopters.

Measuring the off-policy divergence directly to validate the accumulation hypothesis. The paper hypothesizes that BF16-train-FP8-rollout fails because the KL divergence between the BF16 training trajectory distribution and the FP8 rollout trajectory distribution grows with sequence length (Section 3.3). This hypothesis is plausible but never directly tested — it is inferred from the training dynamics, not measured. A targeted experiment would sample hundreds of rollout trajectories from both the BF16 training forward pass and the FP8 inference engine (using the same prompts and same weights), then compute the empirical KL divergence or total variation distance between the resulting logit distributions at each token position, averaging across trajectories. Plotting this divergence as a function of token position $t$ for different maximum rollout lengths (1K, 4K, 8K, 16K) would directly confirm whether the divergence grows roughly linearly with $t$ (as the accumulation hypothesis predicts) and whether the growth rate correlates with observed training instability. For a strong model on easy tasks (where BF16-train-FP8-rollout works, per Figure 4 left panel), the same measurement should show negligible per-step divergence even at long lengths, which would validate the paper's explanation for why some configurations are robust. This experiment would convert the paper's mechanistic claim from inference to evidence.

Extending unified precision flow to actor-critic architectures and the critic model. Jet-RL's experiments all use GRPO, a critic-free RL algorithm that estimates advantages from group-level reward comparisons. The paper does not address how unified precision flow interacts with the critic model, which is a core component of PPO-style RL training. In actor-critic RL, the critic model estimates value functions and is typically updated from the same rollout data as the actor. If the critic sees rollouts generated by FP8 inference but its own training forward pass runs in BF16, it faces the same off-policy mismatch as the actor — its value estimates are trained on data from a different precision graph than what it evaluates. A natural experiment would be to train a critic model under three conditions: (1) BF16 training with FP8 rollouts (naive), (2) BF16 training with BF16 rollouts (oracle, but not accelerated), and (3) Jet-RL applied to both actor and critic (unified FP8 forward passes for both models, with the critic's value head potentially kept in BF16 if it is more quantization-sensitive). Comparing the variance of the critic's value estimates and the resulting advantage estimation quality across conditions would reveal whether unified precision flow is equally important for critic training or whether critics are more robust to the mismatch (since they don't generate autoregressive trajectories). If critics prove more robust, a practical recommendation might be: use Jet-RL for the actor but BF16 for the critic, reducing implementation complexity.

Validation on code generation RL benchmarks with execution-based rewards. The paper's experiments are entirely within math reasoning, where the reward signal comes from string matching against ground-truth answers. Code generation RL — where rewards come from executing the generated code against unit tests — presents a meaningfully different setting: the space of "correct" solutions is larger and more diverse (many different programs can pass the same tests), the reward signal is intrinsically more sparse (execution either passes or fails, with no partial credit), and the base model's initial competence on coding tasks may differ from its math competence. A strong follow-up would replicate Jet-RL's experimental protocol on a code generation benchmark suite (e.g., HumanEval, MBPP, LiveCodeBench) training with a GRPO-style algorithm using execution-based rewards, varying maximum rollout length from 1K to 8K tokens (coding solutions can be verbose), and comparing Jet-RL against BF16-train-FP8-rollout and BF16 baselines. The key question is whether the difficulty-dependent failure mode observed in math (Figure 4 — easy tasks are robust, hard tasks fail) generalizes: does BF16-train-FP8-rollout work for coding tasks that are "easy" for the base model (e.g., simple string manipulation) and fail for "hard" tasks (e.g., complex algorithm implementation)? If so, the paper's diagnostic framework applies broadly. If code generation proves uniformly robust or uniformly fragile regardless of difficulty, the mechanism may be specific to math reasoning or to the structure of reward signals in that domain.

Scaling end-to-end speedup measurements to 32B and 70B models in full RL training loops. The paper's efficiency evaluation has a critical gap: all end-to-end RL training measurements are at 8B scale (1.16× speedup), while the most dramatic rollout speedups (1.33× at 32B) are from standalone inference benchmarks. For the practical community — where RL training of 32B–70B reasoning models is the frontier — the relevant number is end-to-end throughput at those scales, including weight synchronization overhead, communication between training and inference engines, and any load imbalance effects. A targeted follow-up would run full GRPO training (or a representative rollout-update cycle) at 32B on H100 clusters, measuring total step time for Jet-RL vs. BF16 training along with a detailed latency breakdown: weight quantization time, weight transfer time (training GPU → inference GPU), rollout generation time, evaluation time (reference/reward/critic forward passes), and training update time. This would determine whether the 1.33× rollout speedup from Table 4 translates to meaningful end-to-end gains or is diluted by overhead. For 70B models requiring tensor parallelism, the experiment should also test whether the communication overhead from higher TP degrees (which reduced speedup from 1.33× at TP=2 to 1.10× at TP=4 in Table 4) can be mitigated by using a different parallelism strategy (e.g., pipeline parallelism for training with TP=2 for rollout). Without these measurements, the paper's claims about "large-scale RL training" remain aspirational.

Stress-testing Jet-RL on MoE architectures with expert-level quantization. Modern frontier LLMs increasingly use mixture-of-experts (MoE) architectures (DeepSeek-V3, Mixtral, Qwen2.5-MoE), where each transformer block routes tokens to a subset of expert FFN layers. This architecture introduces quantization challenges not present in dense models: the expert weight matrices are sparsely activated (only a fraction of experts receive tokens at each step), the routing decisions themselves may be affected by quantization noise (a small perturbation could route a token to a different expert), and the per-expert token batch sizes are smaller (potentially degrading the accuracy of 1×128 per-group activation quantization when only a handful of tokens go to a given expert). A critical stress test for Jet-RL would train an 8B–12B MoE model (e.g., Qwen2.5-7B-MoE or a similar dense-to-MoE upcycled model) on MATH with GRPO, comparing Jet-RL against BF16-train-FP8-rollout at 8K and 16K rollout lengths. The specific question is whether the routing nondeterminism from FP8 quantization creates an additional source of off-policy divergence (beyond the per-token logit differences that drive the dense model failures), or whether Jet-RL's unified precision flow handles it without modification. If MoE models prove more fragile under quantization than dense models, Jet-RL may need architecture-specific adaptations — for example, keeping the router computation in BF16 while quantizing only the expert FFN layers, or using different quantization granularities for experts that receive very few tokens.

Practical Applications and Downstream Use Cases

Training long-context reasoning models for competition math and science. The most immediate application, directly motivated by the paper's experiments, is RL training pipelines for models like DeepSeek-R1 or Qwen3 that generate 8K–16K token chain-of-thought traces to solve competition-level math and science problems. The paper's Table 3 (DeepMATH training with 16K rollouts) shows this exact scenario: with BF16-train-FP8-rollout, MATH 500 accuracy collapses from 83.4% to 57.8%, while Jet-RL maintains 80.2%. For organizations training reasoning models at the frontier — where a few percentage points on MATH 500 can be the difference between state-of-the-art and mediocre — the choice is not "BF16 vs. Jet-RL" but "Jet-RL vs. no FP8 acceleration at all," because the alternative FP8 strategy produces unusable models. The 1.16× end-to-end speedup at 8B scale, even if modest, compounds over weeks-long training runs to shave days off the training schedule. If the speedup scales to the 1.3× range for 32B models (as the standalone inference benchmarks suggest), a month-long training run becomes roughly 23 days — a meaningful acceleration for research teams iterating on RL training recipes.

FP8 inference in production RL fine-tuning pipelines where BF16-train-FP8-rollout would silently degrade performance. The paper identifies a dangerous scenario: organizations using frameworks like VeRL or NeMo-RL with FP8 rollout enabled may be fine-tuning reasoning models on moderately long sequences (4K–8K) without realizing they are trading accuracy for speed. A team fine-tuning Qwen2.5-7B on MATH with 8K rollouts and BF16-train-FP8-rollout, per Table 2, will see the training curve progressing normally (no error messages, decreasing loss) but produce a final model that lags 5–10% behind what BF16 training would achieve — a silent failure, not a crash. The paper's diagnostic framework provides a concrete check: if your rollout length exceeds ~4K tokens and your base model is not already strong on the task, your FP8 rollout strategy is probably undermining your results. Jet-RL provides the fix for teams that want to keep using FP8 acceleration without this hidden cost. The 1.41× training phase speedup (actor update + reference model inference) is particularly relevant here because it accelerates the non-rollout components that BF16-train-FP8-rollout leaves untouched, providing additional value even for teams whose rollouts are not the bottleneck.

Batch inference pipelines for generating high-quality training data. Many self-improvement and distillation pipelines generate millions of reasoning traces from a teacher model to train a student model (STaR, ReST, rejection sampling fine-tuning). These pipelines run inference at massive scale — often generating hundreds of thousands of long CoT rollouts from the same model checkpoint — making inference throughput the dominant cost. Jet-RL's value proposition here is indirect but significant: if the teacher model was trained with Jet-RL's unified precision flow, its FP8 inference outputs are exactly what its training optimized for, so there is no quality degradation from using FP8 inference at scale. By contrast, a teacher model trained with BF16-train-FP8-rollout would have been optimized through a mismatched training graph, and its FP8 inference quality may be substantively worse than expected — a problem that batch inference pipelines would discover only after generating millions of rollouts and training the student model on them. Jet-RL eliminates this uncertainty by guaranteeing that the FP8 forward pass is what the model was trained to optimize. The 1.33× inference speedup at 32B (Table 4) directly reduces the cost of generating these datasets.

On-policy RL training in resource-constrained academic settings. The paper's experiments at 7–8B scale are directly relevant to academic researchers who want to study RL training for reasoning but lack access to large-scale GPU clusters. For these groups, the rollout phase is often the bottleneck not because of absolute throughput but because limited GPU memory and compute force tradeoffs between batch size, sequence length, and training frequency. Jet-RL's unified precision flow enables FP8 training forward passes, which reduce memory usage (FP8 activations are half the size of BF16), potentially allowing larger batch sizes or longer sequences within the same memory budget. The paper does not measure memory savings (a notable omission), but prior work on FP8 training (COAT, Transformer Engine) consistently demonstrates 30–50% activation memory reduction. For an academic lab training Qwen2.5-7B on 4×H100 GPUs, this could mean fitting 8K instead of 4K rollouts in memory, or increasing the batch size from 128 to 256 — both of which can meaningfully affect RL training dynamics and final model quality. The accuracy preservation (~1% degradation) makes this a practical tradeoff for research prototyping, even if the absolute speedup is modest at this scale.