BEAM: Binary Expert Activation Masking for Dynamic Routing in MoE

ArXiv: 2605.14438

🎯 Pitch

BEAM learns to dynamically deactivate redundant experts on a per-token basis, slashing MoE computation by up to 85% while keeping over 98% of model performance—all without any architecture changes. This simple, plug-and-play mask router eliminates a fundamental waste in large language models, yielding 2.5× faster decoding by skipping computations for easy tokens that standard Top-K routing wastes power on.

1. Executive Summary

This paper introduces BEAM (Binary Expert Activation Masking), a method that learns token-adaptive expert selection in Mixture-of-Experts models by training a lightweight mask router to generate binary masks that selectively deactivate redundant experts from the standard Top-K candidate set — decoupling sparsity control from the primary routing mechanism. Evaluated on three MoE architectures (Qwen1.5-MoE-A2.7B, DeepSeekV2-Lite, and Qwen3-30B-A3B) after supervised fine-tuning on the Tulu 3 SFT Mixture Dataset, BEAM retains over 98% of the original model's performance while reducing MoE layer FLOPs by up to 85%, yielding up to 2.5× faster decoding and 1.4× higher throughput (achieving 0.56 average activated experts at β = 1.0 on Qwen1.5 while still retaining 85% of baseline accuracy). The paper establishes that token-adaptive expert sparsity can be achieved through a plug-and-play mask router without architectural modification, but only when sparsification is decoupled from expert selection — avoiding the gradient conflicts, position bias, and minimum activation floors that limit prior logit-modification and null-expert approaches.

2. Context and Motivation

The Core Problem: Fixed Top-K Routing Wastes Computation

Mixture-of-Experts (MoE) models have become a dominant architecture for scaling large language models efficiently. The key insight behind MoE is conditional computation: rather than activating all parameters for every input token, each token is routed to only a small subset of specialized "expert" feed-forward networks. This allows models to grow to enormous parameter counts (hundreds of billions or trillions) while keeping the per-token computation roughly constant — in principle.

The dominant routing mechanism, dating back to Shazeer et al. (2017) and Lepikhin et al. (2020), is fixed Top-K routing: for every token, the model computes a set of routing logits (one per expert), applies a softmax, and activates only the K experts with the highest logits. The output is a weighted sum of those K expert outputs.

The problem the paper identifies is deceptively simple: Top-K routing treats every token identically, regardless of its complexity. Whether the token is the word "the" in a boilerplate system prompt or a critical mathematical operator in a reasoning problem, exactly K experts fire. This is wasteful in two directions:

• Simple tokens are over-served: tokens with minimal informational content (punctuation, common function words, chat template boilerplate like "You are a helpful assistant") receive the same computational budget as semantically rich tokens that genuinely require expert processing.
• The uniform budget creates a ceiling on inference speed: since every token activates K experts, the FLOPs per MoE layer are fixed regardless of the input. There is no mechanism to reduce computation for the large fraction of easy tokens in a typical sequence.

This matters enormously in practice. MoE models are attractive precisely because they promise sub-linear compute scaling with parameter count, but if K is fixed, the inference cost per token is rigid. As Section 1 notes, "this inefficiency ultimately limits the potential for faster MoE model inference." With MoE models increasingly deployed in latency-sensitive and cost-sensitive settings (chat applications, API services, edge deployment), unlocking the ability to dynamically reduce expert activation per token translates directly to reduced serving costs, higher throughput, and lower latency.

Why Prior Approaches Fall Short

The paper categorizes existing attempts to address this problem into three families, each with fundamental limitations that motivate BEAM's design.

Routing Logits Modification: Fail to Skip High-Weight Experts and Enforce Minimum Activation Floors

These methods modify the router's output logits to allow the number of activated experts to vary per token. MoE-Dynamic (Huang et al., 2024) and XMoE (Yang et al., 2024b) activate experts sequentially until the cumulative softmax probability exceeds a threshold ϕ. Adaptive Gating (Li et al., 2023b) and NAEE (Lu et al., 2024) dynamically choose between Top-1 and Top-2 based on the gap between the top two logits. DA-MoE (Aghdam et al., 2024) uses attention scores to allocate a dynamic K.

The paper identifies two critical failure modes in this family:

First, these methods cannot skip redundant high-weight experts. They implicitly assume that routing rank — the position of an expert in the sorted logit list — is a proxy for importance: they always activate the highest-weight experts first and only consider skipping lower-ranked ones. But the paper empirically demonstrates (Section 5.2, Figure 6b, and Appendix B.4) that this assumption is wrong. In BEAM's masking patterns, the probability of masking Top-1 experts is 0.43, while the probability of masking Top-8 experts is only 0.53 — a remarkably flat distribution. The mask router frequently prunes high-ranked experts and retains low-ranked ones when doing so serves the task. Logit-based methods are architecturally incapable of expressing this pattern because they process experts in sorted order.

Second, these methods enforce a minimum activation of 1 expert per token. Cumulative-probability approaches always activate at least the highest-weight expert. The paper emphasizes this constraint as a fundamental limit on achievable sparsity: "fail to skip redundant high-weight experts, and require at least one active expert, preventing acceleration" (Section 2). As the results show, BEAM routinely achieves average activated expert counts well below 1 (e.g., 0.56 on Qwen1.5, 0.11 at extreme sparsity), indicating that for many tokens, no routed expert is needed at all. Logit-based methods fundamentally cannot express this zero-activation case.

Special Experts (Null Experts): Indirect Control, Extra Hyperparameters, and Passive Sparsity

These methods add special "zero-computation" experts to the expert pool. AdaMoE (Zeng et al., 2024) introduces null experts that output zeros. LongCat (Gui et al., 2025) uses experts that return the input unchanged (identity mapping). MoE++ (Jin et al., 2024) extends this with three types of zero-computation experts.

The paper identifies a subtle but important limitation: these approaches achieve sparsity indirectly and passively. The router must learn to send tokens to null experts rather than real experts, but sparsity is not explicitly optimized — it is a side effect of the router's decision-making. This creates several problems:

• Null experts consume routing capacity: they compete with real experts for Top-K slots, meaning that even when a null expert is selected, it occupies a position that could have gone to a genuinely useful expert. This can degrade model quality because the router must trade off sparsity against expertise.
• Extra hyperparameters complicate deployment: the number of null experts must be tuned per model and per sparsity target. Table 1 shows AdaMoE with 60 null experts (Qwen1.5, mid sparsity) and 120 null experts (high sparsity) — these are architecture-specific and must be selected manually.
• Sparsity cannot be directly controlled: there is no explicit mechanism to target a specific activation count. The router decides when to use null experts, and the resulting sparsity is emergent rather than steerable.

The paper frames this as a plug-and-play usability problem. For a method to be practically deployable, practitioners need simple, predictable control over the sparsity-accuracy tradeoff. BEAM's single β hyperparameter provides this (increasing β monotonically increases sparsity with gradual accuracy loss, as shown across Tables 1–3), while null-expert approaches require adjusting the null expert count and retraining.

Static Expert Merging and Pruning: Cannot Adapt to Token-Level Complexity

These training-free post-hoc methods reduce MoE computation by merging similar experts or pruning less-important ones. DEK (Zhang et al., 2025) groups experts by feature similarity and merges within groups. EEP (Liu et al., 2024b) uses evolutionary search to find optimal pruning and merging patterns. HC-SMoE (Chen et al., 2025) applies hierarchical clustering to expert outputs.

The limitation here is fundamental: these methods are static. Once applied, the same reduced expert set serves every token regardless of its complexity. A semantically complex token that genuinely requires a diverse set of expert perspectives gets the same compressed model as a punctuation token. The paper notes that these methods "cannot adapt to the varying complexity of input tokens at inference time and often suffer performance degradation under high compression" (Section 2). This is an architectural mismatch — the entire premise of MoE is conditional computation, yet static pruning removes the conditionality.

How This Paper Positions Itself: Three Key Architectural Insights

BEAM is positioned as a synthesis that addresses the failure modes of all three prior families simultaneously. The paper's positioning rests on three architectural insights:

1. Decoupling sparsification from expert selection. This is the central design principle (Section 3.2, Step 1 vs. Steps 2–3). The primary router continues to handle expert selection and load balancing — it computes the Top-K set exactly as in standard MoE training. The mask router operates on top of the Top-K output, determining only which of those already-selected experts to keep. This separation avoids the gradient conflicts that the paper argues plague unified approaches: "these methods entangle expert selection, load balancing, and sparsity control in a single router, creating inherent gradient conflicts, thereby degrading model capacity" (Section 3.1). The theoretical analysis in Section 3.4 formalizes this: the load-balancing loss L_bal produces no gradients for the mask router (Theorem 3.2), meaning the mask router optimizes sparsity without being pulled toward uniform expert utilization.

2. Active, explicit sparsity through binary masking. Rather than relying on cumulative probability thresholds (passive, sorted-order constrained) or null-expert routing (passive, competitive), BEAM makes sparsity an explicit optimization objective via the L_reg regularization term (Equation 10). The mask router learns to output binary decisions (keep/prune) for each expert in the Top-K set, with the L1 norm of the mask encouraging fewer activations. The straight-through estimator (STE) makes this discrete decision trainable with standard backpropagation. This is fundamentally different from prior approaches: sparsity is a direct target of optimization, not a side effect.

3. Plug-and-play deployability. The paper emphasizes that BEAM "requires minimal code changes" (Section 1) — specifically, two kernel modifications in vLLM's MoE pipeline (Appendix A.4). There is no architectural modification to the base model beyond adding the mask router parameters, which are initialized to zero so training starts from the original Top-K behavior. This contrasts with DynMoE (which completely replaces the router and collapses performance, as shown in Appendix B.3) and AdaMoE (which requires adding null experts to the model architecture). The paper's framing as "practical, plug-and-play" is supported by the vLLM integration showing real speedups (Figure 4), not just theoretical FLOPs reduction.

The Difficulty Landscape: Why This Problem Is Hard

The paper's positioning also implicitly addresses why this problem has resisted straightforward solutions. There is a fundamental tension:

• During training, the model needs to explore different expert combinations to learn good routing. Fixed Top-K provides stable gradient flow to all selected experts and enables load-balancing losses that prevent expert collapse.
• During inference, many of those experts are redundant for specific tokens, but identifying which ones requires per-token decisions that cannot be made with the same router that was trained for uniform Top-K.

Prior methods that modify routing logits at inference (Top-K Pruning, MoE-Dynamic) break this by applying a new decision rule to weights trained under a different objective. Prior methods that retrain with a different routing scheme (Top-K Reduced, DynMoE) lose the benefits of the original routing structure. BEAM's solution is to keep the original router and add a new component trained specifically for the sparsification task, decoupling the two objectives. The training dynamics shown in Figure 7 (Appendix B.2) confirm this works: the language modeling loss converges to the standard SFT baseline (gray dashed line) while the expert active rate drops from near 100% to a stable plateau within the first ~0.5 epoch, indicating that the mask router learns sparsity without destabilizing the base model's routing.

Summary of the Gap BEAM Fills

The paper's central claim is that BEAM achieves what no prior method achieves simultaneously: the ability to prune any expert from the Top-K set (including the highest-weight ones), the ability to activate zero routed experts when appropriate (reducing the MoE layer to shared-expert-only or residual-path computation), direct and predictable control over the sparsity-accuracy tradeoff through a single hyperparameter β, and deployability without architectural changes to the base model. The experimental results — particularly BEAM's dominance at extreme sparsity levels where prior methods fail catastrophically (e.g., Top-K Pruning K=1 on Qwen3 drops from 81.41 to 11.92 average accuracy while BEAM at K=0.56 achieves 71.91) — are designed to validate this positioning.

3. Technical Approach

3.1 Reader Orientation

BEAM is a learned gating system that sits on top of a standard Mixture-of-Experts transformer and decides, for each individual token, which of the already-selected Top-K experts are actually necessary — and which can be safely skipped. The problem it solves is computational waste: standard MoE models activate exactly K experts for every token regardless of whether the token is a semantically rich verb or boilerplate punctuation, and BEAM learns to suppress the redundant ones via a trainable binary mask, reducing per-token FLOPs without modifying the base model architecture.

3.2 Big-Picture Architecture (Diagram in Words)

The system has four components that operate on each token through every MoE layer:

1. Primary Router (R): The standard MoE router — unchanged from the pretrained model — computes logits for all N experts, applies Top-K to select the K highest-scoring candidates, and produces softmax-normalized routing weights g. Its job remains expert selection and (via the load-balancing loss) ensuring uniform expert utilization during training.

2. Mask Router (new, lightweight): A small linear layer parameterized by W_m that takes the same token embedding x as input and produces a raw mask vector m̂ of length N via a sigmoid activation. This raw mask is then binarized at a fixed threshold τ = 0.5 to produce m ∈ {0,1}^N — a hard keep/prune decision for each expert.

3. Masking Operation: The binary mask m is applied element-wise to the Top-K weights: ĝ = g ⊙ m. Experts where m_i = 0 are dropped regardless of their routing weight, and their contribution to the layer output is zero. The number of active experts can range from 0 (all masked) to K (none masked).

4. vLLM Kernel Integration (deployment): A custom CUDA kernel modification intercepts the expert assignment after masking, writing -1 as the expert index for masked experts and having the alignment kernel skip those entries — so masked experts incur zero computation at inference time, not just zero weight.

Information flow: token embedding x → primary router produces Top-K weights g / mask router produces binary mask m → element-wise product ĝ = g ⊙ m → only unmasked experts compute their FFN outputs → weighted sum produces layer output y → residual connection adds to the hidden state.

3.3 Roadmap for the Deep Dive

• First, the standard MoE formulation (Section 3.1) and the paper's diagnosis of why existing dynamic routing methods fail — because understanding the failure modes motivates every design choice in BEAM.
• Second, the four-step BEAM mechanism (Section 3.2) — how the mask router is constructed, how binarization works, and how masking integrates into the MoE computation — since this is the core technical contribution.
• Third, the training strategy (Section 3.3): the straight-through estimator that makes binary decisions differentiable, the initialization scheme that preserves original behavior, and the three-term loss function — because these are what make the mask router learnable and the sparsity-accuracy tradeoff controllable.
• Fourth, the theoretical analysis (Section 3.4) that formalizes which experts receive gradient signals and how the hyperparameter β controls the pruning decision — since this explains why the method works and what the training dynamics actually optimize.
• Fifth, the vLLM deployment implementation (Appendix A.4) and the zero-activation behavior (Appendix A.3) — because the practical speedups depend on turning masked experts into actual skipped computation, and understanding edge cases (all experts masked) matters for correctness.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a method paper whose core idea is that token-adaptive expert sparsity can be achieved by introducing a separate, lightweight mask router that learns to suppress redundant experts from the Top-K set, decoupling the sparsification objective from the expert selection and load-balancing objectives that the primary router handles.

Standard MoE Formulation and the Diagnosis of Prior Approaches (Section 3.1)

The paper begins by formalizing the standard MoE computation to establish notation and identify the specific limitations that BEAM addresses. An MoE layer replaces a dense feed-forward network with N expert networks E_1, ..., E_N and a router R. Given an input token x ∈ R^{d_h} where d_h is the hidden dimension, the router computes logits r = R(x) ∈ R^N, which are converted into routing weights via softmax.

Under standard Top-K routing, only the K experts with the largest logits are activated. The Top-K operator retains the K largest values in r and sets the remaining entries to -∞ (so they become zero after softmax), yielding:

$g_i = \text{Softmax}(\text{Top-K}(r))_i$

where g_i > 0 only for the top K experts and ∑_{i=1}^N g_i = 1. The MoE output is then a weighted sum of the selected expert outputs:

$y = \sum_{i=1}^N g_i \cdot E_i(x)$

Each expert E_i typically follows a Gated Linear Unit (GLU) structure:

$E_i(x) = \left( \delta(x W^{(i)}_{\text{gate}}) \odot (x W^{(i)}_{\text{up}}) \right) W^{(i)}_{\text{down}}$

where δ is an activation function (typically SiLU), ⊙ is element-wise multiplication, W_gate and W_up project the input to an intermediate dimension, the gate activation gates the up-projection, and W_down projects back to the hidden dimension.

The paper's diagnosis of existing dynamic routing methods identifies three specific failure modes that jointly motivate BEAM's decoupled design:

Failure mode 1: Routing rank is not a reliable proxy for expert necessity. Existing logit-modification methods (MoE-Dynamic, XMoE, Adaptive Gating) process experts in descending order of routing weight, implicitly assuming that higher-ranked experts are more important. The paper argues this is an "unverified heuristic" (Section 2) and provides empirical evidence against it: in BEAM's learned masking patterns across Qwen3-30B-A3B (Figure 6b), the probability of masking a Top-1 expert is 0.43, while the probability of masking a Top-8 expert is only 0.53 — a difference of just 0.10 across the entire rank range. This means the mask router frequently prunes the highest-weight expert and retains lower-ranked ones, a pattern that sorted-order approaches cannot express. Appendix B.4 reinforces this: across all layers and models, "the min masked rank stays as low as 1–3, meaning that even highly-ranked experts are frequently pruned when redundant for a given token."

Failure mode 2: Cumulative-probability thresholds enforce a minimum activation of 1 expert per token. Because MoE-Dynamic and similar methods activate experts until the cumulative softmax probability exceeds a threshold, the first expert (highest weight) is always activated. This prevents the model from reaching the zero-activation regime, where a token bypasses all routed experts entirely. BEAM's results show this regime is practically important: at high sparsity on Qwen1.5 (β = 0.1, Table 1), the average activated expert count is 0.56, meaning that in roughly 44% of cases, no routed expert fires. At extreme sparsity (β = 1.0), the average drops to 0.11. These zero-activation tokens represent computation that logit-based methods fundamentally cannot eliminate.

Failure mode 3: Unified routers create gradient conflicts between expert selection, load balancing, and sparsity control. When a single router must simultaneously optimize for selecting the right experts, maintaining uniform expert utilization (load balancing), and minimizing the number of activated experts, these objectives pull in conflicting directions. The load-balancing loss encourages all experts to be used equally, while sparsity encourages using fewer experts overall. The paper argues that entangling these objectives degrades model capacity, which is why methods like DynMoE (which replaces the router entirely with sigmoid gates) collapse in post-training settings (Appendix B.3: DynMoE drops from 55.15 to 3.59 average accuracy on DeepSeekV2-Lite while activating 30.50 experts on average vs. the original K=6).

These three failure modes jointly motivate BEAM's architectural principle: separate the router that handles expert selection and load balancing (primary router) from the mechanism that controls sparsity (mask router). The primary router continues to operate exactly as in standard MoE training, while the mask router is a new component trained specifically for redundancy elimination.

The BEAM Mechanism: Four-Step Token-Adaptive Expert Masking (Section 3.2)

BEAM processes each token through four sequential steps within each MoE layer. The key innovation is that the mask router operates on the output of the primary router, not as a replacement for it.

Step 1: Standard Top-K Routing (unchanged). The primary router R receives the token embedding x ∈ R^{d_h} and computes routing logits r = R(x) ∈ R^N. The Top-K operator retains the K largest logits and sets the remaining N - K entries to -∞. The normalized weights are:

$g_i = \text{Softmax}(\text{Top-K}(r))_i, \quad i = 1, \ldots, N$

where g_i > 0 exactly for the Top-K experts and zero otherwise. This step is identical to standard MoE inference — the primary router's parameters are inherited from the pretrained model and (during BEAM fine-tuning) continue to be optimized via the language modeling loss and the load-balancing loss.

Step 2: Raw Mask Generation. A lightweight auxiliary mask router, parameterized by a single weight matrix W_m ∈ R^{d_h × N}, processes the same input token embedding x to produce a raw mask vector. A sigmoid activation constrains the values to (0, 1):

$\hat{m} = \sigma(x W_m)$

where m̂_i ∈ (0, 1) can be interpreted as the model's confidence that expert i is necessary for the current token. Values near 1 indicate the expert should likely be kept; values near 0 indicate it should likely be pruned.

The mask router is lightweight by design: it adds only d_h × N parameters per MoE layer, which is negligible compared to the expert parameters (each expert contains d_h × d_intermediate × 2 + d_intermediate × d_h parameters in its GLU structure). For Qwen3-30B-A3B with d_h = 2048 and N = 128, the mask router adds 2048 × 128 = 262,144 parameters per MoE layer, while each expert contains roughly 2048 × 6144 × 2 + 6144 × 2048 ≈ 37.7M parameters — so the mask router is approximately 0.7% of a single expert's size, or roughly 0.005% of the total MoE parameters across all 128 experts.

A crucial design choice: the mask router receives the same input as the primary router (the token embedding x), not the primary router's output logits. This means the mask router makes its decisions based on token-level features rather than on routing weights, which allows it to learn token-expert relevance patterns independently of the primary router's ranking. This is what enables the position-independent masking behavior observed in Figure 6b: the mask router is not simply "the primary router with a threshold" but rather a separate function of the token representation.

Step 3: Binary Masking. The raw mask m̂ is binarized using a fixed threshold τ = 0.5 to produce a hard binary mask m ∈ {0,1}^N:

$m_i = \begin{cases} 1, & \text{if } \hat{m}_i \geq 0.5 \\ 0, & \text{otherwise} \end{cases}$

The choice of τ = 0.5 is ablated in Table 4. The paper tests thresholds in {0.1, 0.3, 0.5, 0.7, 0.9} on Qwen1.5-MoE-A2.7B and finds that τ = 0.5 achieves the best overall performance (59.61 average accuracy), "largely driven by stronger commonsense results." The paper hypothesizes that τ = 0.5 "offers the greatest gradient sensitivity around the decision boundary while maintaining a stable Top-K initialization." Recall that the mask router parameters are initialized to zero, which yields m̂_i = σ(0) = 0.5 for all experts, and therefore m_i = 1 for all experts — the model starts with all Top-K experts active, preserving the original behavior. A threshold of 0.5 is symmetric around this initialization, meaning the model must learn to push m̂_i meaningfully below 0.5 before any expert gets pruned.

Step 4: Masked Aggregation. The final routing weights are obtained by element-wise multiplication of the Top-K weights and the binary mask:

$\hat{g} = g \odot m$

where ⊙ is the Hadamard (element-wise) product. Since g_i = 0 for experts outside the Top-K set (due to the -∞ assignment in Step 1), and m_i ∈ {0,1}, the effective routing weight ĝ_i is non-zero only for experts that are both in the Top-K set and not masked. The number of non-zero entries in ĝ can range from 0 (all Top-K experts masked) to K (none masked).

The layer output is computed as the standard weighted sum, but now with the masked weights:

$y = \sum_{i=1}^N \hat{g}_i \cdot E_i(x)$

This computation is conceptually a full weighted sum, but in practice (during inference with the vLLM kernel), experts where ĝ_i = 0 are never computed — the kernel skips them entirely (see the deployment section below).

What happens when all Top-K experts are masked? This edge case is analyzed in Appendix A.3. The MoE hidden state update in a modern Transformer MoE block is:

$h' = h + \sum_{i=1}^N \hat{g}_i E_i(\mathcal{N}(h)) + \delta_{sh} g_{sh} E_{sh}(\mathcal{N}(h))$

where N(·) is normalization, E_i are routed experts, E_sh is the (optional) shared expert, g_sh is the shared expert's weight, and δ_sh ∈ {0,1} is an indicator for whether a shared expert exists. When all ĝ_i = 0 (all routed experts masked), this reduces to:

$h' = h + \delta_{sh} g_{sh} E_{sh}(\mathcal{N}(h))$

Two cases:

• Architectures with shared experts (Qwen1.5-MoE, DeepSeekV2): δ_sh = 1 and the layer reduces to shared-expert-only computation. The token still receives some expert processing, just from the always-active shared experts rather than the routed ones.
• Architectures without shared experts (Qwen3-MoE): δ_sh = 0 and the token bypasses the entire MoE layer via the residual connection, receiving only h' = h. The paper notes this is effectively "dynamic layer skipping" and connects it to recent work on inference acceleration through conditional computation (Yang et al., 2025b; Lawson and Aitchison, 2025; Amer et al., 2026).

The paper emphasizes that this zero-activation behavior is not an edge case — it occurs frequently. At β = 0.1 on Qwen1.5, the average activated expert count is 0.56 (Table 1), meaning roughly 44% of token-expert decisions result in zero activation. At β = 1.0, this rises to roughly 89%. The fact that BEAM maintains 85% of baseline performance (52.66 vs. 61.71) even when nearly all experts are skipped for most tokens is a striking empirical finding that suggests substantial expert computation is genuinely redundant.

Training Strategy: Making Binary Masks Learnable (Section 3.3)

BEAM's training strategy addresses the fundamental challenge that binary decisions are not differentiable. The mask router must learn to output discrete keep/prune decisions, but gradient-based optimization requires continuous gradients. The solution combines a straight-through estimator for gradient propagation, careful parameter initialization, and a three-term loss function that jointly optimizes language modeling, load balancing, and sparsity.

Straight-Through Estimator (Section 3.3.1)

The binarization in Step 3 (Equation 6) is a hard threshold:

$m_i = \mathbb{1}[\hat{m}_i \geq 0.5]$

where 𝟙[·] is the indicator function. This function has gradient zero almost everywhere (it is flat except at the discontinuity), which would prevent any gradient from flowing back to the mask router parameters W_m.

The straight-through estimator (STE), originally introduced by Bengio et al. (2013) for training neural networks with stochastic binary neurons, solves this by approximating the gradient of the threshold function as the identity function during backpropagation. Formally, during the backward pass:

$\frac{\partial \mathcal{L}}{\partial \hat{m}} \approx \frac{\partial \mathcal{L}}{\partial m}$

where L is the total loss. This means the gradient with respect to the binary mask m is copied directly to the gradient with respect to the raw mask m̂, as if the threshold function were not there. In the forward pass, the actual binary values m_i ∈ {0,1} are used for computing the layer output and loss. In the backward pass, the threshold is treated as transparent, allowing gradients to flow through to m̂ and from there to the mask router parameters W_m via the chain rule through the sigmoid.

What this accomplishes operationally: the mask router parameters W_m receive gradient signals that push m̂_i up (toward 1, keeping the expert) or down (toward 0, pruning the expert) based on how the binary masking decision affected the task loss. The sigmoid derivative σ'(a_i) — which appears in the full gradient expression derived in Theorem 3.2 — provides the actual gradient scaling: when m̂_i is near 0 or 1 (confident decisions), the gradient is small (sigmoid saturation); when m̂_i is near 0.5 (uncertain), the gradient is maximal.

Important training detail: "all Top-K experts are computed regardless of m to ensure proper gradient flow during training" (Section 3.3.1). During training, the full weighted sum over all Top-K experts is computed even though m_i = 0 zeros out some weights in the forward pass. This is necessary because the STE provides gradients through the masked path — if an expert were not computed at all, there would be no gradient signal to inform whether masking it was correct. Only at inference time (with the vLLM kernel) are masked experts truly skipped. This creates a train-inference gap, but the paper shows it does not cause problems (unlike the "Soft" ablation in Table 5, which uses sigmoid gating without binarization and suffers a 69.5% accuracy drop due to train-inference mismatch).

Parameter Initialization

The mask router parameters W_m are initialized to zero. This is a critical design choice for training stability. Since x W_m = 0 for all inputs when W_m = 0, the raw mask becomes:

$\hat{m}_i = \sigma(0) = 0.5 \quad \forall i$

and after binarization at τ = 0.5:

$m_i = 1 \quad \forall i$

This means that at the start of training, BEAM exactly reproduces the standard Top-K behavior — all Top-K experts are active, and the loss is identical to standard MoE fine-tuning. Sparsity emerges gradually as the mask router parameters move away from zero, pushed by the sparsity regularization loss L_reg and pulled by the task loss L_lm. The training curves in Figure 7 (Appendix B.2) confirm this: the expert active rate starts near 100% and drops sharply to a stable plateau within the first ~0.5 epoch, while the language modeling loss remains comparable to the SFT baseline throughout.

Initializing to zero (rather than randomly) ensures that the model does not begin training with arbitrary expert pruning, which could destabilize the primary router's optimization. It also means that the mask router is "neutral" at initialization — it expresses no preference for keeping or pruning any expert.

Three-Term Loss Function (Section 3.3.2)

The total training loss combines three terms:

$\mathcal{L} = \mathcal{L}_{\text{lm}} + \alpha \mathcal{L}_{\text{bal}} + \beta \mathcal{L}_{\text{reg}}$

where L_lm is the standard language modeling loss (cross-entropy on next-token prediction), L_bal is the expert load-balancing loss, and L_reg is the new sparsity regularization term. The hyperparameters are α (load-balancing coefficient, set to 1 × 10^{-3} across all experiments, per Table 6) and β (sparsity coefficient, the primary control knob).

Language modeling loss L_lm: standard autoregressive cross-entropy, ensuring the model continues to predict tokens accurately. This loss produces gradients for all model parameters: primary router, experts, attention layers, and mask router (through the STE path via ĝ).

Load-balancing loss L_bal: a standard auxiliary loss used in MoE training (since Shazeer et al., 2017) that encourages tokens to be distributed uniformly across experts. This prevents the "expert collapse" problem where the router sends all tokens to a few experts, leaving most experts unused. Crucially, L_bal is computed from the primary router's weights g before masking — the mask router does not receive gradients from L_bal (as formalized in Theorem 3.2). This is the decoupling principle in action: the mask router is free to prune experts without being penalized for creating load imbalance, which is the primary router's responsibility.

Sparsity regularization loss L_reg: the key new component. Defined as the L1 norm of the raw mask m̂ restricted to the Top-K candidate set T_K:

$\mathcal{L}_{\text{reg}} = \frac{1}{K} \sum_{i \in T_K} |\hat{m}_i|$

where T_K is the set of indices corresponding to the K experts selected by the primary router.

What it computes: for each of the K experts in the Top-K set, take the absolute value of the raw mask entry m̂_i (which is already non-negative due to the sigmoid, so |m̂_i| = m̂_i), sum them, and divide by K to get the average mask value within the candidate set. The result is a scalar between 0 and 1: 0 if all Top-K experts are completely suppressed (m̂_i → 0 for all i ∈ T_K), and 1 if all Top-K experts are kept at full confidence (m̂_i → 1).

Why this form:

• L1 rather than L2: Table 5 shows that replacing L1 with L2 regularization degrades both sparsity and accuracy (Avg-K increases from 1.23 to 2.01 and average accuracy drops from 77.14 to 75.28). L1 promotes sparsity because it provides constant-magnitude gradient pressure toward zero regardless of the current mask value, whereas L2 provides gradient proportional to the mask value — so as m̂_i approaches zero, L2's sparsity pressure weakens, while L1's remains constant. This is the standard L1-vs-L2 sparsity distinction from the compressive sensing literature.

• Restricted to Top-K candidates: the gradient from L_reg only affects experts in T_K. Experts outside the Top-K set are already masked by the -∞ assignment in Step 1 and their mask values are irrelevant (the mask is ⊙-ed with g_i = 0). Restricting the regularization avoids wasting gradient signal on experts that cannot be activated anyway.

• Normalized by K: dividing by K makes the magnitude of L_reg independent of the Top-K size, so the hyperparameter β can transfer across models with different K values. Without normalization, a model with K = 8 would receive 8× more sparsity gradient than one with K = 1, requiring per-model tuning of β.

• Applied to raw mask m̂ rather than binary mask m: because the binary mask m is not differentiable (that's the whole point of STE), applying L1 directly to m would produce zero gradient except through the STE path. Applying it to m̂ provides a direct, non-STE gradient that pushes the raw mask values downward regardless of the task loss gradient. This direct gradient path is essential for training stability — without it, sparsity would be driven only by the task loss gradient through the STE, which is noisy and indirect.

Control via β: the hyperparameter β directly controls the tradeoff between sparsity and accuracy. Increasing β strengthens the sparsity pressure, causing the mask router to prune more aggressively. The paper sweeps β ∈ {0.01, 0.1, 1.0} across all three models (Tables 1–3) and demonstrates smooth, predictable behavior: β = 0.01 yields mid sparsity (Avg-K = 1.56–4.23 depending on the model), β = 0.1 yields high sparsity (Avg-K = 0.56–1.23), and β = 1.0 yields extreme sparsity (Avg-K = 0.11–0.56). Accuracy degrades gradually as β increases, with β = 0.1 preserving over 95% accuracy across all models — the paper's recommended "good trade-off" setting.

Theoretical Analysis: Selective Gradient Propagation and the Role of β (Section 3.4)

The paper provides a theoretical analysis of BEAM's training dynamics that formalizes why the decoupling works and how β controls the pruning decision. The analysis focuses on the gradient of the total loss L with respect to the mask router's pre-activation a = x W_m (the input to the sigmoid, before applying σ).

Definition 3.1 (Gradient for Mask Router). Under the STE approximation, the full gradient of L with respect to the pre-activation a_i for expert i is:

$(\nabla_a \mathcal{L})_i = \left[ \frac{\partial \mathcal{L}_{\text{lm}}}{\partial \hat{g}_i} \cdot g_i + \frac{\beta}{K} \cdot \mathbb{1}[i \in T_K] \right] \sigma'(a_i)$

where:

• ∂L_lm / ∂ĝ_i is the gradient of the language modeling loss with respect to the masked routing weight for expert i — this captures how much changing the mask for expert i affects the task loss.
• g_i is the primary router's weight for expert i (zero if i is not in Top-K).
• β/K is the sparsity regularization gradient: since L_reg = (1/K) ∑_{j∈T_K} |m̂_j| and m̂_j = σ(a_j), the derivative ∂|m̂_j|/∂a_j = ∂m̂_j/∂a_j = σ'(a_j) (for m̂_j ≥ 0). The L1 norm's derivative is ±1, and since m̂_j ≥ 0, it's +1. Thus ∂L_reg/∂a_i = (β/K) · σ'(a_i) when i ∈ T_K, and 0 otherwise.
• 𝟙[i ∈ T_K] is the indicator function: 1 if expert i is in the Top-K set, 0 otherwise.
• σ'(a_i) = σ(a_i)(1 - σ(a_i)) is the derivative of the sigmoid, which is always positive.

Theorem 3.2 (Selective Gradient Propagation). The gradient satisfies two properties:

Case 1 — Non-selected experts (g_i = 0): If g_i = 0, then (∇_a L)_i = 0.

Proof: If g_i = 0, then expert i is not in the Top-K set, so i ∉ T_K. The task-loss term vanishes because g_i = 0 multiplies the gradient. The regularization term vanishes because L_reg is restricted to T_K and 𝟙[i ∈ T_K] = 0. Therefore the total gradient is zero, and "the mask router receives no learning signal for non-selected experts."

What this means operationally: the mask router only learns about experts that the primary router already considers relevant (those in the Top-K set). Experts outside Top-K are invisible to the mask router — it neither receives task gradients nor sparsity gradients for them. This is a desirable property because those experts are already pruned by the primary router, and learning to mask them would be redundant. It also prevents the mask router from wasting capacity on modeling expert-token relationships for experts that will never be activated.

Case 2 — Selected experts (g_i > 0): If g_i > 0, then i ∈ T_K and:

$(\nabla_a \mathcal{L})_i = \left[ \frac{\partial \mathcal{L}_{\text{lm}}}{\partial \hat{g}_i} \cdot g_i + \frac{\beta}{K} \right] \sigma'(a_i)$

What this computes: the gradient for each Top-K expert has two competing components:

1. The task-loss term: (∂L_lm/∂ĝ_i) · g_i. This term can be positive or negative. A negative gradient ∂L_lm/∂ĝ_i < 0 means that increasing the mask for expert i (making it more likely to be kept) reduces the task loss — this expert is helpful for the current token. A positive gradient means the opposite: keeping this expert hurts the task, suggesting it should be pruned.
2. The sparsity term: β/K. This is always positive (since β > 0 and K > 0), which means it consistently pushes a_i downward — toward lower mask values and thus toward pruning.

The net gradient direction is determined by the sign of ∂L_lm/∂ĝ_i · g_i + β/K:

• Expert retained when: ∂L_lm/∂ĝ_i · g_i < -β/K. The task-loss gradient is sufficiently negative (expert is helpful enough) to outweigh the sparsity pressure. The net gradient is negative, pushing a_i upward → m̂_i increases → expert more likely to be kept.
• Expert pruned when: ∂L_lm/∂ĝ_i · g_i > -β/K (or positive). The task-loss gradient is not negative enough to overcome the sparsity pressure (or is positive, actively harmful). The net gradient is positive, pushing a_i downward → m̂_i decreases → expert more likely to be pruned.

Why this form matters — the role of β: The hyperparameter β directly controls the threshold at which an expert's task contribution outweighs the sparsity pressure. A larger β makes the sparsity term larger, requiring a more negative task gradient (stronger evidence of usefulness) to keep an expert — hence higher sparsity. A smaller β makes the sparsity term weaker, allowing even marginally helpful experts to survive. This provides a clean, interpretable mechanism for the sparsity-accuracy tradeoff: β is essentially a per-expert cost that the mask router must "pay" (in terms of task loss improvement) to justify keeping an expert active.

What ∂L_lm/∂ĝ_i · g_i physically represents: The term g_i (primary router weight) scales the task gradient. Experts with larger routing weights g_i receive proportionally larger gradient signals — both positive and negative — because they contribute more to the layer output. This means the mask router is most sensitive to the task relevance of high-weight experts, which is appropriate because pruning a high-weight expert has a larger impact on the output. Conversely, a low-weight expert (small g_i) receives a weak task gradient signal, making it easier for the sparsity pressure to push it toward pruning — again appropriate, because its contribution was small to begin with.

What is NOT in the mask router gradient: The load-balancing loss L_bal is computed from g (pre-masking) and does not produce gradients for the mask router. The proof in Case 2 does not include any term from L_bal. This is the formal guarantee of decoupling: the mask router optimizes L_lm + β L_reg, while the primary router optimizes L_lm + α L_bal. The two routers have separate parameter sets and separate (mostly) loss terms, avoiding the gradient conflicts that plague unified approaches.

vLLM Deployment Implementation (Appendix A.4)

The practical speedups claimed in the paper depend on turning BEAM's binary mask decisions into actual skipped computation at inference time. The paper implements this as two kernel-level modifications to vLLM's standard MoE pipeline, requiring no changes to the model architecture or inference framework beyond these kernels.

Background on vLLM MoE execution: In vLLM, MoE layers are executed using a two-phase kernel approach. First, a routing kernel assigns each token to its Top-K experts, producing an array topk_ids of shape [num_tokens, top_k] containing expert indices. Second, an alignment kernel (moe_align_block_size_kernel) groups tokens by expert assignment and aligns them into contiguous blocks for efficient batched expert computation. Tokens assigned to the same expert are grouped together so the expert's FFN can be computed on a contiguous batch, maximizing GPU utilization.

Modification 1: mask_route_kernel. This kernel runs after the standard routing kernel and applies the binary mask. For each (token, slot) pair in the Top-K assignments:

1. Read the original expert index from topk_ids.
2. Read the corresponding mask logit (pre-binarization value) from the mask router's output buffer.
3. If mask_logit > 0, keep the expert index unchanged.
4. If mask_logit ≤ 0, overwrite the expert index with -1.

The threshold of 0 (rather than 0.5) in the kernel is notable: it corresponds to the raw mask m̂ before sigmoid, where m̂ = σ(a) and σ(0) = 0.5. So mask_logit > 0 is equivalent to m̂ > 0.5 — the same binarization threshold used in training. The kernel operates on the pre-sigmoid logits for efficiency.

Additionally, the kernel sets the expert index to -1 for invalid entries (out-of-range expert indices, which can occur for padding or edge cases), cleanly handling edge conditions.

Modification 2: moe_align_block_size_kernel. The standard alignment kernel iterates over topk_ids and increments per-expert token counters to build the grouping. The modified version simply adds a check:

if (expert_id != -1) {
    ++tokens_cnts[...];
}

Entries with expert_id = -1 (masked experts) are ignored during grouping. They are not assigned to any expert's compute batch, so the corresponding expert FFN is never executed for that token. The masked expert incurs zero FLOPs — it is not just weighted by zero in the output sum (which would still require computation), but truly skipped at the hardware level.

What makes this "plug-and-play": the paper emphasizes that these modifications are "lightweight, preserves compatibility with vLLM's existing optimizations such as operator fusion and memory coalescing, and introduces negligible integration overhead." Specifically, the only change to the model loading and inference pipeline is adding the mask router parameters (loaded from the fine-tuned checkpoint) and routing the mask logits to mask_route_kernel. There is no change to expert implementations, attention layers, or the KV cache. The paper's code snippet (Appendix A.4) shows the core logic is approximately 25 lines of CUDA, making integration straightforward.

Zero-Activation Behavior and Dynamic Layer Skipping (Appendix A.3)

The paper formalizes the edge case where all routed experts are masked (ĝ_i = 0 ∀i). The Transformer MoE block's hidden state update is:

$h' = h + \sum_{i=1}^N \hat{g}_i E_i(\mathcal{N}(h)) + \delta_{sh} g_{sh} E_{sh}(\mathcal{N}(h))$

where N(h) is the normalization function (typically RMSNorm), E_i is the i-th routed expert, E_sh is the shared expert (if present), g_sh is the shared expert's routing weight (typically 1.0 or a learned scalar), and δ_sh ∈ {0,1} indicates whether the architecture includes shared experts.

When all ĝ_i = 0, the sum term vanishes, and:

$h' = h + \delta_{sh} g_{sh} E_{sh}(\mathcal{N}(h))$

Case 1 — Shared experts present (Qwen1.5-MoE, DeepSeekV2): δ_sh = 1. The layer reduces to shared-expert-only computation. Shared experts are a standard MoE design pattern (introduced in DeepSeekV2 and also used in Qwen1.5-MoE) where a small number of experts (typically 1–2) are activated for every token regardless of routing, providing a "baseline" of expert processing that prevents the model from degenerating when routing fails. In this case, the token still receives expert computation, just from the shared experts rather than the routed ones.

Case 2 — No shared experts (Qwen3-30B-A3B): δ_sh = 0. The layer reduces to h' = h + 0 = h — the token completely bypasses the MoE layer through the residual connection. This is equivalent to dynamic layer skipping: the MoE layer contributes nothing beyond the identity mapping. The paper connects this to recent work on conditional computation for transformers (Yang et al., 2025b; Lawson and Aitchison, 2025; Amer et al., 2026), positioning BEAM's zero-activation behavior as an emergent form of layer-wise computation gating — but learned through the mask router rather than through a separate policy network.

The paper notes that "BEAM maintains strong model performance even when the average number of activated experts is below 1, implying that zero-activation cases occur frequently in practice." At high sparsity (β = 0.1) on Qwen1.5, the average is 0.56 activated experts per token, meaning roughly 44% of token-expert interactions result in zero activation. At extreme sparsity (β = 1.0), the average is 0.11, meaning ~89% zero activation. The fact that performance remains at 85% of baseline (Table 1) in the latter case suggests that the MoE layers are providing surprisingly little value for the majority of tokens — a finding with implications for MoE architecture design more broadly.

The per-layer and per-position sparsity analysis (Section 5.2, Figure 6a) shows that zero activation is not uniform: deeper layers and tokens with limited semantic content (punctuation, function words, chat template tokens) are more likely to trigger zero activation. The encoder-decoder-like pattern observed in Qwen models — where shallow layers activate more experts during prefill (knowledge encoding) and deeper layers activate more during decoding (reasoning) — means BEAM's sparsity adapts not just to token identity but to layer function.

Training Configuration and Hyperparameters (Appendix B.1, Table 6)

All experiments use supervised fine-tuning on the Tulu 3 SFT Mixture Dataset (Lambert et al., 2024), which covers reasoning, coding, and general knowledge tasks. Training configuration is held constant across all baselines and BEAM to ensure fair comparison:

Optimization: Learning rate 5 × 10^{-5} with linear schedule, warmup ratio 0.03, training for 2 epochs. Per-device batch size of 32, maximum token length of 4096.

Load balancing: α = 1 × 10^{-3} for all models, applied to the primary router's weights before masking.

Model-specific resource allocation: Qwen1.5-MoE-A2.7B uses 32 GPUs, DeepSeekV2-Lite uses 32 GPUs, Qwen3-30B-A3B uses 64 GPUs (due to its larger parameter count of 30B total, 3B activated).

Inference configuration: Temperature varies by model (0.7 for Qwen1.5 and Qwen3, 0.3 for DeepSeekV2), top-p varies (0.8 for Qwen1.5 and Qwen3, 0.95 for DeepSeekV2), top-k varies (20, 50, 20 respectively), repetition penalty varies (1.05, 1.00, 1.00). Maximum output tokens: 1024 for Qwen1.5 and DeepSeekV2, 2048 for Qwen3. Batch size 16 for all inference runs.

Why two epochs? The training dynamics in Figure 7 show that expert sparsification concentrates in the first ~0.5 epoch, with the active rate dropping from near 100% to a stable plateau. The remaining 1.5 epochs focus on optimizing the language modeling objective under the learned sparsity pattern. Two epochs is sufficient for convergence — the LM loss approaches the standard SFT baseline (without BEAM) by the end of training.

Baseline configuration: For Top-K Pruning, the original Top-K is used during training and reduced at inference. For Top-K Reduced, the model is trained from scratch with the reduced Top-K. For MoE-Dynamic, the cumulative probability threshold ϕ is swept to match sparsity levels (e.g., ϕ = 0.4 for mid sparsity, ϕ = 0.1 for high sparsity on Qwen1.5). For AdaMoE, the number of null experts is swept (e.g., 60 for mid sparsity, 120 for high sparsity on Qwen1.5). For BEAM, only β is varied (0.01, 0.1, 1.0), with τ = 0.5 fixed — this single-parameter control of the sparsity-accuracy tradeoff is presented as a key practical advantage.

Summary of Design Choices and Their Justifications

• Decoupled mask router rather than modifying the primary router: avoids gradient conflicts between sparsity, expert selection, and load balancing; formalized in Theorem 3.2 which shows the mask router receives no load-balancing gradients.
• Binary masking (hard decisions) rather than soft gating: the "Soft" ablation in Table 5 causes a 69.5% accuracy drop due to train-inference mismatch — the model learns to rely on continuous mask values during training but receives binary decisions at inference. STE bridges this gap.
• L1 regularization on Top-K raw mask rather than L2 or full-expert-set regularization: L1 provides constant sparsity pressure regardless of current mask value (Table 5 shows L2 is worse); restricting to Top-K avoids wasted gradient on non-selected experts.
• Mask router input is token embedding, not routing logits: enables position-independent masking (Figure 6b shows near-flat masking probability across ranks) — the mask router learns token-expert relevance, not rank-based thresholds.
• Zero initialization of mask router parameters: starts training from standard Top-K behavior, allowing sparsity to emerge gradually without destabilizing the primary router.
• Single-parameter sparsity control via β: provides predictable, monotonic sparsity-accuracy tradeoff without retuning other hyperparameters — contrasts with AdaMoE (tune null expert count) and MoE-Dynamic (tune threshold).
• vLLM integration via two kernel modifications: minimal code changes (approximately 25 lines of CUDA), preserves existing optimizations, and delivers real speedups (not just theoretical FLOPs reduction).

4. Key Insights and Innovations

Innovation 1: Decoupling Sparsity Control from Expert Selection as an Architectural Principle

The dominant assumption across all prior dynamic routing methods — whether logit-based, null-expert, or static — is that one mechanism should jointly handle expert selection, load balancing, and sparsification. MoE-Dynamic, XMoE, and related methods modify routing logits directly, attempting to encode sparsity decisions into the same weights that determine which experts to select. AdaMoE and LongCat route tokens to special experts, entangling sparsity with the expert choice problem. DynMoE replaces the router entirely with per-expert sigmoid gates that must simultaneously select, balance, and sparsify.

BEAM's central conceptual move is to reject this entanglement and instead introduce a second router with a single, clean objective: given the Top-K candidates that the primary router already selected (using well-understood, stable training dynamics), which ones are redundant for this token? The primary router continues to handle expert selection and load balancing — objectives it was explicitly designed and trained for, with established auxiliary losses (the Shazeer et al. load-balancing loss) that prevent expert collapse. The mask router handles exactly one thing: binary keep/prune decisions, optimized via a direct sparsity objective (L_reg) with a single interpretable hyperparameter β.

What makes this more than an engineering trick is the theoretical justification provided in Section 3.4. Theorem 3.2 proves that the mask router receives zero gradient from the load-balancing loss — not as an implementation detail, but as a mathematical consequence of masking after Top-K selection. The gradient path is g_i = 0 ⇒ (∇_a L)_i = 0 for non-selected experts, and the load-balancing term never appears in the mask router's gradient for selected experts. This is a formal guarantee that the two routers optimize independent objectives through independent gradient paths.

The significance of this decoupling is amplified by the failure of DynMoE (Appendix B.3), which attempts unified routing with sigmoid gates and collapses — dropping from 55.15 to 3.59 average accuracy on DeepSeekV2-Lite while activating 30.50 experts on average (5× the original Top-K of 6). This is not a minor regression; it is catastrophic failure, and it provides strong empirical evidence that entangling sparsification with expert selection in a single router creates optimization dynamics that are fundamentally unstable in post-training settings. BEAM's decoupling is not just a nice idea — it is what prevents the same collapse.

This is a fundamental conceptual shift, not an incremental refinement. Prior work asked "how do we modify routing to reduce expert activation?" BEAM asks "why should the mechanism that controls sparsity be the same mechanism that selects experts?" The answer — because doing so creates irresolvable gradient conflicts — reframes the problem: sparsity control is a separate optimization problem that should be handled by a separate component, and the architecture should be designed to guarantee (not just hope for) gradient separation between the two objectives.

Innovation 2: Active, Optimization-Driven Sparsity as a Trainable Objective

Prior dynamic routing methods achieve sparsity passively. Cumulative-probability methods (MoE-Dynamic, XMoE) activate experts until a threshold is reached — sparsity is whatever falls out of the softmax distribution, with no explicit pressure to reduce activation count beyond what the probability distribution naturally produces. Null-expert methods (AdaMoE, LongCat) achieve sparsity as a side effect of the router learning to send tokens to zero-computation experts — but the router is not explicitly rewarded for doing so; it must discover that sending tokens to null experts helps the task loss, which creates an indirect and unreliable training signal.

BEAM makes sparsity an active, explicit optimization target. The L_reg term directly penalizes the mask router for keeping experts active, with a constant gradient magnitude (L1 norm) pushing mask values toward zero. This converts the sparsity-accuracy tradeoff from an emergent property — one that requires careful threshold tuning and often disappoints — into a trainable objective with a predictable control parameter.

What makes this distinctive is not the use of L1 regularization (which is standard), but the combination of L1 with STE-based binary masking and the restriction to Top-K candidates. The L1 is applied to the raw mask m̂ rather than the binary mask m, providing a direct gradient path that is independent of the STE-approximated task gradient. This means the mask router receives two distinct, complementary gradient signals: (1) a task-driven signal through the STE that tells it which experts are genuinely useful or harmful for the current token, and (2) a constant sparsity pressure from L1 that pushes all masks toward zero. The interplay between these two signals — formalized in the gradient expression in Definition 3.1 — creates a clean thresholding dynamic: an expert survives only when its task contribution outweighs the sparsity cost β/K.

The practical consequence is that β provides monotonic, predictable control over the sparsity-accuracy tradeoff. Across three different model architectures with different expert counts and Top-K values, increasing β consistently increases sparsity with gradual accuracy loss (Tables 1–3). At β = 0.1, BEAM preserves over 95% accuracy across all models. At β = 1.0, sparsity becomes extreme (Avg-K = 0.11–0.56) with still-usable accuracy. This is in stark contrast to MoE-Dynamic, where adjusting the cumulative probability threshold ϕ produces non-monotonic and model-specific behavior, and AdaMoE, where tuning the null expert count requires architectural changes and produces unpredictable sparsity outcomes.

This is a fundamental shift in framing: from sparsity as a heuristic threshold on routing probabilities (an approach the paper shows fails because routing rank does not reliably indicate expert necessity — Figure 6b) to sparsity as a learned, token-adaptive decision optimized end-to-end with the task objective. The field's prior assumption was that expert importance is rank-ordered by routing weight. BEAM demonstrates this assumption is wrong and provides a mechanism that does not depend on it.

Innovation 3: The Zero-Activation Regime as a Practical and Empirically Validated Operating Point

All prior dynamic routing methods in the logit-modification family enforce a minimum activation of one expert per token. Cumulative-probability thresholds always activate at least the first expert. Adaptive gating methods (Top-1 vs. Top-2) explicitly choose a minimum of one. Even null-expert methods, while capable of zero routed-expert activation in principle (the token could route to a null expert), do not explicitly target or analyze this regime.

BEAM demonstrates that the zero-activation regime is not just achievable but practically important. At high sparsity on Qwen1.5-MoE-A2.7B (β = 0.1, Table 1), the average activated expert count is 0.56 — meaning that in roughly 44% of token-MoE-layer interactions, no routed expert fires at all. At extreme sparsity (β = 1.0), this rises to 0.11 average activated experts, meaning roughly 89% of interactions are zero-activation. The crucial finding is that the model retains 85% of baseline performance even in the latter case (52.66 vs. 61.71 average accuracy).

This is a diagnostic finding with implications beyond BEAM. It reveals that for the majority of tokens in typical inference sequences, the routed experts in MoE layers provide surprisingly little value — the model can function effectively with only shared experts (in architectures that have them) or even just the residual connection (in architectures without shared experts, like Qwen3). This finding echoes the observation in the token-wise sparsity analysis (Section 5.1, Figure 5) that "chat template tokens are highly redundant" and "fixed prompts like 'You are a helpful assistant' activate few experts yet maintain performance."

What makes this an innovation rather than just a result is that BEAM explicitly designs for and formalizes the zero-activation case. Appendix A.3 provides a clean analysis of what happens when all routed experts are masked, distinguishing between architectures with shared experts (where the layer reduces to shared-expert-only computation) and those without (where the layer reduces to the residual path — effectively dynamic layer skipping). Prior work either could not reach zero activation (logit methods) or treated it as an undefined edge case. BEAM treats it as a first-class operating regime and shows it is the dominant regime at high sparsity.

The connection to dynamic layer skipping (Section A.3, citing Yang et al., 2025b; Lawson and Aitchison, 2025; Amer et al., 2026) positions BEAM's zero-activation behavior within a broader research direction on conditional computation for transformers. The innovation is that BEAM achieves this skipping emergent from the mask router's per-expert decisions rather than through a separate policy network or layer-wise gating mechanism — it is a unified framework where token-level expert sparsity naturally extends to layer-level skipping as a limiting case.

Innovation 4: Empirical Refutation of the Routing-Rank-as-Importance Heuristic

A largely unstated but widely adopted assumption in the MoE literature is that an expert's position in the sorted routing logits — its rank — reflects its importance to the current token. This assumption is baked into every cumulative-probability method (which activates experts in descending rank order) and every adaptive-gating method (which decides between Top-1 and Top-2 based on the gap between the top two ranks). It is intuitive: the softmax assigns the highest probability to the most relevant expert, so why would you skip it and keep a lower-ranked one?

BEAM provides the first direct empirical evidence that this assumption is substantially wrong, at least for the post-training setting studied in this paper. Figure 6b shows the probability of each Top-K position being masked on Qwen3-30B-A3B under comparable sparsity conditions. For BEAM, the masking probability ranges from 0.43 (Top-1) to 0.53 (Top-8) — a spread of only 0.10 across the entire rank range. The mask router is almost equally likely to prune the highest-weight expert as the lowest-weight one. In contrast, MoE-Dynamic shows extreme position bias: it never masks Top-1 (0.00 probability), while masking probability jumps to 0.64 at Top-5 and 0.94 at Top-8. AdaMoE shows intermediate but still monotonic bias from 0.06 (Top-1) to 0.66 (Top-8).

Appendix B.4 reinforces this finding with layer-wise analysis: across all three models and all layers, "the min masked rank stays as low as 1–3, meaning that even highly-ranked experts are frequently pruned when redundant for a given token. Meanwhile, the max kept rank extends to the lower end of the Top-K range, confirming that low-ranked experts can be retained when critical."

This is a negative result with significant implications: it falsifies the core heuristic underlying an entire family of prior methods. If routing rank is not a reliable proxy for expert necessity, then any method that processes experts in sorted order — as all cumulative-probability and adaptive-gating methods do — is architecturally incapable of expressing the optimal sparsity pattern. This explains why BEAM substantially outperforms MoE-Dynamic and similar methods at equivalent sparsity levels (Tables 1–3): BEAM can prune the highest-weight expert and keep a lower-weighted one when the token-expert relevance warrants it, while logit-based methods cannot express this pattern.

This finding is not just comparative — it is diagnostic. It tells the field that future work on dynamic routing should not rely on routing rank as a signal for sparsification decisions, and should instead develop mechanisms (like BEAM's mask router) that evaluate token-expert relevance independently of the primary router's ranking. The fact that the mask router achieves this with a simple linear layer on the token embedding (same input as the primary router, not its output) suggests that token-expert relevance and routing weight capture different aspects of the token-expert relationship — a distinction that prior work had not recognized.

5. Experimental Analysis

Evaluation Methodology

• Dataset. All experiments use the Tulu 3 SFT Mixture Dataset (Lambert et al., 2024) for supervised fine-tuning, which covers reasoning, coding, and general knowledge tasks. For accuracy evaluation, the paper uses eight benchmarks from OpenCompass (Contributors, 2023) across three domains: Reasoning (MATH, GSM8K, HumanEval), Knowledge (MMLU, CEVAL, CMMLU), and Common Sense (CommonsenseQA, BoolQ). For acceleration evaluation, a custom benchmark of 5000 test samples is used with vLLM, with fixed input/output lengths of 128 and 32 tokens respectively.

• Base model(s). Three representative MoE architectures are evaluated: Qwen1.5-MoE-A2.7B (Bai et al., 2023) — 14.3B total parameters, 2.7B activated, K=4 routed + 4 shared experts per token from 60 total experts across 24 MoE layers; DeepSeekV2-Lite (Liu et al., 2024a) — 16B total, 2.4B activated, K=6 routed + 2 shared experts per token from 64 total experts across 26 MoE layers; and Qwen3-30B-A3B (Yang et al., 2025a) — 30B total, 3B activated, K=8 routed experts per token from 128 total experts across 48 MoE layers with no shared experts. The models span different scales, shared-expert configurations, and expert granularities, providing a diversified testbed.

• Metrics. For accuracy, the paper reports individual benchmark scores and average accuracy across all eight benchmarks at each sparsity level. For sparsity, the metric is average activated experts per token ("Avg. K") — the mean number of routed experts that survive masking across all tokens and layers. For acceleration, three metrics are reported: Time per Output Token (TPOT) in milliseconds, Time to First Token (TTFT) in milliseconds, and offline throughput (samples/second), all measured under vLLM on a single NVIDIA H20 GPU.

• Baselines. Five methods are compared. (1) Top-K Pruning: trains with original Top-K, reduces K at inference — this is the simplest post-hoc method and represents what happens when a deployed model is simply run with fewer experts. (2) Top-K Reduced: trains from scratch with a smaller Top-K — representing the "oracle" of what fixed-K could achieve if retrained. (3) MoE-Dynamic (Huang et al., 2024): activates experts sequentially until cumulative routing probability exceeds a threshold φ, representing the logit-modification family. (4) AdaMoE (Zeng et al., 2024): adds null experts that output zero, with the number of null experts swept per sparsity level, representing the special-experts family. (5) DynMoE (Guo et al., 2024): replaces the softmax router with per-expert sigmoid gates, evaluated in Appendix B.3 rather than the main tables. For MoE-Dynamic and AdaMoE, hyperparameters (φ and null expert count respectively) are tuned to match comparable sparsity levels with BEAM at each setting.

• Generation budget / compute accounting. The primary longitudinal axis is sparsity level rather than training budget: methods are compared at matched average activated expert counts (Avg-K), grouped into "Mid Sparsity," "High Sparsity," and "Extreme Sparsity" regimes. The paper achieves comparable sparsity levels by sweeping each method's control parameter (Top-K for pruning/reduced methods, φ for MoE-Dynamic, null count for AdaMoE, β for BEAM). For acceleration experiments, the comparison is at "High Sparsity" settings (the specific β values from Tables 1–3's high-sparsity rows). Training is controlled: all methods use identical optimization hyperparameters, dataset, and training duration (2 epochs, learning rate 5 × 10^{-5}, linear schedule, 32 per-device batch size) on the same GPU hardware.

• Cross-validation / statistical protocol. No cross-validation or statistical significance testing is reported. The evaluation uses the full test sets from OpenCompass benchmarks; exact sample counts per benchmark are not specified but follow standard OpenCompass evaluation protocols. The selection of β values (0.01, 0.1, 1.0) is fixed rather than tuned per-dataset — the paper emphasizes this as a feature (single-parameter control), but it means there is no held-out validation procedure for hyperparameter selection. For acceleration benchmarks, 5000 samples are used per test condition, providing reasonable measurement stability for latency/throughput metrics, though no error bars or confidence intervals are reported.

Main Quantitative Results

Performance Under Varying Sparsity Levels (Tables 1–3)

The central results are organized across Tables 1, 2, and 3 (one per model), each reporting accuracy on eight benchmarks and average activated experts (Avg-K) for BEAM at three β values and for all baselines at matched sparsity levels.

Headline result: BEAM preserves over 98% of original accuracy at mid sparsity across all three models while reducing Avg-K by 47–61%. At high sparsity, it retains over 95% accuracy with Avg-K dropping to as low as 14% of the original. Even at extreme sparsity (Avg-K well below 1), BEAM maintains usable performance — 85% of baseline on Qwen1.5, 87% on DeepSeekV2, and 88% on Qwen3 — while all baselines either fail to reach these sparsity levels or suffer catastrophic accuracy collapse when they do.

Qwen1.5-MoE-A2.7B (Table 1, original K=4, baseline accuracy 61.71):

At mid sparsity, BEAM (β=0.01) achieves Avg-K=1.56 (reduced 61% from K=4) with 61.36 average accuracy — retaining 99.4% of baseline. The best baseline at comparable sparsity is Top-K Reduced K=2 (Avg-K=2.00, 60.77 accuracy), meaning BEAM matches accuracy with 22% fewer activated experts. MoE-Dynamic (φ=0.4) activates 2.20 experts but achieves only 58.45 accuracy, while AdaMoE (Null=60) activates 1.53 experts with severely degraded accuracy of 49.57 — a 12-point gap to BEAM at similar sparsity.

At high sparsity, BEAM (β=0.1) achieves Avg-K=0.56 (only 14% of original K=4) with 59.53 accuracy — retaining 96.5% of baseline. Top-K Reduced K=1 achieves 54.20 accuracy at Avg-K=1.00 — BEAM uses 44% fewer experts while achieving 5.3 points higher accuracy. Top-K Pruning K=1 collapses to 46.25. MoE-Dynamic (φ=0.2) achieves Avg-K=1.47 with only 53.05 accuracy. AdaMoE (Null=120) achieves Avg-K=1.26 with 45.72 accuracy. The critical comparison is BEAM vs. Top-K Reduced at K=1: BEAM has fewer average experts (0.56 vs. 1.00) yet substantially outperforms (59.53 vs. 54.20), demonstrating that token-adaptive sparsity is more efficient than uniform reduction.

At extreme sparsity, BEAM (β=1.0) achieves Avg-K=0.11 — meaning roughly 89% of token-expert interactions result in zero activation — with 52.66 accuracy, retaining 85.3% of baseline. No baseline reaches this sparsity level; the closest is Top-K Reduced K=1 (Avg-K=1.00, 54.20) which uses 9× more experts for only 1.5 points higher accuracy.

Qwen3-30B-A3B (Table 2, original K=8, baseline accuracy 81.41):

At mid sparsity, BEAM (β=0.01) achieves Avg-K=4.23 (reduced 47% from K=8) with 79.99 accuracy — retaining 98.3% of baseline. Top-K Reduced K=4 achieves 78.90 (Avg-K=4.00): BEAM uses 5.7% more experts but achieves 1.1 points higher accuracy. Top-K Pruning K=4 degrades sharply to 69.60 — an 11.8-point drop from baseline, demonstrating that simply running the trained K=8 model with K=4 at inference is catastrophic for this architecture. MoE-Dynamic (φ=0.3) activates 5.04 experts (more than BEAM) but achieves only 77.49. AdaMoE (Null=128) activates 4.02 experts (fewer than BEAM) but collapses to 66.81 — a 13.2-point gap.

At high sparsity, BEAM (β=0.1) achieves Avg-K=1.23 (only 15% of original K=8) with 77.14 accuracy — retaining 94.8% of baseline. Top-K Reduced K=2 achieves 72.18 (Avg-K=2.00): BEAM uses 38% fewer experts and achieves 5.0 points higher accuracy. Top-K Pruning K=2 collapses catastrophically to 11.92 — an 86% accuracy loss, with near-zero scores on MATH (0.68), GSM8K (1.14), and HumanEval (0.00). This is the most dramatic evidence that post-hoc pruning of MoE models fails when the inference K diverges from the training K. MoE-Dynamic (φ=0.1) activates 1.74 experts but achieves only 67.93 — a 9.2-point gap to BEAM. AdaMoE (Null=256) activates 2.64 experts with only 51.01 accuracy.

At extreme sparsity, BEAM (β=1.0) achieves Avg-K=0.56 with 71.91 accuracy — retaining 88.3% of baseline. This exceeds Top-K Reduced K=1 (Avg-K=1.00, 53.95) by 18.0 points while using 44% fewer experts. This is a striking result: BEAM with fewer-than-one average expert outperforms a model retrained with exactly one expert per token by a massive margin, demonstrating that the ability to selectively activate zero experts for easy tokens and multiple experts for hard ones is far more efficient than uniform allocation.

DeepSeekV2-Lite (Table 3, original K=6, baseline accuracy 55.15):

At mid sparsity, BEAM (β=0.01) achieves Avg-K=2.61 (reduced 57% from K=6) with 55.06 accuracy — retaining 99.8% of baseline, essentially matching the original model's performance. Top-K Reduced K=4 achieves 54.32 but at Avg-K=4.00 (53% more experts than BEAM). MoE-Dynamic (φ=0.3) activates 4.31 experts with only 47.70 accuracy — a 7.4-point gap despite using 65% more experts. AdaMoE (Null=64) activates 3.25 experts with 47.09 accuracy. The pattern is consistent: BEAM achieves comparable or superior accuracy with substantially fewer experts.

At high sparsity, BEAM (β=0.1) achieves Avg-K=1.08 with 53.32 accuracy — retaining 96.7% of baseline. Top-K Reduced K=2 achieves 50.27 (Avg-K=2.00): BEAM uses 46% fewer experts with 3.1 points higher accuracy. MoE-Dynamic (φ=0.1) activates 3.90 experts — nearly 4× BEAM's count — but achieves only 44.90 accuracy. AdaMoE (Null=128) activates 2.11 experts with 41.31 accuracy. BEAM with Avg-K≈1 nearly matches the original K=6 baseline (53.32 vs. 55.15).

At extreme sparsity, BEAM (β=1.0) achieves Avg-K=0.48 with 47.39 accuracy — retaining 85.9% of baseline. The comparison with Top-K Reduced K=1 is decisive: Avg-K=1.00 yields only 35.77 accuracy, meaning BEAM uses 52% fewer experts and achieves 11.6 points higher accuracy. At this sparsity level, BEAM still outperforms Top-K Pruning K=2 (42.10 at Avg-K=2.00) by 5.3 points while using 76% fewer experts.

Summary of performance-sparsity dynamics: Across all three models, BEAM's advantage over baselines grows as sparsity increases. At mid sparsity, BEAM modestly outperforms Top-K Reduced (1-3 points). At high sparsity, the gap widens substantially (5-18 points, depending on the model). At extreme sparsity, BEAM dominates all baselines (12-33 points over Top-K Reduced K=1 on Qwen3 and DeepSeekV2). This monotonic improvement in relative advantage is the clearest evidence that token-adaptive masking provides benefits that compound with sparsity — uniform methods cannot adapt, while BEAM selectively preserves computation for tokens that need it.

The DynMoE comparison (Appendix B.3, Table 8) is a striking negative result. DynMoE over-activates experts massively: 61.66 on Qwen3 (vs. K=8), 30.06 on Qwen1.5 (vs. K=4), 30.50 on DeepSeekV2 (vs. K=6). Performance collapses: 3.59 average accuracy on DeepSeekV2 (from 55.15 baseline), 41.14 on Qwen1.5 (from 61.71), 43.52 on Qwen3 (from 81.41). This validates the paper's claim that replacing the router entirely is ill-suited for post-training sparsification — DynMoE's sigmoid gates cannot maintain the pretrained routing structure.

Acceleration Results (Figure 4)

The paper evaluates inference acceleration under both online (varying QPS) and offline (fixed large batch) settings, comparing BEAM at high sparsity against Top-K reduction (medium K, small K), MoE-Dynamic, and AdaMoE.

Headline results: BEAM achieves 1.3-2.5× TPOT improvement, 1.1-1.5× TTFT improvement, and 1.1-1.4× throughput improvement across models, with the largest gains on Qwen3-30B-A3B (no shared experts, enabling full MoE layer FLOPs reduction).

TPOT (Time per Output Token):

On Qwen1.5-MoE-A2.7B (QPS range 8–32), BEAM achieves TPOT reductions of 1.5-1.7× versus the K=4 baseline. At QPS=32, BEAM achieves 26.1ms vs. 43.7ms for K=4 baseline (1.67× speedup). Top-K Reduced K=2 achieves 41.7ms and K=1 achieves 27.4ms — BEAM uses fewer average experts (0.56 vs. 1.0) and achieves comparable or better latency. MoE-Dynamic and AdaMoE show negligible improvement over K=4 or are slower (MoE-Dynamic at 59.3ms, AdaMoE at 40.4ms at QPS=32), likely due to the overhead of cumulative-probability computation and null-expert routing.

On Qwen3-30B-A3B, BEAM achieves 1.5-1.9× TPOT speedup. At QPS=32, BEAM achieves 103.7ms vs. 199.8ms for K=8 baseline (1.93×). The gap to baselines widens with QPS: at QPS=8, BEAM (14.9ms) vs. K=8 (22.6ms) is 1.5×; at QPS=32, BEAM (103.7ms) vs. K=8 (199.8ms) is 1.9×. This is because higher QPS increases queuing pressure, amplifying the benefit of per-token compute reduction. Top-K Reduced K=4 achieves 173.8ms and K=2 achieves 154.4ms at QPS=32 — BEAM is 1.49× faster than K=2 despite using fewer average experts (1.23 vs. 2.0).

On DeepSeekV2-Lite, BEAM achieves the largest individual speedup: 2.5× at QPS=24 (30.8ms vs. 76.2ms for K=6 baseline). At QPS=32, BEAM (56.8ms) vs. K=6 (94.2ms) is 1.66×. MoE-Dynamic is actually slower than K=6 at higher QPS (144.8ms vs. 94.2ms at QPS=32), and AdaMoE is substantially slower (156.3ms) — both due to dynamic routing overhead exceeding any sparsity benefit.

TTFT (Time to First Token): Measured at QPS=32 (high load). On Qwen1.5, BEAM achieves 48.8ms vs. 51.6ms for K=4 (1.06×). On Qwen3, 103.7ms vs. 117.1ms (1.13×). On DeepSeekV2, 78.0ms vs. 113.5ms (1.46×). TTFT improvements are more modest than TPOT because TTFT is dominated by the prefill phase, which processes all input tokens in parallel and is typically compute-bound rather than memory-bound — the sparsity benefit is partially offset by kernel launch overhead and the parallel nature of prefill computation.

Offline throughput: BEAM achieves 91.5 vs. 80.3 samples/s on Qwen1.5 (1.14×), 87.2 vs. 60.6 on Qwen3 (1.44×), and 101.2 vs. 82.9 on DeepSeekV2 (1.22×). The largest throughput gain is on Qwen3, consistent with its architecture lacking shared experts — shared expert computation is always-on and cannot be reduced by BEAM, so models without shared experts benefit more proportionally from routed-expert sparsification.

Architecture-dependent speedup limits: The paper explicitly notes that speedup is constrained by the shared-expert ratio. Qwen1.5 has 4 shared experts out of 8 total activated experts per token — meaning at best, BEAM can eliminate the 4 routed experts, reducing MoE layer FLOPs by at most 50%. Qwen3 has no shared experts, enabling up to 85% FLOPs reduction (Table 2, extreme sparsity: Avg-K=0.56 from original K=8, corresponding to roughly 93% routed expert reduction, though the paper's claim of 85% likely accounts for normalization and residual path overhead that cannot be eliminated). DeepSeekV2 has 2 shared experts out of 8 total — at most 75% reduction in routed expert FLOPs, but shared experts still consume 25% of original MoE compute.

Practical significance of MoE-Dynamic and AdaMoE acceleration failures: Figure 4 shows both methods often achieve worse latency than the unmodified baseline despite their sparsity, because the overhead of dynamic threshold computation (MoE-Dynamic) or null-expert routing (AdaMoE) exceeds any FLOPs savings from reduced expert activation. This is a critical practical finding: theoretical sparsity does not translate to wall-clock speedup if the sparsification mechanism itself is computationally expensive. BEAM's binary mask evaluation is a single linear layer + sigmoid + threshold — negligible overhead — explaining its ability to convert sparsity into actual speedups.

Ablation Studies and Robustness Checks

Binary threshold τ (Table 4): Evaluated on Qwen1.5-MoE-A2.7B with five threshold values τ ∈ {0.1, 0.3, 0.5, 0.7, 0.9}. Increasing τ monotonically reduces Avg-K from 0.78 at τ=0.1 to 0.28 at τ=0.9, confirming expected behavior (higher thresholds require stronger mask router confidence to keep experts). τ=0.5 achieves the best overall accuracy (59.61), driven primarily by commonsense performance (76.23 vs. 66.43 at τ=0.1 and 69.72 at τ=0.7). The paper hypothesizes this is because τ=0.5 provides maximum gradient sensitivity around the decision boundary and is symmetric around the zero-initialization point (σ(0)=0.5). A non-obvious pattern: reasoning accuracy remains relatively stable across τ (41.09–44.41), while commonsense accuracy shows a sharp peak at τ=0.5 — suggesting the commonsense tasks are more sensitive to having the right experts available, and τ extremes either keep too many (τ=0.1, wasting capacity) or prune too aggressively (τ=0.9, losing necessary expertise). Based on this ablation, τ=0.5 is fixed as the universal threshold, with β alone controlling sparsity.

Training configuration ablation (Table 5): Evaluated on Qwen3-30B-A3B at high sparsity (β=0.1). Five variants are tested against the full BEAM configuration (Avg-K=1.23, 77.14 average accuracy):

• Without L_reg (β=0): Removing the sparsity loss increases Avg-K to 6.31 — nearly the full K=8 — because the mask router receives no explicit sparsity pressure. Accuracy marginally improves to 77.80 (+0.9%), demonstrating that the L_reg penalty imposes a small but measurable accuracy cost in exchange for 5.1× sparsity improvement (1.23 vs. 6.31 activated experts). This is the expected sparsity-accuracy tradeoff working as designed.

• L1 replaced with L2: Avg-K increases to 2.01 (less sparse than L1's 1.23) and accuracy drops to 75.28 (2.4% below BEAM). This is a non-trivial finding: L2 not only achieves less sparsity but also degrades accuracy compared to L1, suggesting that the constant gradient magnitude of L1 (which does not diminish as mask values approach zero) is important not just for achieving sparsity but also for learning better mask patterns. L2's diminishing gradient near zero may cause the mask router to get "stuck" with intermediate mask values that are neither fully active nor fully pruned, degrading both sparsity and accuracy.

• Soft mask without STE (plain sigmoid gating): Catastrophic failure — Avg-K=1.34 (similar to BEAM) but accuracy collapses to 23.56, a 69.5% drop. This is the train-inference mismatch problem: during training, the model learns to rely on continuous mask values (e.g., m̂_i = 0.3 means the expert contributes at 30% weight), but during inference, these are binarized to 0 or 1, producing a distribution shift that the training did not account for. The model has learned to use partial expert contributions (e.g., blending multiple experts at fractional weights) that cannot be replicated at inference time. This validates the necessity of STE-based binarization for closing the train-inference gap.

• Soft mask with temperature scaling (gradually sharpening sigmoid): Partially mitigates the train-inference mismatch — accuracy recovers to 73.31 (5.0% below BEAM) with Avg-K=1.78. Temperature scaling provides a softer landing than pure STE but still underperforms, likely because the continuous-to-discrete transition during training introduces optimization instability compared to STE's consistent binarization throughout.

Robustness across β values (Tables 1–3): The three β values (0.01, 0.1, 1.0) are applied identically across all three models without model-specific tuning. The resulting sparsity and accuracy follow consistent trends: β=0.01 yields mid sparsity (Avg-K 1.56–4.23, 98%+ accuracy retention), β=0.1 yields high sparsity (Avg-K 0.56–1.23, 95%+ accuracy retention), β=1.0 yields extreme sparsity (Avg-K 0.11–0.56, 85–88% accuracy retention). The fact that a single β sweep produces qualitatively similar behavior across architecturally diverse models (different K values, different shared-expert configurations, different scales) suggests the sparsity-accuracy tradeoff is governed by a relatively invariant dynamic rather than being highly model-specific — an important practical property for deployment.

Training dynamics (Figure 7, Appendix B.2): The training curves show that expert sparsification concentrates in the first ~0.5 epoch, where the active rate drops sharply from near 100% to a stable plateau. The language modeling loss converges to levels comparable to the SFT baseline (gray dashed line) across all three models and β=0.1. This is an important robustness check: it confirms that the mask router can learn to sparsify without destabilizing the base model's language modeling capability. The rapid initial sparsification followed by stable LM optimization suggests the mask router quickly identifies redundant experts and then the model adapts to the reduced expert set.

Expert load balancing after BEAM fine-tuning (Figure 9, Appendix B.5): The utilization rates of experts before and after BEAM fine-tuning are visualized for all three models. The paper reports that "BEAM performs uniform masking across experts, maintaining relatively balanced expert loads." This is a critical robustness check: if the mask router learned to always prune specific experts (e.g., always mask Expert 3 regardless of the token), the expert load balance would be destroyed, leading to some experts being unused and others overburdened — a form of expert collapse distinct from the training-time collapse that load-balancing losses prevent. The fact that load balance is preserved suggests the mask router is making token-adaptive decisions (pruning different experts for different tokens) rather than learning a static pruning pattern.

Task-specific acceleration (Table 9, Appendix B.6): BEAM's inference speedup is measured per benchmark on Qwen3-30B-A3B. Speedups range from 1.10× (BoolQ) to 1.53× (HumanEval), with a weighted average of 1.32× across all tasks. HumanEval shows the largest speedup (1.53×), likely because code generation prompts contain substantial boilerplate (function signatures, docstrings) that BEAM recognizes as low-information and sparsifies aggressively. BoolQ shows the smallest speedup (1.10×), possibly because yes/no questions are shorter and have less redundant context. This per-task variation demonstrates that BEAM's acceleration is not uniform — it depends on the informational content of the specific benchmark's inputs — but the consistent >1.10× speedup across all tasks confirms the acceleration is broadly applicable, not an artifact of a particular benchmark's characteristics.

Critical Assessment

Mapping Claims to Evidence: What Was Actually Tested

The paper's principal claims, as articulated in the abstract and introduction, are:

Claim 1: "BEAM retains over 98% of the original model's performance while reducing MoE layer FLOPs by up to 85%."

This claim requires careful decomposition. The "98% performance retention" figure is supported at mid sparsity (β=0.01) across all three models (Tables 1-3): Qwen1.5 achieves 99.4% retention (61.36/61.71), DeepSeekV2 achieves 99.8% (55.06/55.15), and Qwen3 achieves 98.3% (79.99/81.41). At high sparsity (β=0.1), retention drops to 95-97%, which no longer meets the "98%" threshold but is still impressive. The "85% FLOPs reduction" figure likely refers to Qwen3 at β=1.0 (Avg-K=0.56 vs. original K=8, meaning 93% reduction in routed expert activation), though the paper explicitly claims "up to 85%" rather than "93%" — presumably accounting for computation that cannot be eliminated (normalization, residual connections, attention layers, and the mask router's own overhead). The issue is that "up to 85%" and "98% performance retention" do not hold simultaneously: the 85% FLOPs reduction occurs at extreme sparsity where accuracy retention is 88% (Qwen3, β=1.0) or 85% (Qwen1.5, β=1.0), not 98%. The paper does not explicitly state this tradeoff, which could mislead readers into thinking both numbers apply to the same operating point.

Claim 2: "BEAM achieves up to 2.5× faster decoding and 1.4× higher throughput."

The 2.5× decoding speedup is achieved on DeepSeekV2-Lite at QPS=24 (Figure 4): TPOT of 30.8ms for BEAM vs. 76.2ms for K=6 baseline. This is a specific operating point (specific model, specific QPS) rather than a general claim, but the paper qualifies it with "up to." The 1.4× throughput figure corresponds to Qwen3 in offline mode (87.2 vs. 60.6 samples/s). The paper does not report what happens at higher QPS values — the TPOT curves suggest speedup increases with QPS on Qwen3 (1.5× at QPS=8, 1.9× at QPS=32), so the 2.5× number may be specific to the QPS=24 sweet spot on DeepSeekV2 rather than a general upper bound. More critically, the throughput gains (1.1-1.4×) are substantially more modest than the TPOT gains (1.3-2.5×), and the paper does not fully explain this gap. A likely reason: throughput at large batch sizes is compute-bound, and BEAM's sparsity reduces per-token FLOPs but also introduces irregular computation patterns (variable expert counts per token) that reduce GPU utilization compared to the uniform Top-K case. The paper's custom kernel (Appendix A.4) mitigates but does not fully solve this scheduling irregularity.

Claim 3: "BEAM decouples sparsity control from expert selection, avoiding gradient conflicts."

Theorem 3.2 provides the theoretical justification: the mask router receives zero gradient from the load-balancing loss. The experimental evidence for this claim is indirect. The paper demonstrates that BEAM preserves load balance (Figure 9) and maintains stable training (Figure 7), which is consistent with the decoupling claim but does not directly test it. A direct test would be: train BEAM with and without the load-balancing loss applied to the mask router (violating decoupling) and show that the coupled version performs worse. This ablation is not performed. The DynMoE results (Table 8) provide circumstantial evidence — DynMoE entangles everything and collapses — but DynMoE differs from BEAM in multiple ways (different gating mechanism, no primary router, sigmoid gates), so its failure cannot be attributed solely to the lack of decoupling. The decoupling claim is theoretically well-motivated and behaviorally consistent with the evidence, but it has not been experimentally isolated.

Claim 4: "Routing rank is not a reliable proxy for expert necessity."

This claim is directly supported by Figure 6b and Appendix B.4. BEAM's masking probability across Top-K ranks is remarkably flat (0.43 to 0.53), while MoE-Dynamic's masking is strongly rank-dependent (0.00 at Top-1, 0.94 at Top-8). The layer-wise analysis in Appendix B.4 shows that minimum masked rank is consistently 1-3 across all layers and models, meaning highly-ranked experts are frequently pruned. This is strong evidence. However, the interpretation requires nuance: these results are from the post-training SFT setting with BEAM's specific training procedure. It is possible that the rank-importance correlation is stronger in pretraining (where routers are primarily trained) and weakens after SFT because the task distribution shifts — experts that were important for pretraining may become redundant for the SFT tasks, and vice versa, decorrelating rank from post-training importance. The paper does not discuss this possibility, which would limit the generality of the claim.

Genuine Weaknesses and Missing Evidence

Single evaluation paradigm (post-training SFT). All results are after supervised fine-tuning on the Tulu 3 SFT Mixture Dataset. The paper does not evaluate BEAM in a pretraining setting, where MoE routing is primarily learned. This matters because: (1) pretraining data is far more diverse than SFT data, potentially changing the relationship between routing rank and expert importance; (2) the load-balancing dynamics during pretraining (where expert collapse is a genuine risk) may interact differently with BEAM's sparsity pressure; (3) the practical use case for MoE efficiency is arguably largest at pretraining scale (where models are trained once and deployed many times). The paper acknowledges this limitation implicitly (Section 6, "BEAM is evaluated on three MoE architectures; its effectiveness on other MoE designs remains to be validated") but does not discuss the pretraining vs. post-training distinction.

No comparison to token-dropping or early-exit methods. The paper compares against routing-based methods (logit modification, null experts, static pruning) but not against approaches that achieve efficiency by skipping computation entirely for certain tokens or layers. Methods like token merging (ToMe), token pruning, or early exiting (layer-wise dynamic depth) operate on a different axis — reducing computation by dropping tokens or layers rather than experts — but compete for the same efficiency objective. The zero-activation analysis (Appendix A.3) explicitly connects BEAM to dynamic layer skipping, yet no layer-skipping baseline is compared. A comparison against a method that simply drops low-attention tokens or skips MoE layers for certain tokens would contextualize whether BEAM's fine-grained per-expert masking is more efficient than coarser computation-skipping strategies.

Limited acceleration benchmarking. All acceleration results are on a single GPU (NVIDIA H20) with fixed input/output lengths (128/32 tokens). Real-world deployments involve variable-length sequences, batching of heterogeneous requests, and often multi-GPU expert parallelism. BEAM's dynamic sparsity creates irregular computation patterns (variable expert counts per token) that may interact poorly with: (1) batch padding and batching efficiency — sequences with different sparsity patterns cannot be efficiently batched because their computation graphs diverge; (2) expert parallelism across GPUs — if certain experts are disproportionately masked (even if load balance is preserved on average, per-batch imbalance could cause GPU underutilization); (3) memory coalescing — the custom kernel's filtering of -1 entries creates non-contiguous memory access patterns that may reduce effective bandwidth. The paper mentions these as future work (Section 6) but does not evaluate them.

No statistical error reporting. All accuracy and acceleration numbers are reported as point estimates without confidence intervals, standard deviations, or significance tests. The eight-benchmark average (Avg. column in Tables 1-3) is a simple mean across benchmarks with different scales and variances. For a paper making fine-grained claims about accuracy retention (98% vs. 95% vs. 88%), knowing whether these differences are statistically reliable — especially on the smaller benchmarks like HumanEval (which typically has only 164 problems in the standard split) — matters for interpreting the results.

The β sweep is coarse. Three β values (0.01, 0.1, 1.0) span three orders of magnitude. The transition from "mid sparsity" to "high sparsity" occurs between β=0.01 and β=0.1, where the sparsity-accuracy tradeoff may have interesting structure (e.g., is there a β where accuracy drops sharply? Is the tradeoff convex or concave?). A finer sweep would characterize the Pareto frontier more precisely and help practitioners select the optimal operating point for their specific latency/accuracy requirements.

No ablation on mask router capacity. The mask router is a single linear layer W_m ∈ R^{d_h × N}. The paper does not test whether a deeper mask router (e.g., two-layer MLP) would improve masking decisions, or whether a shallower one (e.g., low-rank factorization to further reduce overhead) would suffice. Given that the mask router's parameter count is already negligible (~0.005% of expert parameters), the single-layer design is a reasonable default, but understanding the capacity requirements for the masking task would inform deployment on models with different hidden dimensions or expert counts.

No analysis of how mask patterns evolve during training. Figure 7 shows the aggregate active rate over training, but there is no analysis of which experts get pruned when, whether masking patterns stabilize early or continue to evolve, or whether different layers learn different sparsity patterns (the layer-wise analysis in Figure 6a is from after training, not during). Understanding the training dynamics at a finer granularity could reveal whether the mask router learns generalizable token-expert relevance rules or overfits to the SFT data's specific sparsity patterns.

Where the Claims Hold Conditionally

The 98% accuracy retention claim is condition-specific. It holds at β=0.01 (mid sparsity) across all three models, but it is not a general property of BEAM — it is a property of BEAM at a specific operating point with a specific β value. Increasing β to achieve higher sparsity necessarily reduces accuracy retention. The paper is transparent about this (the β sweep is explicitly shown), but the abstract's phrasing ("retains over 98% of the original model's performance while reducing MoE layer FLOPs by up to 85%") juxtaposes two numbers from different operating points in a way that could be read as applying simultaneously.

The speedup claims are architecture-dependent and QPS-dependent. BEAM's speedup is larger on models without shared experts (Qwen3: 1.44× throughput) than on models with shared experts (Qwen1.5: 1.14×). It is larger at higher QPS (Qwen3 TPOT: 1.5× at QPS=8, 1.9× at QPS=32) because queuing amplifies the benefit of per-token compute reduction. A deployment with a different shared-expert ratio or operating at low QPS would see more modest gains.

The superiority over baselines is most pronounced at high sparsity. At mid sparsity, BEAM's advantage over Top-K Reduced is modest (1-3 accuracy points). At high and extreme sparsity, the gap widens dramatically (5-33 points). This means BEAM's value proposition is strongest when aggressive sparsity is the goal — if the deployment only needs modest sparsity (e.g., 50% reduction), Top-K Reduced (retraining with smaller K) may be competitive and simpler. However, BEAM's ability to reach sparsity levels that other methods cannot approach at all (Avg-K < 1, extreme sparsity) is a qualitative advantage: for practitioners who need maximum efficiency, Top-K Reduced is architecturally limited to Avg-K ≥ 1 while BEAM is not.

The decoupling claim is validated by elimination rather than isolation. The paper demonstrates that methods that entangle routing and sparsification (DynMoE, MoE-Dynamic, AdaMoE) perform poorly, and that BEAM (which decouples) performs well. This is consistent with the decoupling hypothesis but does not isolate it as the causal mechanism — other differences between BEAM and these baselines (binary masking vs. soft thresholds, explicit L1 sparsity objective vs. implicit sparsity, STE-based training vs. standard backprop) could also explain the performance gap. A direct ablation that entangles BEAM's own mask router with load balancing (e.g., by feeding L_bal gradients to W_m) would isolate the decoupling effect but is not performed.

6. Limitations and Trade-offs

The Difficulty Estimation Cost Is Not Accounted For in Reported Efficiency Gains

The assumption or constraint. BEAM achieves its sparsity gains through a mask router that is trained end-to-end during supervised fine-tuning. The paper explicitly states in Appendix A.2: "BEAM requires a post-training SFT phase to learn the mask router, which incurs additional training cost proportional to the model size." This training cost is not amortized or accounted for in any of the headline efficiency numbers. The 2.5× decoding speedup and 1.4× throughput improvement (Figure 4) are measured at inference time after the SFT phase is complete, treating the training cost as a sunk cost. Similarly, the performance-sparsity tradeoff curves (Figure 1, Tables 1–3) compare the fine-tuned BEAM model against baselines, without factoring in that a practitioner must first run 2 epochs of SFT on 32–64 GPUs (Table 6) with a learning rate of 5 × 10⁻⁵ to achieve these results.

The consequence. The practical calculus for deploying BEAM depends critically on the ratio of inference tokens to training tokens. If a model is fine-tuned once and then serves millions or billions of inference queries, the amortized training cost per query is negligible, and BEAM's inference-time speedups represent net savings. But if a practitioner needs to fine-tune for each new task or domain (as is common in multi-tenant serving or rapidly-evolving applications), the SFT cost may dominate. For Qwen3-30B-A3B, 64 GPUs running for 2 epochs represents a substantial compute investment that must be recouped through downstream inference savings — and the paper provides no break-even analysis. Furthermore, the training requires access to the Tulu 3 SFT Mixture Dataset, which may not match every deployment's target distribution; mismatch between SFT data and deployment tasks could degrade the mask router's sparsity decisions.

What evidence exists in the paper. The training dynamics in Figure 7 (Appendix B.2) show that the language modeling loss converges to baseline levels within 2 epochs, confirming that the SFT phase is sufficient but not providing evidence about whether it is necessary. No ablation studies test shorter training (e.g., 1 epoch, 0.5 epochs), and no experiments measure whether the mask router can be trained on a smaller dataset or with fewer GPUs. The paper also does not report wall-clock training time, making it impossible for practitioners to estimate the total cost of adopting BEAM for their specific hardware configuration.

Mitigation status. The paper does not address this limitation beyond acknowledging it in Appendix A.2. There is no discussion of whether the mask router could be pre-trained on a generic corpus and then adapted with minimal task-specific fine-tuning, or whether the β hyperparameter could be adjusted at inference time to trade sparsity for accuracy without retraining (the paper uses fixed β during both training and inference). The paper frames BEAM as "plug-and-play" (Section 1) and "practical" (Section 4.3), but the plug-and-play characterization applies only to inference deployment, not to the training pipeline that produces the deployable model.

All Results Are Limited to Post-Training SFT on Three MoE Architectures; Pretraining Behavior Is Unknown

The assumption or constraint. Every experiment in the paper — accuracy benchmarks (Tables 1–3), acceleration measurements (Figure 4), ablation studies (Tables 4–5), and analysis (Section 5) — uses models fine-tuned on the Tulu 3 SFT Mixture Dataset via supervised fine-tuning. The base models (Qwen1.5-MoE-A2.7B, DeepSeekV2-Lite, Qwen3-30B-A3B) come from pretraining, but BEAM's mask router is trained only during SFT, and all evaluations measure post-SFT performance on standard benchmarks. The paper does not evaluate BEAM during pretraining, where MoE routing dynamics are fundamentally different: pretraining data distributions are far broader, the language modeling objective is the sole training signal (no instruction-following), and expert load balancing is a more acute concern because expert collapse during pretraining is a well-documented failure mode (Shazeer et al., 2017; Fedus et al., 2022). The paper acknowledges this scope limitation in Appendix A.2: "BEAM is evaluated on three MoE architectures; its effectiveness on other MoE designs (e.g., with different gating mechanisms or expert granularities) remains to be validated."

The consequence. The paper's central finding — that routing rank is a poor proxy for expert necessity, with BEAM masking Top-1 experts at probability 0.43 and Top-8 at 0.53 (Figure 6b) — may be specific to the post-training SFT setting. During pretraining, routers are primarily optimized for next-token prediction on diverse corpora, and the relationship between routing weight and expert importance may be stronger. After SFT on a narrower task distribution, experts that were important for pretraining may become redundant for the SFT tasks, decorrelating rank from post-training importance. If this is the case, the rank-independence finding — which the paper uses to motivate BEAM's decoupled design over logit-based methods — would not generalize to pretraining, and methods like MoE-Dynamic (which relies on rank ordering) might be more competitive during pretraining than they appear in this paper's post-training evaluation. More broadly, the paper's claim that BEAM "avoids gradient conflicts" between sparsity and load balancing (Section 3.1, Section 3.4) is theoretically sound but has been tested only in the SFT regime where load balancing may be less critical (the pretrained model already has a well-balanced router, and SFT data is less diverse, reducing the risk of expert collapse).

What evidence exists in the paper. None. The paper provides no pretraining experiments, no comparison of router behavior before vs. after SFT, and no analysis of whether the mask router's learned sparsity patterns transfer across tasks or datasets. All three models are fine-tuned on the same dataset (Tulu 3 SFT Mixture), so it is unknown whether BEAM's performance depends on the specific characteristics of that dataset. Additionally, the three architectures share design commonalities (all are decoder-only transformer MoE models with standard Top-K routing and GLU experts), and the paper does not test on architectures with different gating mechanisms (e.g., BASE layers, expert-choice routing, soft mixtures) or different expert resolutions (e.g., DeepSeek-V3 style fine-grained experts).

Mitigation status. The paper does not attempt to mitigate this limitation, nor does it propose specific follow-up work to evaluate BEAM during pretraining. The limitation is acknowledged in Appendix A.2 as a scope constraint rather than an active area of investigation. Given that pretraining is where the bulk of MoE FLOPs are consumed and where efficiency gains would have the largest absolute impact, this is a significant gap between the paper's demonstrated capability and its claimed practical value.

Reported Speedups Are Architecture-Dependent and Limited by Shared-Expert Ratios

The assumption or constraint. BEAM's practical inference speedup is fundamentally bounded by the proportion of computation spent in routed experts versus computation that BEAM cannot reduce: shared experts, attention layers, normalization, and the residual path. The paper quantifies this directly in Section 4.3: "The achievable speedup is limited by model architecture. For example, Qwen1.5-MoE-A2.7B contains 4 shared experts out of 8 total, limiting their MoE layer FLOPs reduction to at most 50%. In contrast, Qwen3-30B-A3B has no shared experts, enabling an 85% FLOPs reduction and substantially higher throughput gains."

The consequence. The 2.5× decoding speedup (headline number in the abstract) is achieved on DeepSeekV2-Lite at a specific operating point (QPS=24, Figure 4), while throughput improvements are substantially more modest: 1.14× on Qwen1.5, 1.22× on DeepSeekV2, and 1.44× on Qwen3. A practitioner deploying BEAM on a model with a high shared-expert ratio (e.g., DeepSeekV2 with 2 shared experts out of 8 total activated, or Mixtral-style models where all experts in certain layers are always active) would see speedups closer to the 1.1–1.2× range, not the 2.5× headline figure. This matters because the most widely deployed open-source MoE models in production (Mixtral 8×7B, Mixtral 8×22B, DeepSeek-V2/V3) all use shared experts or hybrid architectures where the shared expert ratio is non-negligible. The paper's results on Qwen3 — an architecture without shared experts — represent a best-case scenario for BEAM's speedup potential that may not transfer to the most commonly used MoE architectures.

What evidence exists in the paper. Figure 4 provides per-model speedups that clearly show the architecture dependence: Qwen3 throughput improvement (1.44×) is substantially larger than Qwen1.5 (1.14×) and DeepSeekV2 (1.22×). The TPOT improvements follow the same pattern: Qwen3 achieves 1.9× at QPS=32, while Qwen1.5 achieves 1.7× at QPS=32, and DeepSeekV2 achieves 1.7× at QPS=32 (though DeepSeekV2 peaks at 2.5× at QPS=24 — an interesting non-monotonicity that the paper does not explain). The offline throughput numbers (1.14×, 1.44×, 1.22×) are the most representative of sustained serving efficiency, and they are uniformly modest compared to the FLOPs reduction percentages.

Mitigation status. The paper is transparent about this limitation in Section 4.3, explicitly noting the shared-expert constraint and connecting it to the quantitative results. However, the abstract and introduction emphasize the headline 2.5× and 1.4× numbers without qualification about architecture dependence. A practitioner reading only the abstract might expect these speedups on their MoE model of choice, which would be misleading if that model has shared experts. The paper does not propose techniques to extend BEAM to shared experts (e.g., learning masks for shared experts too, or dynamically skipping shared expert computation for tokens that don't benefit from it), which would broaden the method's applicability.

Single-GPU Acceleration Benchmarks May Not Transfer to Multi-GPU Expert-Parallel Deployments

The assumption or constraint. All acceleration benchmarks (Figure 4, Table 9) are conducted on a single NVIDIA H20 GPU with fixed input/output lengths of 128/32 tokens. In production, large MoE models (like Qwen3-30B-A3B with 30B total parameters) are typically deployed across multiple GPUs using expert parallelism — different experts reside on different devices, and tokens are dispatched across GPUs for expert computation. The paper acknowledges this in Appendix A.2: "our acceleration benchmarks are conducted on single-GPU settings, and the interaction between BEAM's dynamic sparsity and multi-GPU expert parallelism strategies needs further investigation."

The consequence. BEAM's dynamic sparsity creates irregular computation patterns that may interact poorly with multi-GPU expert parallelism in several ways. First, expert parallelism relies on load-balanced expert assignment to avoid GPU idle time: if some GPUs host experts that are frequently masked while others host experts that are frequently kept, the imbalance creates stragglers. The paper's load-balance analysis (Figure 9) shows that BEAM preserves aggregate load balance across experts, but this is measured over the entire evaluation set — per-batch or per-microbatch load imbalance could still occur, especially with small batch sizes or bursty request patterns. Second, the token dispatch and all-to-all communication pattern in expert-parallel deployments assumes a fixed number of activated experts per token (Top-K). BEAM's variable activation count means tokens require different amounts of communication bandwidth, potentially underutilizing the pre-allocated communication buffers. Third, the custom vLLM kernel (Appendix A.4) that skips masked experts by writing -1 and filtering during block alignment relies on per-GPU kernel logic; in a multi-GPU setting, the masking happens after expert dispatch, meaning tokens may still be communicated to GPUs hosting experts that will be masked — wasting inter-GPU bandwidth.

What evidence exists in the paper. None. There are no multi-GPU experiments, no analysis of per-batch load balance (only aggregate across all tokens), and no discussion of how the vLLM integration handles expert parallelism. The paper's custom kernel modifications (Appendix A.4) are described at the single-GPU CUDA level, with no indication of how they compose with NCCL all-to-all collectives or tensor parallelism. The single-GPU benchmarks use 5000 test samples with fixed sequence lengths, which avoids the variable-length batching challenges that would further complicate multi-GPU scheduling.

Mitigation status. The paper identifies this as a limitation in Appendix A.2 but does not attempt to characterize or mitigate it. The vLLM integration is presented as a key practical contribution ("plug-and-play solution for efficient MoE inference," Section 1), but the plug-and-play characterization has only been validated in single-GPU settings. A practitioner deploying BEAM on a multi-GPU serving system (e.g., 4× or 8× H20 for Qwen3-30B-A3B) cannot rely on the reported speedups without additional validation.

The Train-Inference Mismatch Between Soft Expert Computation and Binary Masking Is Resolved by STE, but the Resolution Is Imperfect

The assumption or constraint. BEAM's training procedure relies on the straight-through estimator (STE) to propagate gradients through the non-differentiable binarization operation (Equation 6). During training, "all Top-K experts are computed regardless of m to ensure proper gradient flow" (Section 3.3.1). This means the training-time computation graph evaluates every expert in the Top-K set, even though the binary mask zeros out their contribution in the forward pass. At inference time, with the vLLM kernel, masked experts are truly skipped — their FFNs are never executed. This creates a systematic gap between training and inference: during training, the model sees expert outputs that are computed but multiplied by zero, while during inference, those outputs are never produced at all.

The consequence. The training-inference gap means the model is optimized under conditions that do not match deployment. During training, the gradient flow through STE is:

$\frac{\partial \mathcal{L}}{\partial \hat{m}} \approx \frac{\partial \mathcal{L}}{\partial m}$

which approximates the binary threshold as an identity function in the backward pass. The mask router parameters W_m are updated based on this approximate gradient, which assumes that small changes in the raw mask m̂ produce proportional changes in the loss — an assumption that does not hold at the discrete boundary (m̂ = 0.5). The "Soft" ablation in Table 5 demonstrates the severity of the train-inference mismatch when it is not addressed: soft sigmoid gating without STE causes a 69.5% accuracy drop (23.56 vs. 77.14 for BEAM). While STE mitigates this, it does not eliminate it — the "Soft with temperature scaling" variant (which provides a softer transition from continuous to discrete during training) still underperforms BEAM by 5.0%. This suggests that even with STE, the model may learn mask patterns that rely on subtle gradient effects that are artifacts of the training-time computation of all experts, and these effects do not transfer perfectly to the inference-time setting where masked experts are truly absent.

What evidence exists in the paper. Table 5 provides the primary evidence: the "Soft" (no STE) ablation collapses, validating that the mismatch is real and severe. The "Soft with temperature scaling" ablation partially recovers but still lags BEAM, suggesting STE is not a complete solution. Figure 7 (training dynamics) shows that the language modeling loss under BEAM converges to levels comparable to the standard SFT baseline, which suggests the STE-based training is stable — but stability does not imply that the learned mask patterns are optimal for the inference-time computation graph. The paper does not compare BEAM's accuracy against an oracle that trains with actual expert skipping (which would require a REINFORCE-style gradient estimator or similar discrete optimization method, and is computationally prohibitive for large models).

Mitigation status. The paper treats STE as sufficient evidence that the train-inference gap is closed, based on the convergence of training dynamics (Figure 7) and the strong downstream performance (Tables 1–3). However, no experiments directly measure the gap: for example, comparing BEAM's accuracy when all experts are computed at inference (matching training) versus when they are truly skipped (the deployment setting). If these two settings produce different accuracy, it would indicate residual train-inference mismatch. The paper does not perform this comparison, nor does it discuss potential approaches to further close the gap (e.g., fine-tuning with actual expert skipping using discrete gradient estimators, or distillation from the training-time model to the inference-time model).

The Mask Router's Decisions Are Learned from SFT Data but No Analysis of Distribution Shift or Adversarial Robustness Is Provided

The assumption or constraint. BEAM's mask router learns token-adaptive masking patterns from the Tulu 3 SFT Mixture Dataset during fine-tuning. The paper demonstrates that these learned patterns generalize across the eight evaluation benchmarks (Tables 1–3) and that the sparsity patterns are semantically meaningful (Section 5.1: content words trigger more experts than function words). However, the paper provides no analysis of how the mask router behaves under distribution shift — when the inference-time input distribution differs from the SFT training distribution in domain, style, language, or task.

The consequence. The mask router is a learned component that makes discrete keep/prune decisions for each token-expert pair. If it encounters inputs from a distribution it was not trained on, it may make incorrect pruning decisions — suppressing experts that are actually necessary for the novel task while keeping experts that are irrelevant. Because the pruning decision is binary and irreversible at inference time (masked experts are never computed), a pruning error cannot be recovered by the primary router adjusting its weights. This is fundamentally different from methods like MoE-Dynamic, where sparsity is based on routing probability thresholds and at least the highest-weight expert is always activated — providing a safety net for out-of-distribution tokens. BEAM removes this safety net: if the mask router incorrectly decides that an out-of-distribution token requires zero routed experts, that token receives only shared-expert computation (or just the residual path, for architectures without shared experts), and the model has no mechanism to recover the missing expertise.

The per-task acceleration analysis (Table 9) provides indirect evidence of distribution sensitivity: BEAM's speedup on BoolQ (1.10×) is substantially smaller than on HumanEval (1.53×), suggesting that the SFT data distribution may be more similar to code generation tasks (where boilerplate tokens are reliably sparsified) than to reading comprehension tasks (where more tokens may be judged as requiring expert processing). A practitioner deploying BEAM on a novel domain cannot predict whether the mask router will maintain its accuracy-sparsity tradeoff or collapse to either over-conservatism (keeping too many experts, no speedup) or over-aggressiveness (pruning necessary experts, accuracy degradation).

What evidence exists in the paper. The eight-benchmark evaluation provides some evidence of cross-domain generalization: BEAM's accuracy retention at high sparsity is consistent across reasoning (MATH, GSM8K, HumanEval), knowledge (MMLU, CEVAL, CMMLU), and commonsense (BoolQ, CommonsenseQA) benchmarks, with no single domain showing catastrophic failure. However, all benchmarks are standard evaluation tasks that are likely in-distribution relative to the Tulu 3 SFT Mixture (which covers "reasoning, coding, and general knowledge tasks," Section 4.1). The paper does not evaluate on purposefully out-of-distribution inputs (e.g., non-English text, domain-specific technical documents, adversarial inputs designed to trigger unnecessary expert activation or undesirable masking). The token-wise analysis in Figure 5 shows plausible sparsity patterns (content words activate more experts), but plausibility does not guarantee robustness.

Mitigation status. The paper does not address distribution shift or adversarial robustness. The β hyperparameter provides a global sparsity-accuracy tradeoff, but it cannot be adjusted per-domain or per-token at inference time. There is no discussion of calibration techniques (e.g., using the raw mask m̂ values as confidence scores and falling back to standard Top-K when confidence is low), no evaluation of whether the mask router's decisions are monotonic with respect to semantic perturbation (e.g., paraphrasing a question should not change which experts are pruned), and no adversarial evaluation where inputs are constructed to exploit BEAM's masking mechanism.

7. Implications and Future Directions

How This Work Changes the Landscape

BEAM introduces a diagnostic reframing of dynamic MoE routing rather than a paradigm shift. The paper's core contribution is not a new architecture — it adds a single lightweight linear layer to existing MoE models — but a new way of thinking about the sparsification problem: sparsity control should be a separate optimization objective handled by a separate component, not entangled with expert selection and load balancing in a single router. This reframing matters because it explains the persistent failures of prior approaches and provides a clean architectural principle (decoupling) that is theoretically grounded (Theorem 3.2 proves the gradient separation), empirically validated (BEAM outperforms all baselines at comparable sparsity, Tables 1–3), and practically deployable (the vLLM kernel integration requires approximately 25 lines of CUDA).

The magnitude of this contribution is best understood as resolving a contradiction in the prior literature. The dynamic routing literature contained a puzzle: logit-modification methods (MoE-Dynamic, XMoE, Adaptive Gating) showed modest improvements in some settings but failed to reach extreme sparsity, while null-expert methods (AdaMoE, LongCat) could theoretically achieve high sparsity but suffered from instability and hyperparameter sensitivity. Static pruning methods could reduce expert count but gave up on token-level adaptivity. No prior work had explained why these approaches hit their respective ceilings. BEAM's diagnostic contribution is the empirical demonstration (Figure 6b, Appendix B.4) that routing rank is not a reliable proxy for expert necessity in post-training settings — the probability of masking a Top-1 expert is 0.43, while the probability of masking a Top-8 expert is only 0.53, a remarkably flat distribution. This single finding falsifies the core heuristic underlying every cumulative-probability and adaptive-gating method, which all process experts in descending rank order and therefore cannot express the optimal sparsity pattern. The paper thus provides a unified explanation for why prior methods plateau: they are architecturally constrained to a suboptimal sparsity space.

This diagnostic has immediate consequences for research prioritization. Directions that become more attractive:

• Post-training sparsification as a first-class research area. The paper demonstrates that substantial expert redundancy exists in pretrained MoE models after SFT — at extreme sparsity (β=1.0), Qwen1.5-MoE activates an average of 0.11 experts per token while retaining 85% of baseline accuracy (Table 1). This suggests that post-training MoE models contain massive unused capacity, and techniques specifically designed to exploit post-training redundancy (rather than pretraining-efficient routing) represent a high-impact research direction. BEAM's decoupled architecture provides a template for such techniques.

• Independent token-expert relevance modeling. The finding that the mask router makes position-independent masking decisions (Figure 6b) while the primary router produces rank-ordered weights implies that routing weight and token-expert relevance capture different information. This opens a research direction on learning independent representations of expert relevance — not just through linear layers as in BEAM, but potentially through attention-based mechanisms, contrastive learning objectives, or information-theoretic approaches that explicitly separate "which experts are relevant" from "how much should each expert contribute."

• Hardware-algorithm co-design for irregular sparsity. BEAM's custom CUDA kernel demonstrates that learned binary masking can be efficiently mapped to hardware through simple modifications (masking via -1 index and filter during block alignment). This suggests a broader opportunity: rather than constraining sparsity patterns to be hardware-friendly (e.g., uniform Top-K, group-wise structured sparsity), we can design hardware kernels that accommodate learned, unstructured sparsity patterns — and BEAM provides a concrete, simple example that this is viable.

Directions that become less attractive:

• Cumulative-probability threshold methods for post-training sparsity. Figure 6b shows that MoE-Dynamic's masking is heavily rank-biased (never masking Top-1, masking Top-8 at probability 0.94), and Tables 1–3 show that it underperforms BEAM at all sparsity levels. The paper demonstrates both the mechanistic limitation (sorted-order processing cannot express the necessary sparsity patterns) and the performance consequence (5–18 point accuracy gaps at high sparsity). Future work on dynamic routing should move beyond threshold-on-logits approaches, at least for post-training settings.

• Null-expert approaches as general-purpose sparsity mechanisms. AdaMoE requires model-specific tuning of the null expert count (60, 120, 128, 256 null experts across different models and sparsity levels in Tables 1–3), introduces extra hyperparameters, and achieves sparsity passively rather than through explicit optimization. Its performance uniformly lags BEAM and often underperforms even Top-K Reduced. The paper's ablation of DynMoE (Table 8) further demonstrates that replacing the pretrained router entirely is unstable in post-training settings. These results collectively suggest that adding special experts or replacing the routing mechanism is unnecessarily complex when a lightweight mask router on top of the existing routing structure can achieve better sparsity-accuracy tradeoffs with simpler hyperparameter control.

• Static expert pruning as a standalone solution. The paper does not directly compare against methods like DEK, EEP, or HC-SMoE, but the conceptual limitation is clear: static methods "cannot adapt to the varying complexity of input tokens at inference time" (Section 2), and BEAM's per-layer, per-token sparsity analysis (Figures 6a, 10–12) shows substantial variation in expert activation across tokens, layers, and prefill vs. decode phases. Static pruning cannot capture this variation and is therefore architecturally limited for fine-grained efficiency.

How this work reconciles conflicting prior findings. Before BEAM, the dynamic routing literature contained an unresolved tension. Some papers reported success with logit-based dynamic routing (Huang et al., 2024; Yang et al., 2024b), while others found that such methods provide limited sparsity and introduce overhead (Zeng et al., 2024; Jin et al., 2024). The paper's results suggest that both sets of findings were correct but incomplete: logit-based methods can achieve modest sparsity (the "mid sparsity" regime where MoE-Dynamic achieves Avg-K ~2–5) but fundamentally cannot reach extreme sparsity (Avg-K < 1) because of the minimum-activation-floor constraint and the rank-ordering heuristic. The divergent conclusions in prior work likely reflected different sparsity targets — papers targeting modest sparsity found logit methods adequate, while those targeting aggressive sparsity found them insufficient. BEAM provides a method that spans the full sparsity range (0.11 to 4.23 average activated experts across β values and models), unifying the regime where logit methods work with the regime where they fail under a single architectural principle.

Follow-Up Research This Work Enables

1. Evaluating BEAM during pretraining to determine whether the routing-rank independence finding generalizes. The paper's central diagnostic — that routing rank is a poor proxy for expert necessity — is demonstrated exclusively in post-training SFT settings on three models. A critical open question is whether this finding holds during pretraining, where (a) the data distribution is far broader, (b) the language modeling objective is the sole training signal, (c) expert load balancing is more acute, and (d) the relationship between router weights and expert importance may be stronger because the router was primarily trained on that distribution. A strong follow-up would train BEAM's mask router during a continued pretraining phase (starting from a pretrained MoE checkpoint, continuing with the original pretraining corpus, adding the mask router and L_reg loss) and measure: (1) whether the masking probability remains flat across Top-K ranks or becomes rank-correlated; (2) whether the sparsity-accuracy tradeoff (controlled by β) is similar in magnitude to the SFT results; (3) whether load balancing remains stable under the mask router's sparsity pressure during pretraining, where expert collapse is a genuine risk. A negative result (masking becomes strongly rank-correlated during pretraining) would not invalidate BEAM's utility for post-training deployment but would significantly reframe the routing-rank diagnostic as a post-training phenomenon rather than a general property of MoE routing.

2. Combining BEAM with dynamic layer skipping for end-to-end conditional computation. The paper explicitly connects BEAM's zero-activation behavior to dynamic layer skipping (Appendix A.3, citing Yang et al., 2025b; Lawson and Aitchison, 2025; Amer et al., 2026), noting that when δ_sh = 0 (no shared experts) and all routed experts are masked, the MoE layer reduces to the identity mapping through the residual connection. For architectures like Qwen3-30B-A3B (no shared experts), this means BEAM naturally performs layer skipping as a limiting case of per-expert masking. A natural extension is to learn layer-level skip decisions jointly with expert-level mask decisions, potentially using a hierarchical mask router where a layer-level gating module decides whether to even invoke the per-expert mask router. The concrete experiment: train a layer-gating module (a single linear layer with sigmoid output, binarized via STE) that decides whether to execute the MoE layer at all, and condition the per-expert mask router on the layer gate's decision. Compare against BEAM alone and against standalone layer-skipping methods (e.g., DASH, ConfLayers) on Qwen3-30B-A3B. The hypothesis: layer-level skipping captures coarse-grained redundancy (entire layers that are unnecessary for certain tokens), while BEAM's per-expert masking captures fine-grained redundancy (unnecessary experts within necessary layers), and the combination yields multiplicative sparsity gains beyond either method alone.

3. Training a difficulty predictor to eliminate the SFT overhead and enable deployment-time sparsity adaptation. BEAM currently requires a full SFT phase to train the mask router, which the paper acknowledges as a limitation (Appendix A.2: "BEAM requires a post-training SFT phase to learn the mask router, which incurs additional training cost proportional to the model size"). A practical follow-up would train a lightweight difficulty predictor that estimates per-token expert requirements directly, without end-to-end SFT. The approach: take a BEAM-trained model, record the per-token mask decisions (m_i values) across a diverse corpus, and train a small predictor model (e.g., a 2-layer MLP or a distilled student) to predict the mask from token embeddings alone. This predictor could be deployed alongside any MoE model without retraining its parameters, enabling "plug-and-play" sparsification without the SFT cost. The experiment: train the predictor on BEAM mask data from one model (e.g., Qwen3-30B-A3B at β=0.1) and evaluate on the other two models (Qwen1.5-MoE, DeepSeekV2-Lite) to test cross-architecture generalization. Metrics: accuracy of mask prediction (per-expert binary classification), end-to-end task accuracy under predicted masks vs. original BEAM masks, and inference speedup relative to the unmodified model. A negative result (poor cross-architecture generalization) would still be informative — it would suggest that mask patterns are architecture-specific and that the SFT phase is genuinely necessary.

4. Stress-testing BEAM under distribution shift and adversarial inputs. The paper provides no evaluation of BEAM's robustness to input distributions that differ from the SFT training data. A critical stress-test would evaluate BEAM on (a) purposefully out-of-distribution inputs: non-English text (e.g., Chinese, Arabic, code-switching), highly technical domain-specific documents (legal contracts, medical records, mathematical proofs), and inputs with unusual formatting or structure; (b) adversarial inputs designed to exploit the mask router: construct prompts where token-level masking decisions are semantically meaningful (e.g., replacing a critical mathematical operator with a synonym that the mask router incorrectly prunes) and measure whether accuracy degrades more for BEAM than for the unmodified model; (c) long-context scenarios: the paper's acceleration benchmarks use 128 input tokens, but in long-context settings (4K–128K tokens), the mask router's decisions must remain consistent across very long sequences — does the mask router's token-level decision generalize to tokens in long documents, or does it exhibit position-dependent bias? The key metric is whether the accuracy gap between BEAM and the baseline widens under distribution shift, indicating that the mask router has learned spurious correlations from the SFT data rather than genuine token-expert relevance.

5. Extending BEAM to shared experts and investigating whether "always-on" experts are genuinely necessary. The paper identifies the shared-expert ratio as a fundamental limit on BEAM's speedup (Section 4.3: Qwen1.5 achieves only 1.14× throughput vs. 1.44× for Qwen3, which lacks shared experts). This raises a natural question: are shared experts always necessary, or can BEAM's mask router learn to selectively skip them too? A concrete experiment: extend BEAM to produce masks for shared experts alongside routed experts, train on Qwen1.5-MoE or DeepSeekV2-Lite, and measure whether (a) the model can maintain accuracy when shared experts are dynamically masked, and (b) the resulting speedup approaches that of Qwen3 (no shared experts). If shared experts can be sparsified without accuracy loss, it would challenge the architectural assumption — widespread in MoE design since DeepSeekV2 — that shared experts provide essential baseline computation. If they cannot be sparsified, it would validate the shared-expert design principle and focus future efficiency work on architectures that minimize or eliminate shared experts (like Qwen3).

6. Investigating whether BEAM's sparsity patterns can be distilled into a statically pruned architecture. BEAM demonstrates that many experts are token-adaptively redundant, but it does not answer the question: how many experts would be sufficient if we statically removed the universally redundant ones and kept only those that BEAM ever activates? A follow-up experiment: analyze BEAM's masking patterns across a large corpus, identify experts that are never activated (mask probability ≈ 1.0 for all tokens), and statically prune them from the model. Then measure whether the pruned model (with standard Top-K routing, no mask router) can match BEAM's accuracy at the reduced expert count. If yes, this provides a training-free path to model compression: use BEAM once to identify redundant experts, prune them permanently, and deploy the smaller model with standard inference. If no (the mask router's token-adaptive decisions are essential, and static pruning loses necessary experts for some tokens), it would validate the claim that dynamic, token-adaptive sparsity is fundamentally more efficient than static compression — a finding with implications for the broader model compression literature.

Practical Applications and Downstream Use Cases

1. Cost-efficient API serving of MoE models with mixed-complexity traffic. In production API deployments (e.g., chat completions, code generation), the incoming token distribution includes a mix of high-complexity tokens (mathematical expressions, code logic, domain-specific terminology) and low-complexity tokens (boilerplate, punctuation, chat template tokens like "You are a helpful assistant"). BEAM's token-adaptive masking naturally allocates more expert computation to complex tokens and less to simple ones. The practical benefit: a serving system that deploys BEAM at β=0.1 (high sparsity, 95%+ accuracy retention per Tables 1–3) would achieve 1.3–1.9× lower per-token latency (TPOT) and 1.1–1.4× higher throughput compared to the unmodified model (Figure 4), directly translating to reduced GPU costs per query. For a deployment serving Qwen3-30B-A3B at QPS=32, the TPOT reduction from 199.8ms to 103.7ms (1.93×, Figure 4) means nearly half the GPU-seconds per output token. For a high-volume API serving millions of queries daily, this represents substantial infrastructure cost savings without meaningful accuracy degradation. The single-parameter β control means the deployment team can adjust the sparsity-accuracy tradeoff without retraining — simply select a different checkpoint trained with the desired β.

2. On-device or edge deployment of MoE models where memory bandwidth is the bottleneck. MoE models are typically too large to deploy on edge devices (phones, laptops, embedded systems) because expert parameters consume substantial memory. BEAM's extreme sparsity regime (β=1.0, Avg-K as low as 0.11–0.56 across models, Tables 1–3) demonstrates that the model can function with the vast majority of expert computation eliminated for most tokens. While the full expert parameters still need to be stored (BEAM does not reduce model size), the reduction in expert activation means that at inference time, the device loads and executes far fewer expert FFNs per token. For memory-bandwidth-bound edge devices, where loading expert weights from DRAM dominates latency, reducing the number of activated experts from K=8 to an average of 0.56 (Qwen3, β=1.0) represents a ~14× reduction in expert weight traffic per token. Combined with the vLLM kernel that truly skips masked expert computation (Appendix A.4), this could make MoE inference viable on devices where it was previously infeasible due to memory bandwidth constraints. The practical tradeoff is the accuracy degradation at extreme sparsity — 88% retention for Qwen3 (71.91/81.41, Table 2) — which may be acceptable for applications where latency or battery life is prioritized over benchmark-level accuracy (e.g., on-device autocorrect, real-time translation, voice assistants).

3. Training data generation pipelines where inference cost dominates the total budget. Large-scale synthetic data generation (e.g., generating training data for distillation, instruction-tuning, or RLHF) requires running inference on millions of prompts. In these pipelines, the inference cost typically dominates the total compute budget, and small per-query efficiency improvements compound substantially. BEAM's offline throughput improvement — 1.44× for Qwen3-30B-A3B, 1.22× for DeepSeekV2-Lite (Figure 4) — translates directly to generating 1.2–1.4× more training data for the same GPU budget. At the high sparsity β=0.1 operating point (95%+ accuracy retention), the generated data quality is nearly indistinguishable from the baseline model, making the efficiency gain essentially free. Furthermore, because data generation queries often contain repetitive structure (e.g., "Solve the following math problem: ..." templates), the chat-template token sparsity that BEAM exploits (Figure 5: "chat template tokens are highly redundant") would be particularly effective — the fixed prompt portions would consistently activate few experts, while the variable content portions would receive appropriate expert allocation. A concrete scenario: a team using Qwen3-30B-A3B to generate 10 million math reasoning solutions for a distillation dataset would need approximately 30% fewer GPU-hours with BEAM, saving thousands of dollars in cloud compute for a single generation run.

4. MoE model selection for new architecture design: prioritizing minimized or eliminated shared experts. The paper's architecture-dependent speedup analysis (Section 4.3) provides a clear, quantitative guideline for MoE architecture designers: the shared-expert ratio is the primary bottleneck for inference efficiency gains from dynamic sparsification. Qwen3-30B-A3B (no shared experts) achieves 1.44× throughput improvement and up to 85% FLOPs reduction, while Qwen1.5-MoE-A2.7B (4 shared experts out of 8 total activated) achieves only 1.14× throughput and at most 50% FLOPs reduction. For teams designing new MoE architectures targeting inference-efficient deployment, these results argue strongly for (a) eliminating shared experts entirely, or (b) making shared experts themselves amenable to dynamic masking (see follow-up direction 5), or (c) reducing the shared-to-routed expert ratio to minimize the "unsparsifiable" computation fraction. The paper provides concrete evidence that models without shared experts (Qwen3) can achieve strong performance (81.41 baseline accuracy, Table 2) while being significantly more amenable to dynamic sparsification — a design insight that directly informs the next generation of MoE architectures.

When to Prefer This Method

The paper articulates a clear tradeoff between BEAM and alternative approaches based on two axes: the desired sparsity level and the tolerance for training overhead. The decision rules, grounded in the paper's empirical results, are:

• Prefer BEAM when targeting high or extreme sparsity (Avg-K < 2 for models with K ≥ 4). At these sparsity levels, Top-K Pruning collapses (Qwen3 K=2: 11.92 accuracy vs. 81.41 baseline, Table 2), MoE-Dynamic plateaus due to the minimum-activation-floor constraint (Figure 6b shows it cannot reach zero activation), AdaMoE requires extensive hyperparameter tuning with inferior accuracy (Tables 1–3), and DynMoE catastrophically over-activates (Table 8). BEAM is the only method that achieves Avg-K well below 1 while retaining usable accuracy (Qwen1.5: Avg-K=0.11, 85% accuracy retention; Qwen3: Avg-K=0.56, 88% retention). If the deployment requires aggressive compute reduction, BEAM is the only viable option among the evaluated methods.

• Prefer BEAM when the deployment architecture lacks shared experts (or has a low shared-expert ratio). The speedup gains are strongly architecture-dependent: Qwen3 (no shared experts) achieves 1.44× throughput and 1.9× TPOT at QPS=32, while Qwen1.5 (4 shared experts out of 8) achieves only 1.14× throughput and 1.7× TPOT (Figure 4). The shared-expert computation is "unsparsifiable" by BEAM (it operates only on routed experts), so the net speedup is diluted proportionally to the shared-expert fraction. For architectures like Qwen3, BEAM's speedup is substantial; for architectures with high shared-expert ratios, the speedup may not justify the SFT overhead.

• Prefer Top-K Reduced (retraining with smaller fixed K) when only moderate sparsity is needed and training cost is not a constraint. At mid sparsity (Avg-K ~2–4), Top-K Reduced achieves competitive accuracy to BEAM (Tables 1–3: on Qwen1.5, Top-K Reduced K=2 achieves 60.77 vs. BEAM β=0.01 at 61.36; on DeepSeekV2, Top-K Reduced K=4 achieves 54.32 vs. BEAM at 55.06). Top-K Reduced has the advantage of conceptual simplicity (no mask router, no STE, no β tuning) and uses standard inference pipelines without custom kernels. However, it is limited to Avg-K ≥ 1 (you cannot have K=0.5 with fixed Top-K) and requires retraining for each sparsity target — if the deployment later needs more or less sparsity, the model must be retrained from scratch with the new K.

• Prefer Top-K Pruning (inference-only K reduction, no retraining) only as a quick baseline or when no training budget exists at all. The paper demonstrates that Top-K Pruning degrades catastrophically at high sparsity on some architectures (Qwen3 K=2: 11.92 accuracy, Table 2) because the model was trained with a different K and the routing weights are miscalibrated for the reduced K. It should only be used when the sparsity target is very modest (e.g., reducing K from 8 to 6 on Qwen3, or K from 4 to 3 on Qwen1.5) and even then, the paper provides no evidence that such modest reductions are safe across architectures.

• Avoid DynMoE and AdaMoE in post-training settings based on the paper's evidence. DynMoE collapses across all three models (Table 8: 3.59 average accuracy on DeepSeekV2), making it unsuitable for post-training sparsification. AdaMoE requires per-model tuning of the null expert count, introduces significant accuracy degradation at comparable sparsity (Tables 1–3: 10–15 point accuracy gaps to BEAM at high sparsity), and provides no acceleration benefit in some configurations (Figure 4: AdaMoE TPOT is sometimes worse than the unmodified baseline). These methods may have value in other settings (e.g., pretraining, where DynMoE's sigmoid gates might learn more stable routing), but for the post-training deployment scenario studied in this paper, they are strictly dominated by BEAM and Top-K Reduced.