MolHIT: Advancing Molecular-Graph Generation with Hierarchical Discrete Diffusion Models

ArXiv: 2602.17602

🎯 Pitch

Graph diffusion models for molecule generation have always trailed 1D sequence models in validity, but MolHIT breaks that barrier with 99.1% validity on MOSES—the first near-perfect score for graph diffusion—while simultaneously achieving the highest scaffold novelty of any model. It does this by diffusing atoms through a chemically meaningful hierarchy (coarse functional groups → fine atomic identities) rather than directly to a mask, resolving a fundamental one-to-many ambiguity that caused prior graph models to catastrophically fail on charged and aromatic atoms.

---

1. Executive Summary

This paper introduces MolHIT, a molecular-graph generation framework that overcomes long-standing validity-performance tradeoffs in discrete diffusion models by combining a Hierarchical Discrete Diffusion Model (HDDM) (a coarse-to-fine noising process that clusters atom types into chemically meaningful intermediate groups before refining to specific identities) with Decoupled Atom Encoding (DAE) (splitting atom tokens by aromaticity, formal charge, and hydrogen saturation to resolve one-to-many mapping ambiguities). MolHIT achieves new state-of-the-art results on the MOSES benchmark, including near-perfect validity (99.1%) for the first time in graph diffusion while simultaneously reaching a Quality score of 94.2% and the highest Scaffold Novelty (0.39) among all compared models, establishing that graph-based molecular generation can match or exceed 1D sequence models in validity without sacrificing structural exploration. The framework also demonstrates strong performance on downstream conditional tasks—reducing multi-property guided generation mean absolute error by 52.4% over baselines and improving scaffold extension validity from 50.8% to 83.9%—but the benefits are bounded by the base model's capacity to reconstruct the target chemical space, as shown by the dramatic reconstruction failure rates of prior atom encodings on charged and aromatic species.

2. Context and Motivation

The Core Problem: Graph-Based Molecular Generation Cannot Match 1D Models in Validity

The fundamental question driving this work is surprisingly straightforward to state but has proven stubbornly resistant to solution: Can graph-based molecular generative models produce molecules that are as chemically valid as those from 1D sequence models, while preserving their advantage in structural novelty?

To understand why this matters, we need to step back and recognize that there are two dominant paradigms for representing molecules in generative AI. The first treats molecules as linear strings—most commonly using the SMILES notation (Weininger, 1988), which encodes atoms, bonds, branches, and rings as a sequence of characters following a deterministic grammar. Models operating in this space inherit the full machinery of modern language modeling: transformers, masked language modeling objectives, and well-understood training dynamics. The second paradigm treats molecules as graphs—nodes representing atoms, edges representing bonds—and applies graph neural networks or graph-aware transformers. This is a more natural representation of molecular topology, since molecules are fundamentally non-linear structures with connectivity patterns that are awkwardly flattened into strings.

For years, the field has been stuck in an uncomfortable tradeoff captured vividly in Figure 1 of the paper. 1D sequence models achieved high validity scores—approaching 99% in some cases—because their training objective (next-token prediction or masked token reconstruction) was well-aligned with the task of producing syntactically valid SMILES strings. But these models paid a price: their novelty suffered. They tended to memorize subsequences from the training set, reproducing common structural motifs rather than genuinely exploring new chemical space. As Figure 1 (right) shows, sequence models cluster in a region of the quality-novelty tradeoff curve far below the training set's frontier—they are valid but not meaningfully novel.

Graph diffusion models occupied the opposite corner. By operating directly on atoms and bonds, they internalized topological principles that allowed them to generalize beyond the training distribution. DiGress (Vignac et al., 2022), DisCo (Xu et al., 2024), Cometh (Siraudin et al., 2024), and DeFoG (Qin et al., 2024) all demonstrated the ability to generate novel structural scaffolds. But their validity was substantially worse—DiGress achieved only 87.1% validity on MOSES, meaning roughly one in every eight generated molecules was chemically impossible. This is a devastating failure rate for any downstream application: a drug discovery pipeline cannot afford to waste computational resources on molecules that violate basic chemistry.

The paper's explicit framing of this gap in Section 1 captures the research challenge:

"Can we leverage the inductive biases of graph modeling to match the validity of sequence models while maintaining their superior capacity for structural novelty?"

Why This Gap Is Significant: Validity Is Non-Negotiable in Practice

The validity-novelty tradeoff is not merely an academic benchmark issue—it has direct consequences for real-world molecular design workflows.

Drug discovery and materials design are needle-in-a-haystack problems. The chemical space of drug-like molecules is estimated at $10^{23}$ to $10^{60}$ possible compounds (Dobson, 2004). A generative model's job is to propose candidates from within this space that satisfy multiple constraints: synthetic accessibility, drug-likeness, target binding affinity, metabolic stability, and low toxicity. If the model produces invalid proposals—structures with pentavalent carbons, physically impossible ring systems, or atoms with incorrect formal charges—every one of those proposals is wasted computation. In a high-throughput virtual screening pipeline where millions of candidates might be generated, a 13% invalidity rate (as in DiGress) means hundreds of thousands of wasted samples. Worse, if invalid molecules slip through filtering stages, they could waste expensive downstream computation (docking simulations, synthetic feasibility assessment) or even lead to attempted synthesis of impossible compounds.

The problem is asymmetric. It is easier to increase novelty from a valid baseline (e.g., by adding exploration mechanisms to a sequence model) than to increase validity from a novel baseline (e.g., by patching a graph model's chemical errors). This is because validity constraints are hard constraints—the model must learn the precise rules of chemical bonding, valence, aromaticity, and formal charge assignment. Novelty, by contrast, is a soft property: any departure from the training distribution counts as novel, even if the departure is chemically meaningless. A model generating random graph structures might score highly on novelty while failing every validity check. This asymmetry means that validity should be the primary target for graph-based methods; achieving it without sacrificing novelty is the fundamental challenge.

Practical deployment demands both. In scaffold extension—where a chemist provides a core molecular framework and asks the model to grow it into a complete drug candidate—the model must produce valid extensions that maintain synthetic tractability. In multi-property guided generation—where the model must satisfy quantitative targets for drug-likeness, solubility, molecular weight, and synthetic accessibility simultaneously—invalid outputs cannot even be evaluated on property conditioning metrics, rendering the guidance mechanism meaningless for a fraction of generated samples. The paper's scaffold extension results (Table 4) illustrate the real cost: DiGress achieves only 50.8% validity, meaning nearly half of its scaffold completions are chemically garbage. MolHIT's 83.9% directly translates to more useful output per computation.

Where Prior Approaches Fall Short: Two Specific Limitations Identified

The paper does not simply observe that graph models underperform sequence models. It identifies two specific, previously unrecognized technical limitations that explain the gap and motivate its solutions. These are not general complaints about graph models being hard to train; they are precise, falsifiable claims about what is broken.

Limitation 1: Uniform and Absorbing Transition Kernels Ignore Chemical Group Structure

Existing discrete diffusion models for graphs—regardless of whether they use uniform, marginal, or absorbing transitions—treat each atom category as an independent, equally distant category in the diffusion space. Under a uniform transition (Equation 3), the noising process interpolates between the clean atom type and a uniform distribution over all $K$ categories. This means that a carbon atom being noised has equal probability of transitioning to fluorine, bromine, or nitrogen—despite the fact that carbon and nitrogen share far more chemical similarity (both are period-2 elements that form covalent bonds, can participate in aromatic systems, and commonly appear in drug-like scaffolds) than carbon and bromine (which differ in size, electronegativity, and bonding patterns).

The authors state this explicitly in Section 1:

"current uniform or absorbing transition treats each atom category as an independent category, even though there is well known chemical relationship that some atoms are easier to be replaced with another. Neglecting well-established domain priors often makes the learning unnecessarily hard, especially in molecular settings where high-quality molecule data is scarce."

Why does this matter for learning? During training, the denoising model must learn to predict the original clean atom from a noised version. If the noising process randomly scatters atoms across unrelated categories, the denoiser's task is essentially a $K$-way classification problem with no structural hints. When $K$ is small (MOSES has only 7 atom types in the standard encoding), this is manageable. But when the vocabulary is expanded to capture fine-grained chemical roles (aromatic vs. aliphatic, charged vs. neutral), $K$ grows (to 12 in MOSES DAE, 56 in GuacaMol DAE), and the classification problem becomes substantially harder. The proposed HDDM instead introduces intermediate states—chemically meaningful groups of atoms—that create a coarse-to-fine denoising trajectory: first determine whether the atom is a halogen, a chalcogen, or an aromatic species, then refine to the specific element. This is structurally analogous to how humans classify objects (animal → mammal → dog → golden retriever) and reduces the learning burden by decomposing a hard $K$-way problem into a sequence of easier sub-problems.

The authors do not claim that prior work was unaware that atoms have chemical relationships. Rather, they identify that no existing discrete diffusion framework incorporated this hierarchical structure into the transition kernel itself. Marginal transitions (as used in Cometh) use the empirical data distribution as the noise prior, which implicitly captures some atom frequency information, but this is a global statistic—it doesn't encode pairwise substitutability relationships. Two atoms that are both rare might have equal marginal probabilities but be chemically interchangeable or chemically orthogonal; the marginal prior doesn't distinguish these cases.

Limitation 2: Naïve Atom Encodings Make the Generation Task Ill-Posed

The second limitation is more surprising and arguably more fundamental. The paper identifies that standard atom encoding schemes used by all prior graph diffusion models create an ill-posed reconstruction problem—and this reconstruction failure directly explains the validity gap.

Here is the issue. In standard encoding, each atom is represented by its atomic number: carbon is a single token, nitrogen is a single token, and so on. But a single atomic element can exist in multiple chemically distinct states:

• Aromatic vs. aliphatic: A carbon in a benzene ring (aromatic, typically denoted 'c' in SMILES) has fundamentally different bonding and electronic properties from a carbon in a saturated hydrocarbon chain (aliphatic, denoted 'C'). They participate in different reaction types, have different hybridization, and contribute differently to molecular properties like lipophilicity.
• Formal charge: A nitrogen atom can be neutral (N), positively charged (N+), or negatively charged (N-), with each form having completely different bonding patterns—N+ is tetravalent, analogous to carbon; N- is divalent, analogous to oxygen.
• Hydrogen saturation: An aromatic nitrogen can be hydrogen-bearing (denoted [nH] in SMILES, as in pyrrole) or hydrogen-free (denoted 'n', as in pyridine). The presence or absence of this hydrogen determines the ring's aromaticity pattern, hydrogen bond donor/acceptor character, and overall molecular properties.

When a model uses only atomic-number encoding, the mapping from token to chemical state is one-to-many: the token 'N' could refer to neutral aliphatic nitrogen, aromatic pyridine-type nitrogen, or (in principle) charged nitrogen forms. The model has no way to know which is intended from the token alone. This means that the reconstruction task—predicting the original clean molecule from a noised version—is fundamentally ambiguous. Given a noisy graph at timestep $t$, the denoiser must reconstruct atoms that have multiple valid ground-truth interpretations.

The paper demonstrates this failure concretely in Figure 3 (left). When the authors measure reconstruction success rate on the MOSES dataset—the fraction of molecules that can be perfectly reconstructed from their noisy versions by the denoising model—standard encoding achieves dramatically lower rates than DAE, particularly for molecules containing pyrrolic nitrogen ([nH]). The standard encoding simply cannot distinguish between 'n' and '[nH]' when reconstructing from noise, because both map to the same coarse token in the vocabulary.

The consequences extend to generation. Figure 3 (right) shows that prior graph models trained with standard encoding under-generate pyrrolic nitrogen motifs compared to the training distribution. This is not because the models don't understand the training data—it's because the ill-posed reconstruction problem prevents them from learning to reliably produce these motifs. The model learns a diffuse posterior over the ambiguous token that averages across multiple chemical states, resulting in a generative distribution that doesn't match the training data. This is a form of representational bias: the model is structurally incapable of representing the training distribution accurately because its encoding scheme has collapsed distinct chemical states into a single symbol.

The pathology is even more dramatic on the GuacaMol dataset (Figure 5), where molecules with formal charges are present. The reconstruction failure rate for charged atoms is severe under standard encoding, and this directly translates to the generative model almost never producing charged molecules—despite these molecules being abundant in the training set.

How This Paper Positions Itself

Building Directly on DiGress's Architecture, Replacing Its Core Mechanisms

MolHIT is not a completely new architecture from scratch. The paper explicitly uses the graph transformer architecture from DiGress (Vignac et al., 2022) with the same model size (12 transformer blocks, ~16.2M parameters) and the same dual prediction of node and edge features. This is a deliberate choice: by keeping the architecture constant, the paper isolates the contributions of its two novel components (HDDM and DAE) as the direct cause of performance improvements, rather than confounding them with architectural innovations.

What changes are the forward diffusion process (from uniform or marginal transitions to hierarchical transitions for atoms) and the token vocabulary and encoding scheme (from coarse atomic-number encoding to decoupled atom encoding). These are conceptually lightweight modifications—they don't increase model parameters or inference time—but they fundamentally reshape the learning problem by (1) providing a chemically meaningful noise schedule and (2) making the reconstruction task well-posed.

Reframing the Validity-Novelty Tradeoff as a Problem of Representational Adequacy

The paper's key rhetorical move is to argue that prior graph models' validity failures are not due to inherent limitations of graphs as a representation, but rather due to inadequate encoding and noising schemes that failed to capture the true chemical structure of the data. In other words, the failure wasn't that graph models couldn't learn chemistry—it was that they were given an impoverished representation of chemistry to learn from.

This reframing is significant because it changes the research agenda. If graph models are fundamentally limited, the right response is to abandon them in favor of sequence models or hybrid approaches. If they are merely using suboptimal representations, the right response is to fix the representation. The paper provides evidence for the latter by showing that DAE alone—applied to existing DiGress architecture—raises validity from 87.1% to 96.2% (Table 5, "DiGress + DAE" row). This is a +9.1 percentage point improvement from fixing the encoding alone, before any changes to the diffusion process. It strongly supports the claim that representation, not architecture, was the bottleneck.

A Bridge Between Sequence and Graph Models That Exploits the Best of Both

The paper's Figure 1 (bottom) positions MolHIT on a Pareto frontier that neither sequence models nor prior graph models could reach. Sequence models (SAFE-GPT, CharRNN) achieve high validity but low scaffold novelty—they are in the upper-left region. Prior graph models (DiGress, Cometh, DeFoG) achieve better novelty but lower quality—the lower-right region. MolHIT achieves both high quality (94.2%, close to the training set's 95.4%) and high scaffold novelty (0.39, substantially above all comparators). This is the first graph-based method to occupy this region of the tradeoff space.

The key insight is that these two axes were not in fundamental tension—they were both constrained by the same underlying problem (inadequate representation). By solving the representation problem, MolHIT improves both validity and novelty simultaneously, moving the entire tradeoff curve outward rather than sliding along it.

Theoretical Grounding in a Provably Valid ELBO

Unlike some prior work that introduces hierarchical diffusion as a heuristic, MolHIT provides a complete mathematical treatment. Lemma 3.1 establishes that the HDDM forward process satisfies the Chapman–Kolmogorov equation (the Markov consistency property necessary for a valid diffusion model), and Theorem 3.2 derives the continuous-time negative evidence lower bound (NELBO), proving that the training objective is a valid bound on the negative log-likelihood. This theoretical grounding is important because it means HDDM is not an ad-hoc modification—it is a principled generalization of discrete diffusion that preserves the ELBO guarantee. The generalization to arbitrary numbers of hierarchy levels (Proposition C.1 in Appendix C.1) further shows that the framework is extensible beyond the two-level case used in the paper.

Acknowledging What This Does NOT Solve

The paper is notably honest about its limitations. The reconstruction experiments (Figure 3, Figure 5) make clear that the benefits of DAE are conditional on the model having the capacity to learn the extended vocabulary—on GuacaMol, where DAE expands the vocabulary from 12 to 56 tokens, the learning problem becomes harder, and the FCD metric (which measures distributional fidelity) actually worsens for MolHIT compared to original DiGress (54.9 vs. 68.0, Table 2). The paper also explicitly states that it has not explored scaling model size or using architectural improvements beyond the base DiGress transformer, leaving open the possibility that larger models might be needed to fully exploit the expanded vocabulary on more complex datasets. This is responsible positioning: the method addresses a specific representational bottleneck but does not claim to solve all challenges in molecular generation.

3. Technical Approach

3.1 Reader Orientation

The system being built is MolHIT, a diffusion model that generates molecular graphs by gradually denoising random structures into chemically valid molecules. It solves the problem that prior graph diffusion models produce too many invalid molecules because their atom encoding loses critical chemical information and their noising process ignores natural relationships between atom types. The "shape" of the solution is twofold: (1) expand the atom vocabulary to explicitly encode aromaticity, charge, and hydrogen saturation so the model knows exactly what chemical state each atom is in, and (2) structure the diffusion process hierarchically so atoms are first denoised into broad chemical groups (halogens, chalcogens, aromatics) before being refined to specific elements, mimicking how chemists think about molecular structure.

3.2 Big-Picture Architecture (Diagram in Words)

The MolHIT system has four major components that operate in sequence during generation, and in reverse during training:

1. Decoupled Atom Encoding (DAE) — a preprocessing step that expands the atom vocabulary from 7–12 coarse atomic-number tokens to 12–56 fine-grained tokens that explicitly encode aromaticity, formal charge, and hydrogen saturation. Applied once to the training data and to all generated outputs. This ensures the model always works with chemically unambiguous tokens.

2. Hierarchical Discrete Diffusion Model (HDDM) — the core forward noising and reverse denoising process. Unlike standard diffusion that transitions atoms either uniformly across all categories or directly to a masked state, HDDM introduces intermediate group states (e.g., {N, O, S}, {F, Cl, Br}) between clean atom tokens and the final masked state. The forward process proceeds through two stages: clean atoms → group identities → full masking. The reverse process reconstructs atoms in a coarse-to-fine manner: first decide the chemical group, then pick the specific element within that group.

3. Graph Transformer Backbone (from DiGress) — a permutation-equivariant graph transformer with 12 layers, ~16.2M parameters, that takes a noisy molecular graph $G_t = (X_t, E_t)$ (noisy atom tokens and noisy bond types) at timestep $t$ and predicts the clean atom types $p_\theta(X_0 \mid G_t, t)$ and clean bond types $p_\theta(E_0 \mid G_t, t)$. The architecture is identical to DiGress except for conditional variants that add adaptive layer normalization.

4. Project-and-Noise (PN) Sampler — an inference-time sampling procedure that improves diversity over standard posterior sampling. At each denoising step, rather than sampling from the analytically derived posterior $q(G_{t-\Delta t} \mid G_t, \hat{G}_0)$, the PN sampler projects the model's soft prediction $\hat{G}_0$ to a hard one-hot sample via categorical sampling (with optional temperature and top-p truncation for atoms), then re-noises this discrete candidate forward to timestep $t-\Delta t$ using the known forward kernel. This bypasses the posterior's tendency to stay close to the current noisy graph, encouraging exploration.

Information flows through these components as follows: During training — a clean molecule passes through DAE to get its decoupled token representation → HDDM's forward process (Equation 10) adds noise to both atoms and edges over random timesteps → the noisy graph enters the graph transformer → the transformer predicts clean atoms and edges → cross-entropy loss penalizes errors against the ground-truth clean tokens. During generation — a random graph is initialized from the masking prior → the PN sampler iteratively predicts clean tokens, samples them, and re-noises backward through HDDM's reverse process → the final clean tokens are decoded via DAE back to molecular structures → RDKit validation checks chemical correctness.

3.3 Roadmap for the Deep Dive

What follows is a comprehensive technical walkthrough of MolHIT, ordered to build understanding incrementally:

• First, the Decoupled Atom Encoding (DAE) — because every other component operates on DAE tokens, and the encoding choice fundamentally determines what the model learns. We'll examine exactly what tokens are created, why the prior encoding failed, and how DAE resolves the one-to-many mapping problem.

• Second, the Hierarchical Discrete Diffusion Model (HDDM) — the mathematical core of the paper. We'll build up from the standard discrete diffusion framework, motivate why hierarchical transitions help, derive the forward process (Lemma 3.1), and examine the training ELBO (Theorem 3.2). We'll also cover the grouping strategy that determines which atoms share intermediate states.

• Third, the forward and reverse process of MolHIT — how HDDM is specialized for molecular graphs, including the choice to use HDDM only for atoms (not bonds), the specific diffusion schedules, and the computation of the loss.

• Fourth, the PN Sampler and temperature sampling — the inference-time mechanisms that control the quality-diversity tradeoff and enable the high validity numbers reported in the experiments. We'll examine why standard posterior sampling was insufficient and how the PN sampler addresses that limitation.

• Fifth, the conditional modeling extensions — how the graph transformer is modified with adaptive layer normalization to accept property conditioning vectors, and how classifier-free guidance is used during conditional generation.

• Sixth, we'll step back and examine why these design choices matter together — the ablation study logic that shows DAE, HDDM, and the PN sampler each contribute independently to the final performance.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a methodological innovation paper whose core idea is that molecular graph generation with discrete diffusion can be dramatically improved by two complementary changes: (1) expanding the atom vocabulary to make the reconstruction task well-posed rather than ambiguous, and (2) structuring the diffusion noising process hierarchically so that chemically related atoms share intermediate states during denoising. The graph transformer architecture is inherited from prior work; the innovations are in the representation and the diffusion dynamics.

---

Decoupled Atom Encoding (DAE)

Before explaining how DAE works, we must understand why the standard encoding fails. This failure is not a matter of degree—it is a categorical failure of representation that makes the generative task ill-posed.

The one-to-many mapping problem. In standard encoding, atoms are represented by their atomic number only. On the MOSES dataset, this yields 7 tokens: {C, N, S, O, F, Cl, Br}. The problem is that a single element can exist in multiple chemically distinct states that this encoding cannot distinguish:

• Aromaticity: Carbon in a benzene ring ('c' in SMILES notation) has $sp^2$ hybridization, participates in a delocalized $\pi$ system, and has bond lengths of ~1.40 Å. Aliphatic carbon ('C') has $sp^3$ or $sp^2$ hybridization depending on context, but does not participate in aromatic delocalization. These are fundamentally different chemical environments—yet standard encoding collapses both to a single token.

• Formal charge: Neutral nitrogen (N) is trivalent with a lone pair. Positively charged nitrogen ($\text{N}^+$) is tetravalent, analogous to carbon. Negatively charged nitrogen ($\text{N}^-$) is divalent, analogous to oxygen. Each form has completely different bonding rules—an $\text{N}^+$ must have four bonds, an $\text{N}^-$ must have two bonds and two lone pairs. Collapsing these to one token means the denoiser cannot determine from the token alone what valency to enforce.

• Hydrogen saturation on aromatic nitrogens: Pyrrolic nitrogen ([nH]) carries an explicit hydrogen and contributes two electrons to the aromatic $\pi$ system. Pyridine-type nitrogen (n) carries no hydrogen and contributes one electron. The presence of this hydrogen changes the ring's aromaticity count, hydrogen bond donor capability, and synthetic accessibility. Standard encoding cannot distinguish [nH] from n.

The reconstruction experiment in Figure 3 (left) quantifies the consequences. The authors measure reconstruction success rate—the fraction of molecules that can be perfectly recovered after being encoded, noised, and then decoded by the model. For molecules containing pyrrolic nitrogen motifs, standard encoding achieves a dramatically lower success rate. The model literally cannot reconstruct what it was given because the encoding has destroyed the information needed to distinguish chemical states.

This is what makes the generative task ill-posed. During diffusion training, the model learns to predict $p_\theta(X_0 \mid G_t)$—the clean atoms given a noisy graph. If a noisy token at timestep $t$ could correspond to multiple chemically distinct clean tokens (e.g., 'n' vs. '[nH]'), there is no unique ground truth. The model must learn a posterior distribution over ambiguous tokens, and it will inevitably average across possibilities. In generation, this averaging manifests as a systematic under-production of motifs that are rare in the ambiguous category—exactly what Figure 3 (right) shows for pyrrolic nitrogen.

What DAE does differently. DAE resolves the ambiguity by making the encoding one-to-one with chemical state. Each unique combination of (element, aromaticity, formal charge, hydrogen count) becomes its own token. On MOSES, the 7 coarse tokens expand to 12 fine-grained tokens:

On GuacaMol, the expansion is more dramatic because the dataset includes charged species. The 12 elemental types expand to 56 tokens, including: $\text{C}^+$, $\text{C}^-$, $\text{N}^+$, $\text{NH}^+$, $\text{NH}_2^+$, $\text{NH}_3^+$, $\text{N}^-$, $\text{NH}^-$, $\text{O}^+$, $\text{O}^-$, and aromatic variants thereof (c+, c-, n+, n-, s+, o+, etc.). The full vocabulary is provided in Table 7 of the paper.

The critical design choice is that DAE encodes these attributes as primary node features rather than attempting to infer them from bond context. Prior models implicitly relied on RDKit's valence rules to determine hydrogen counts and formal charges from the bonding pattern after generation. This post-hoc inference is lossy and error-prone—as Figure 5 shows, it fails to correctly reconstruct charged species in GuacaMol 20% of the time even with relaxation. DAE instead makes these properties explicit in the token identity, so the model must learn to generate them correctly as part of the denoising objective. The model sees '[nH]' as a distinct category from 'n', learns their different co-occurrence patterns with bond types and neighboring atoms, and generates them with the correct frequencies.

Why this is not equivalent to adding explicit hydrogen nodes. A natural question is whether DAE is just a roundabout way of modeling hydrogens explicitly. The paper explicitly distinguishes these approaches: "tokenizing [nH] as a distinct state is fundamentally different from modeling explicit hydrogen atoms as separate nodes." Adding explicit hydrogen nodes would dramatically increase the graph size (each heavy atom would need 1–4 hydrogen neighbors), making the $O(n^2)$ attention in the graph transformer substantially more expensive. DAE keeps the graph sparse—the same number of nodes as heavy atoms—while encoding hydrogen information in the node features. This preserves computational efficiency while adding chemical precision.

The empirical evidence for DAE's importance. The ablation in Table 5 shows that DAE alone, applied to the base DiGress architecture with no other changes, raises validity from 87.1% to 96.2%—a +9.1 percentage point improvement. This is the single largest jump in the incremental component analysis. It confirms that representation was the primary bottleneck, not the architecture or the diffusion process. The reconstruction experiments in Figures 5 and 6 further confirm that DAE enables the model to learn the full distribution of charged and aromatic species, which prior models systematically failed to capture.

---

Hierarchical Discrete Diffusion Model (HDDM)

HDDM is a principled generalization of discrete diffusion from a single-level state space to a multi-level hierarchy. To understand it, we must first review standard discrete diffusion, then see exactly where HDDM extends it.

Standard discrete diffusion recap. In discrete diffusion (Section 2.1), we have $K$ categories in a state space $\mathcal{S}$. The forward process corrupts clean data $x_0$ through a Markov chain:

$q(x_t \mid x_{t-1}) = \text{Cat}(x_t; x_{t-1} Q_t)$

where $x_t$ is the noisy state at time $t$, $Q_t$ is a $K \times K$ row-stochastic transition matrix, and $\text{Cat}(\cdot; v)$ denotes a categorical distribution with probability vector $v$. The cumulative forward transition from time 0 to time $t$ is:

$q(x_t \mid x_0) = \text{Cat}(x_t; x_0 \bar{Q}_t), \quad \bar{Q}_t = Q_t Q_{t-1} \cdots Q_1$

where $\bar{Q}_t$ is the product of per-step transition matrices, computable in closed form for the standard transition types (uniform, marginal, absorbing).

Three types of transitions are commonly used (Section 2.1):

• Uniform transition (Equation 3): $q(x_t \mid x_0) = \text{Cat}(x_t; (1 - \bar{\alpha}_t) \frac{1}{K} \mathbf{1}^T + \bar{\alpha}_t x_0)$, which interpolates between the clean one-hot $x_0$ and a uniform distribution over all $K$ categories. The scalar $\bar{\alpha}_t$ is a monotonic decreasing schedule from 1 to 0.

• Absorbing transition (Equation 4): $q(x_t \mid x_0) = \text{Cat}(x_t; \bar{\alpha}_t x_0 + (1 - \bar{\alpha}_t) e_{\mathbf{m}})$, which interpolates between $x_0$ and a single absorbing masked state $\mathbf{m}$. This is analogous to masking in BERT-style training.

• Marginal transition: $q(x_t \mid x_0) = \text{Cat}(x_t; \bar{\alpha}_t x_0 + (1 - \bar{\alpha}_t) \pi)$, where $\pi$ is the empirical data distribution (the marginal frequency of each category in the training set), a closer approximation to the true prior than a uniform distribution.

The limitation HDDM addresses. In all three transition types, the forward process treats every category as equally distant from every other category. Under absorbing diffusion, for example, the only possible transition is toward the masked state—there is no notion that carbon and nitrogen are more similar than carbon and bromine. The denoiser must learn to recover $x_0$ from partially masked data without any structural hints about which atom types are plausible given the partial information. On small vocabularies, this is feasible. On the expanded DAE vocabulary (12–56 tokens), the learning problem becomes substantially harder because the model must distinguish 56 categories with no intermediate structure.

The HDDM state space. HDDM introduces additional categories that serve as intermediate waypoints in the noising process. The total state space $\mathcal{T}$ with dimensionality $D = K + G + 1$ is partitioned into three disjoint subsets:

• $\mathcal{S}_0$: the $K$ clean atom states (the standard vocabulary)
• $\mathcal{S}_1$: $G$ mid-level group states that represent chemically meaningful clusters of atoms
• $\mathcal{S}_2 = \{m\}$: a single absorbing masked state, identical to the standard absorbing diffusion prior

The forward process now proceeds in two stages: first, clean states in $\mathcal{S}_0$ transition to their assigned group state in $\mathcal{S}_1$; second, all states in $\mathcal{S}_0 \cup \mathcal{S}_1$ transition to the masked state in $\mathcal{S}_2$.

The group mapping operator $\Phi$. The transition from $\mathcal{S}_0$ to $\mathcal{S}_1$ is governed by a row-stochastic matrix $\Phi \in [0, 1]^{K \times G}$, where $\Phi_{ij}$ is the probability that clean state $i \in \mathcal{S}_0$ maps to group $j \in \mathcal{S}_1$. In the paper, $\Phi$ is designed to be deterministic—each clean atom maps to exactly one group with probability 1. The specific groupings for MOSES are:

• Group 1: {C} — carbon alone, because it is the most abundant element and forms the backbone of most organic molecules
• Group 2: {N, O, S} — the chalcogens and pnictogens that commonly appear in drug-like heterocycles
• Group 3: {F, Cl, Br} — the halogens, which share similar electronegativity and bonding patterns
• Group 4: {c, o, n, nH, s} — all aromatic atoms, which share participation in delocalized $\pi$ systems

These groups encode explicit chemical knowledge: atoms within a group are more substitutable for each other than atoms across groups. A chlorine atom is chemically more similar to a bromine atom (both halogens, similar size and electronegativity trends) than to a nitrogen atom—and HDDM's hierarchical structure reflects this by having Cl and Br share a group state.

The two transition matrices. The paper defines two transition matrices on the full space $\mathcal{T}$:

$Q^{(1)} = \begin{bmatrix} \mathbf{0}_{K \times K} & \Phi & \mathbf{0}_{K \times 1} \\ \mathbf{0}_{G \times K} & \mathbf{I}_G & \mathbf{0}_{G \times 1} \\ \mathbf{0}_{1 \times K} & \mathbf{0}_{1 \times G} & 1 \end{bmatrix}$

where $Q^{(1)}$ maps clean states $\mathcal{S}_0$ to their group assignments in $\mathcal{S}_1$ (via $\Phi$), while states already in $\mathcal{S}_1$ remain there (the identity block $\mathbf{I}_G$), and the masked state is absorbing. Similarly:

$Q^{(2)} = \begin{bmatrix} \mathbf{0}_{(K+G) \times (K+G)} & \mathbf{1}_{(K+G) \times 1} \\ \mathbf{0}_{1 \times (K+G)} & 1 \end{bmatrix}$

where $Q^{(2)}$ maps all states in $\mathcal{S}_0 \cup \mathcal{S}_1$ to the single masked state in $\mathcal{S}_2$ (the column of ones), while the masked state remains absorbing.

What this means concretely: $Q^{(1)}$ is the "group identity" transition—a carbon atom becomes group 1, a fluorine atom becomes group 3, etc. $Q^{(2)}$ is the "masking" transition—any atom, regardless of its clean identity or current group, becomes fully masked. This two-stage forward process means that at intermediate noise levels, the model sees atoms that have been reduced to their group identity but not yet fully masked—providing a coarse chemical signal that is easier to denoise than full masking.

The HDDM forward process (Lemma 3.1). The key mathematical contribution that makes HDDM a valid diffusion model is Lemma 3.1, which provides a closed-form transition kernel satisfying the Chapman–Kolmogorov equation. The forward process from time $s$ to time $t$ (with $s \leq t$) is:

$Q_{t|s} = \alpha_{t|s} \mathbf{I} + (\beta_{t|s} - \alpha_{t|s}) Q^{(1)} + (1 - \beta_{t|s}) Q^{(2)}$

where $\alpha_{t|s} := \alpha_t / \alpha_s$ and $\beta_{t|s} := \beta_t / \beta_s$, and $\alpha_t, \beta_t$ are two diffusion schedules (monotonic decreasing functions from 1 to 0) satisfying the ordering constraint $\alpha_t \leq \beta_t$ for all $t$.

Interpreting the three terms in $Q_{t|s}$:

• $\alpha_{t|s} \mathbf{I}$: with weight $\alpha_{t|s}$, the state stays exactly as it was at time $s$ (no noise added). This term dominates at early times when $\alpha_t$ is close to 1.

• $(\beta_{t|s} - \alpha_{t|s}) Q^{(1)}$: with weight $\beta_{t|s} - \alpha_{t|s}$, the state transitions according to $Q^{(1)}$—that is, clean states move to their group identities, but states already in groups or masked remain unchanged. This term is active in the intermediate regime where $\beta_{t|s} > \alpha_{t|s}$.

• $(1 - \beta_{t|s}) Q^{(2)}$: with weight $1 - \beta_{t|s}$, the state transitions according to $Q^{(2)}$—that is, everything becomes masked. This term dominates at late times when $\beta_t$ is close to 0.

The design of having two schedules ($\alpha_t$, $\beta_t$) rather than one is what enables the hierarchical structure. A standard absorbing diffusion has only one schedule $\bar{\alpha}_t$, and the forward process is simply $Q_t = \bar{\alpha}_t \mathbf{I} + (1 - \bar{\alpha}_t) Q^{(2)}$—clean-to-masked with nothing in between. HDDM inserts the intermediate $Q^{(1)}$ term by having $\beta_t$ decay faster than $\alpha_t$, creating a window where $(\beta_{t|s} - \alpha_{t|s})$ is positive and the group transition is active.

Why this satisfies Chapman–Kolmogorov. The proof (Proposition C.1) relies on the fact that $Q^{(1)}$ and $Q^{(2)}$ are projection operators satisfying $Q^{(i)} Q^{(j)} = Q^{\max(i,j)}$. This means that applying $Q^{(1)}$ followed by $Q^{(2)}$ is the same as applying $Q^{(2)}$ directly—once you're masked, you stay masked; once you're in a group, applying the group transition again doesn't change anything. This algebraic property ensures that the product of two $Q_{t|s}$ matrices of the form in Lemma 3.1 yields another matrix of the same form, which is exactly what the Chapman–Kolmogorov equation $Q_{t|s} Q_{s|r} = Q_{t|r}$ requires.

The simplified cumulative forward transition from time 0 to $t$ is:

$Q_t = \alpha_t \mathbf{I} + (\beta_t - \alpha_t) Q^{(1)} + (1 - \beta_t) Q^{(2)}$

What this means for the noising trajectory in plain language: Starting from a clean atom at $t=0$, the forward process does the following over time (see Figure 2a for the Markov chain visualization):

1. Early timesteps ($\alpha_t \approx 1$, $\beta_t \approx 1$): The atom stays in its clean state $\mathcal{S}_0$ with high probability. The group transition is negligible.

2. Intermediate timesteps ($\beta_t$ has decayed, but $\alpha_t$ is still moderate): The atom transitions to its assigned group state in $\mathcal{S}_1$ with probability $\beta_t - \alpha_t$. This is the hierarchical stage—the specific element identity is lost, but the chemical group (halogen, chalcogen, aromatic, etc.) is preserved.

3. Late timesteps ($\beta_t \approx 0$): Whatever state the atom is in (clean or group), it transitions to the fully masked state $\mathcal{S}_2 = \{m\}$ with probability $1 - \beta_t$. This is the final absorbing stage.

During the reverse (generation) process, this trajectory is traversed backward (Figure 2b): from fully masked, the model first predicts which chemical group each atom belongs to; then, given the group, it predicts the specific element within that group. This coarse-to-fine denoising is HDDM's core mechanism.

The NELBO and training objective (Theorem 3.2). The paper derives the continuous-time negative evidence lower bound (NELBO) for HDDM, which provides the theoretically justified training objective. For the case where $\Phi$ is deterministic (each clean atom maps to exactly one group), the NELBO simplifies to:

$\mathcal{L}_{\text{NELBO}}^{\infty}(\theta) = \mathbb{E}_{Q, t} \left[ \frac{\alpha_t (\beta_t' - \alpha_t')}{\beta_t - \alpha_t} \log \langle \mathbf{x}_\theta, \mathbf{x} \rangle \cdot \mathbb{I}[\mathbf{z}_t \in \mathcal{S}_1] + \frac{\beta_t' (\beta_t - \alpha_t)}{\beta_t (1 - \beta_t)} \log \langle Q^{(1)} \mathbf{x}_\theta, Q^{(1)} \mathbf{x} \rangle \cdot \mathbb{I}[\mathbf{z}_t = \mathbf{m}] + \frac{\alpha_t \beta_t'}{\beta_t (1 - \beta_t)} \log \langle \mathbf{x}_\theta, \mathbf{x} \rangle \cdot \mathbb{I}[\mathbf{z}_t = \mathbf{m}] \right] + C$

where:

• $\mathbf{x} \in \{0, 1\}^K$ is the one-hot encoding of the ground-truth clean atom type
• $\mathbf{x}_\theta(\mathbf{z}_t, t) \in \Delta^{K-1}$ is the denoiser's predicted probability distribution over clean atom types, given the noisy state $\mathbf{z}_t$ at time $t$
• $\langle \mathbf{x}_\theta, \mathbf{x} \rangle$ is the inner product, i.e., the predicted probability assigned to the true clean atom type
• $\langle Q^{(1)} \mathbf{x}_\theta, Q^{(1)} \mathbf{x} \rangle$ is the inner product after projecting both vectors through the deterministic group mapping $Q^{(1)}$—this is the predicted probability assigned to the correct group of the true clean atom
• $\mathbb{I}[\mathbf{z}_t \in \mathcal{S}_1]$ is 1 when the noisy token is in a group state, 0 otherwise
• $\mathbb{I}[\mathbf{z}_t = \mathbf{m}]$ is 1 when the noisy token is fully masked, 0 otherwise
• $\beta_t'$ and $\alpha_t'$ are time derivatives of the schedules
• $C$ is a constant independent of the model parameters

What this loss computes operationally: The NELBO decomposes into three terms corresponding to three noise regimes:

1. Group-state term (first line): When the noisy token $\mathbf{z}_t$ is in a group state $\mathcal{S}_1$, the loss is $-\log \langle \mathbf{x}_\theta, \mathbf{x} \rangle$—the model must predict which specific clean atom generated this group, weighted by $\frac{\alpha_t (\beta_t' - \alpha_t')}{\beta_t - \alpha_t}$. This is the "denoise from group to clean atom" term, active when the atom has been reduced to its group but not yet masked.

2. Masked-state group term (second line): When the noisy token is fully masked $\mathbf{m}$, this term measures whether the model correctly predicts which group the clean atom belongs to, via $-\log \langle Q^{(1)} \mathbf{x}_\theta, Q^{(1)} \mathbf{x} \rangle$, weighted by $\frac{\beta_t' (\beta_t - \alpha_t)}{\beta_t (1 - \beta_t)}$. This is the coarse prediction—even when fully masked, the model should get the chemical group right.

3. Masked-state clean term (third line): When the noisy token is fully masked $\mathbf{m}$, this term measures whether the model correctly predicts the specific clean atom type, via $-\log \langle \mathbf{x}_\theta, \mathbf{x} \rangle$, weighted by $\frac{\alpha_t \beta_t'}{\beta_t (1 - \beta_t)}$. This is the fine-grained prediction on top of the group prediction.

Sanity check: When $\beta_t = \alpha_t$ (i.e., no intermediate group stage), the first two terms vanish (because $\beta_t - \alpha_t = 0$ and $\beta_t' - \alpha_t' = 0$), and the third term reduces to the standard masked diffusion loss. HDDM is thus a strict generalization: it reduces to standard absorbing diffusion when the group stage is removed.

Why the paper uses the standard cross-entropy loss instead of the NELBO. The authors empirically find that "regularization loss in Eq. 9 does not improve the performance, so we take the original loss in Eq. 6." This means they train with a simple weighted cross-entropy:

$\mathcal{L}_\theta = \mathbb{E}_{t, G_t \sim q(\cdot \mid G_0)} \left[ \sum_{i=1}^n -\log p_\theta^X(X_{0,i} \mid G_t, t) + \lambda \sum_{1 \leq i < j \leq n} -\log p_\theta^E(E_{0,ij} \mid G_t, t) \right]$

where $\lambda > 0$ is a weighting factor balancing node and edge loss contributions, and $p_\theta^X$ and $p_\theta^E$ are the model's predicted probability distributions over clean atom types and bond types respectively. This is the same loss form used in DiGress, but the noise distribution $q(\cdot \mid G_0)$ is now given by the HDDM forward process (Equation 10) rather than a uniform or marginal transition.

The decision to use the simple cross-entropy loss rather than the full NELBO weighting is practical: the NELBO would require computing the three separate terms with their time-dependent coefficients and switching based on whether each token is in a group state versus fully masked. The simpler loss trains on the same noisy data distribution but with uniform weighting across noise levels, which the authors find empirically sufficient.

Generalization to arbitrary hierarchies (Proposition C.1). Appendix C.1 proves that HDDM extends naturally to $n+1$ hierarchical levels with $n$ transition matrices $Q^{(1)}, Q^{(2)}, \ldots, Q^{(n)}$ and $n$ diffusion schedules $\alpha_t^{(1)}, \alpha_t^{(2)}, \ldots, \alpha_t^{(n)}$ satisfying $0 \leq \alpha_t^{(1)} \leq \cdots \leq \alpha_t^{(n)} \leq 1$. The forward process generalizes to:

$Q_{t|s} = \alpha_{t|s}^{(1)} \mathbf{I} + \sum_{i=1}^{n-1} (\alpha_{t|s}^{(i+1)} - \alpha_{t|s}^{(i)}) Q^{(i)} + (1 - \alpha_{t|s}^{(n)}) Q^{(n)}$

This means one could in principle design deeper hierarchies—e.g., clean atoms → element groups → metal/nonmetal → fully masked. The paper uses only two levels ($n=2$) with the two schedules renamed as $\alpha_t^{(1)} = \alpha_t$ and $\alpha_t^{(2)} = \beta_t$, but the framework is general.

---

Forward and Reverse Process of MolHIT

MolHIT applies HDDM specifically to atom types while using a standard uniform transition for bond types. This asymmetric treatment is a deliberate design choice based on empirical observation.

The decoupled forward process (Equation 10). For a molecular graph $G = (X, E)$ with atom matrix $X \in \mathbb{R}^{n \times d_X}$ and bond matrix $E \in \mathbb{R}^{n \times n \times d_E}$, the forward process operates independently on atoms and bonds:

For atoms (HDDM): $Q_{X,t} = \alpha_{X,t} \mathbf{I} + (\beta_{X,t} - \alpha_{X,t}) Q_{X,t}^{(1)} + (1 - \beta_{X,t}) Q_{X,t}^{(2)}$

For bonds (uniform): $Q_{E,t} = \alpha_{E,t} \mathbf{I} + (1 - \alpha_{E,t}) \frac{1}{d_E} \mathbf{1}_{d_E} \mathbf{1}_{d_E}^T$

where $d_E$ is the number of bond types (typically 4: none/no bond, single, double, triple, plus aromatic in some implementations), and $\frac{1}{d_E} \mathbf{1}_{d_E} \mathbf{1}_{d_E}^T$ is the matrix that maps any state to the uniform distribution over all bond types.

What this means: Atoms undergo the hierarchical two-stage noising (clean → group → masked), while bonds undergo standard uniform noising (clean → uniform distribution over bond types). The choice to use HDDM only for atoms is motivated by the empirical observation that "a uniform transition kernel is essential for edge generation, whereas HDDM yields superior performance for atom types compared to a uniform approach." In other words, the chemical priors that make hierarchical diffusion helpful for atoms (shared group membership, substitutability patterns) do not have a natural analog for bonds—bond types are ordinal (none < single < double < triple) rather than categorical with cluster structure, making uniform diffusion more appropriate.

Diffusion schedules. The paper uses simple linear schedules for both processes. For atoms:

$\alpha_{X,t} = 1 - t, \quad \beta_{X,t} = 1 - t^2$

For bonds:

$\alpha_{E,t} = 1 - t$

with $t \in [0, 1]$. The key observation is that $\beta_{X,t}$ decays faster than $\alpha_{X,t}$ (quadratic vs. linear), creating the condition $\alpha_{X,t} \leq \beta_{X,t}$ required by Lemma 3.1. At $t=0$, both are 1 (clean). At $t=1$, both are 0 (fully noised). At intermediate times, say $t=0.5$: $\alpha_{X,0.5} = 0.5$, $\beta_{X,0.5} = 0.75$. The difference $\beta_{X,t} - \alpha_{X,t} = (1-t^2) - (1-t) = t - t^2 = t(1-t)$ is positive for $t \in (0,1)$, providing a window where the group transition $Q^{(1)}$ is active. This window is maximized at $t=0.5$ where $t(1-t) = 0.25$, meaning at the midpoint of diffusion, 25% of the probability mass goes through the group state while the rest is split between remaining clean (50%) and becoming masked (25%).

The paper notes that "our preliminary experiments show robustness on the HDDM scheduler," suggesting the exact form of the schedules is not critical as long as they satisfy the ordering constraint and provide a reasonable window for the group transition.

The grouping strategy specifics. The deterministic grouping kernel $\Phi$ for MOSES creates four groups as described earlier. For GuacaMol, the larger vocabulary (56 tokens) requires a more elaborate grouping, shown in Table 8:

• Group 1: halogens and halide ions — {F, Cl, Br, I, F$^-$, Cl$^-$, Br$^-$}
• Group 2: neutral main-group elements — {C, N, O, P, S, Se}
• Group 3: neutral aromatic atoms — {c, n, [nH], o, s, se, p}
• Group 4: cationic species — {$\text{N}^+$, $\text{n}^+$, [nH]$^+$, $\text{P}^+$, [NH]$^+$, [NH$_2$]$^+$, [NH$_3$]$^+$, $\text{Br}^{+2}$, $\text{Cl}^{+2}$, $\text{Cl}^{+3}$, $\text{I}^{+2}$, $\text{I}^{+3}$}
• Group 5: anionic species and other charged atoms (18 types including $\text{O}^-$, $\text{N}^-$, $\text{C}^+$, $\text{c}^+$, etc.)
• Group 6: silicon and boron — {B, Si}

This grouping reflects more detailed chemical knowledge: cationic species share the property of being electron-deficient and forming bonds with anions; anionic species share electron richness. The groups are chosen to balance intra-group chemical similarity with inter-group distinctness, making the hierarchical classification problem well-structured.

Training procedure. The graph transformer receives a noisy graph $G_t = (X_t, E_t)$ at a randomly sampled timestep $t$, where $X_t \sim \text{Cat}(X_t; X_0 Q_{X,t})$ and $E_t \sim \text{Cat}(E_t; E_0 Q_{E,t})$. It outputs predicted probabilities $p_\theta(X_0 \mid G_t, t)$ and $p_\theta(E_0 \mid G_t, t)$ for every node and edge. The loss is cross-entropy against the true clean tokens as given in Equation 6 of the paper, with a weighting factor $\lambda$ to balance node and edge contributions. The model is trained with gradient clipping (threshold 2.0) and Exponential Moving Average (EMA rate 0.999) for stability. Training runs for 100 epochs on MOSES, 40–50 epochs on GuacaMol.

Comparison to standard absorbing diffusion. If we were to set $\beta_t = \alpha_t$ (removing the group stage), HDDM would collapse to standard absorbing (masked) diffusion. The forward process would become simply $Q_t = \alpha_t \mathbf{I} + (1 - \alpha_t) Q^{(2)}$, which is exactly Equation 4 with $\bar{\alpha}_t = \alpha_t$. The key difference in HDDM is that at intermediate noise levels, the model sees tokens that have been reduced to their group identities rather than being partially masked—and this group-level signal provides a stronger learning signal because the group identity constrains which specific atoms are possible.

---

PN Sampler and Temperature Sampling

The reverse (generation) process must convert a fully masked random graph into a chemically valid molecule. The paper introduces a modified sampling procedure—the Project-and-Noise (PN) Sampler—that improves upon standard posterior sampling.

Why standard posterior sampling was insufficient. In standard discrete diffusion (Austin et al., 2021), the reverse step from $G_t$ to $G_{t-\Delta t}$ uses the analytically derived posterior:

$q(G_{t-\Delta t} \mid G_t, \hat{G}_0)$

where $\hat{G}_0$ is the model's prediction of the clean graph. This posterior is computed in closed form from the forward transition kernel and Bayes' rule, as derived for HDDM in Equations 21–23 (Appendix C.2). The posterior inherently pulls the denoised state toward the current noisy state $G_t$, which the paper describes as "restricting the structural exploration necessary for complex molecular generation." This means that if the model is uncertain about an atom's identity early in generation, the posterior will keep it close to its current (random) state rather than allowing it to explore alternatives.

How the PN sampler works (Algorithm 1). At each reverse step from $G_t$ to $G_{t-\Delta t}$:

1. The model predicts clean atom probabilities $p_\theta(X_0 \mid G_t, t)$ and clean bond probabilities $p_\theta(E_0 \mid G_t, t)$.

2. For atom predictions, temperature scaling and top-p (nucleus) sampling are applied to the predicted probabilities:

   $p'_0(X) = \text{TopP}\left(\text{Softmax}\left(\frac{p_\theta(X_0 \mid G_t, t)}{\tau}\right), p\right)$

   where $\tau$ is a temperature parameter (lower = sharper distribution, more deterministic) and $p$ is the nucleus threshold (only tokens whose cumulative probability exceeds $p$ are kept, all others set to zero and the distribution renormalized).

3. A discrete clean candidate $\hat{X}_0$ is sampled from this modified distribution: $\hat{X}_0 \sim \text{Categorical}(p'_0(X))$. For bonds, a clean candidate $\hat{E}_0$ is sampled directly from the model's predicted probabilities without temperature scaling.

4. Instead of computing the posterior $q(G_{t-\Delta t} \mid G_t, \hat{G}_0)$ and sampling from it, the PN sampler re-noises the clean candidate $\hat{G}_0 = (\hat{X}_0, \hat{E}_0)$ forward to timestep $t - \Delta t$ using the known forward kernel:

   $G_{t-\Delta t} \sim q(G_{t-\Delta t} \mid \hat{G}_0)$

   where $q$ is the forward process defined by Equation 10 (HDDM for atoms, uniform for bonds).

Key insight of the PN sampler: The standard posterior conditions on $G_t$ (the current noisy state) as well as $\hat{G}_0$, meaning the next state must be consistent with the current noise. The PN sampler bypasses this by projecting to a clean estimate $\hat{G}_0$ (via categorical sampling) and then re-noising directly from the clean estimate, ignoring $G_t$ entirely in the transition. This means the next state $G_{t-\Delta t}$ is constrained only by the forward process applied to the clean estimate, not by the history of noise. The result is greater diversity because the model can "change its mind" about atoms early in generation without being anchored to previous noisy states.

This is analogous to the "projection" samplers developed in the continuous-time discrete diffusion literature (Campbell et al., 2022), where the model's clean prediction is used directly rather than the analytically correct posterior. The tradeoff is that the PN sampler is not exactly reversing the forward process—it introduces additional variance that is not accounted for in the ELBO—but the empirical results show that this variance helps escape local optima during generation.

Temperature sampling for quality-diversity control. The temperature parameter $\tau$ and top-p threshold $p$ provide explicit knobs for controlling the quality-diversity tradeoff. As shown in Figure 4:

• At top-p = 1.0 (no truncation), the model achieves scaffold novelty of ~0.39 but quality around 93%.
• At top-p = 0.8, quality rises to 95.1% and validity to 99.4%, but scaffold novelty drops to ~0.32.
• At top-p below 0.6, both quality and novelty degrade sharply—the sampling becomes too deterministic and the model collapses to a narrow set of high-probability outputs.

The paper's reported results in Table 1 use top-p = 0.8 to achieve the 99.1% validity and 94.2% quality scores, representing a sweet spot in this tradeoff.

Important detail: Temperature sampling is applied only to atom predictions (line 7 of Algorithm 1), not to bond predictions. The paper states that "doing temperature sampling only for the atom prediction results in the best performance." This is chemically sensible: bond type prediction is more constrained (given two atom types, the plausible bond types are heavily restricted by valence rules), so introducing additional stochasticity hurts more than it helps. Atom type prediction has more degrees of freedom and benefits more from controlled exploration.

Number of atoms sampling. Algorithm 1 shows that the number of atoms $N$ for each generated molecule is sampled from the empirical distribution of molecule sizes in the training set ($N \sim P_{\text{train}}(N)$). This means MolHIT does not learn to predict graph size as part of the diffusion process—it samples a size first, then generates a molecule of that size. This is a standard approach in graph generation and avoids the complexity of variable-size diffusion.

---

Conditional Modeling

For downstream tasks like multi-property guided generation and scaffold extension, MolHIT includes conditional modeling capabilities.

Architecture modifications. The base graph transformer from DiGress is modified with Adaptive Layer Normalization (AdaLN) for node features:

$\text{AdaLN}(\mathbf{x}, \mathbf{C}) = (1 + \gamma(\mathbf{C})) \cdot \text{LayerNorm}(\mathbf{x}) + \beta(\mathbf{C})$

where $\mathbf{C}$ is a conditioning vector composed of a sinusoidal timestep embedding and an MLP-encoded property vector $\mathbf{c}$ (e.g., target QED, SA, logP, MW values). The functions $\gamma(\mathbf{C})$ and $\beta(\mathbf{C})$ are affine transformations (learned linear layers) of the conditioning vector that produce scale and shift parameters for the normalized node features. This allows the global context (time + desired properties) to modulate node representations at every layer.

The original DiGress architecture had a persistent global feature vector $\mathbf{y}$ updated at every layer. MolHIT's conditional variant removes this and replaces it with $\mathbf{C}$, which is computed once from the timestep and property inputs and fed to all layers. This is a simpler conditioning mechanism that the authors found sufficient.

Property conditioning inputs. For multi-property guided generation on MOSES, the conditioning vector $\mathbf{c}$ contains four continuous molecular descriptors, min-max normalized using the training split statistics (Table 13):

• QED (Quantitative Estimate of Drug-likeness): range [0.1912, 0.9484]
• SA (Synthetic Accessibility): range [1.2694, 7.4831]
• logP (Octanol-Water Partition Coefficient): range [-5.3940, 5.5533]
• MW (Molecular Weight): range [250.017, 349.999] Da

These are calculated using RDKit and the sascorer module. The property vector is dropped during training with some probability (dropout) to enable classifier-free guidance at inference time, while the timestep embedding is always present to maintain denoising stability.

Classifier-Free Guidance (CFG). At inference time, CFG (Ho and Salimans, 2022) interpolates between the conditional and unconditional model predictions:

$\tilde{p}_\theta(X_0 \mid G_t, t, \mathbf{c}) = (1 + w) \cdot p_\theta(X_0 \mid G_t, t, \mathbf{c}) - w \cdot p_\theta(X_0 \mid G_t, t)$

where $w$ is the guidance strength. The paper reports using $w = 1.0$ and notes that "increasing the guidance weight beyond unity did not consistently yield better property alignment." This is an interesting negative result—in continuous diffusion for images, higher guidance weights ($w = 3$–$7.5$) are standard, but in this discrete graph diffusion setting, the benefit saturates at $w=1$. The authors suggest that better sampler or model designs for higher guidance strength are a direction for future work.

Scaffold extension sampling. For scaffold extension (Section 4.4), the task is to complete a molecule given a fixed substructure (the scaffold). During reverse diffusion, the scaffold region is forced to follow a fixed trajectory: at each timestep $t$, the atoms and bonds belonging to the scaffold are set to their clean (ground-truth) tokens rather than being denoised. The extension region (atoms not in the scaffold) follows the standard reverse process, initialized from the prior distribution. The number of total atoms is set to match the ground-truth molecule size to isolate the generative capability from size prediction errors.

---

Why These Design Choices Matter Together: Ablation Logic

Table 5 provides the clearest evidence for how the components interact. Starting from DiGress (Quality: 82.5, FCD: 1.25, Validity: 87.1%):

1. + DAE only: Quality rises to 87.6, FCD improves to 0.89, Validity jumps to 96.2%. This is the representation fix alone—better encoding makes the learning problem well-posed, and validity improves dramatically because the model can now learn to produce aromatic and charged motifs correctly.

2. + DAE + PN Sampler: Quality further rises to 92.9, but FCD worsens to 1.65 and Validity reaches 99.4%. The PN sampler adds exploration that helps quality (more diverse valid molecules) but hurts distributional fidelity (FCD) because the increased variance in sampling pushes the generated distribution slightly away from the training manifold. The near-perfect validity suggests the sampler is particularly good at avoiding invalid token combinations.

3. + DAE + PN Sampler + HDDM (full MolHIT): Quality reaches 94.2, FCD improves to 1.03, Validity settles at 99.1%. HDDM adds back distributional fidelity—the coarse-to-fine denoising constrains generation to chemically meaningful trajectories, partially correcting the FCD degradation from the PN sampler while maintaining the quality gains.

The interplay between HDDM and DAE. These two innovations are synergistic: DAE creates a larger, more chemically precise vocabulary that would be harder to learn with standard flat diffusion (56 categories with no structure); HDDM provides the hierarchical structure that makes learning on this expanded vocabulary feasible by grouping related tokens. Without DAE, HDDM's groups would be overkill for the small 7-token vocabulary. Without HDDM, DAE's 56-token vocabulary would be substantially harder to learn (as evidenced by the DiGress+DAE result on GuacaMol in Table 2, where validity is only 65.2% without HDDM versus 87.1% with full MolHIT). Together, they form a complete solution: DAE provides chemical precision, HDDM provides the learning structure to handle that precision.

4. Key Insights and Innovations

Innovation 1: The Ill-Posed Reconstruction Problem as a Root-Cause Diagnosis

The paper's most fundamental conceptual contribution is not any specific algorithmic fix, but rather the diagnosis that prior graph diffusion models fail at molecular generation because their atom encoding makes the reconstruction task ill-posed. This is a diagnostic insight, not an engineering improvement, and it reframes the entire validity problem from "graph models are hard to train on molecules" to "graph models are given an ambiguous representation of molecules."

What makes this distinctive at the idea level. Prior work on graph diffusion for molecules (DiGress, DisCo, Cometh, DeFoG) treated the validity gap as a generic optimization or architecture problem—perhaps graph transformers needed better inductive biases, or the diffusion schedule needed tuning, or more training data was required. The implicit assumption was that the standard atomic-number encoding (C, N, O, F, etc.) was sufficient because it captured the fundamental chemical identity of each atom. MolHIT's authors instead asked a deeper question: is the encoding even capable of representing the training distribution? Their answer—demonstrated in Figure 3—is that it is not. A single token like 'N' maps to multiple chemically distinct states (aliphatic amine nitrogen, pyridine-type aromatic nitrogen, pyrrolic [nH] nitrogen) that have different valencies, different bonding patterns, and different roles in molecular properties. The reconstruction failure is not a matter of model capacity or training difficulty—it is a categorical information loss at the encoding stage. The model literally cannot distinguish what it is being asked to reconstruct.

Comparison to prior work's framing. The field's dominant approach to diagnosis had been to measure aggregate metrics (validity, novelty, FCD) and attribute failures to model architecture or training. Vignac et al. (2022) added geometric priors to DiGress to improve validity; Siraudin et al. (2024) introduced continuous-time Markov chain formulations in Cometh; Qin et al. (2024) adopted flow matching in DeFoG. All of these approaches accepted the standard encoding as given and worked around its limitations. MolHIT's diagnostic move—isolating the encoding as the bottleneck through reconstruction experiments rather than just generation metrics—is what distinguishes this work. The reconstruction experiment in Figure 3 (left) is the key piece of evidence: it shows that even when the model is asked to simply recover its input (not generate novel molecules), the encoding prevents success on specific motifs. This is a much cleaner signal than validity, which confounds encoding issues with generation difficulty.

Significance beyond performance gains. The +9.1 percentage point validity improvement from adding DAE to DiGress (Table 5) is impressive, but the deeper significance is methodological. The paper provides a template for diagnosing generative models: when generation quality is poor, test whether the model can even reconstruct its training data perfectly. If reconstruction fails, the representation—not the generative process—is the bottleneck. This diagnostic principle transfers to any domain where the mapping from discrete tokens to physical states is one-to-many: protein residue encoding, materials composition tokens, or any domain-specific vocabulary where a single symbol masks multiple distinct physical realities. The paper essentially argues that representational adequacy should be verified before optimizing the generative process, which is a conceptual reframing of the standard ML workflow that typically treats tokenization as a preprocessing step and focuses optimization on the model.

Is this incremental or fundamental? This is a fundamental diagnostic insight rather than an incremental refinement. The idea that tokenization matters is not new—NLP has long recognized that subword tokenization choices affect model performance. But applying this lens to identify a categorical failure mode in molecular graph generation—and showing that it explains both reconstruction failures and generation biases—is novel. The paper does not merely improve an existing approach; it identifies a previously unrecognized class of error (encoding-induced ill-posedness) that all prior graph diffusion models suffered from.

---

Innovation 2: Hierarchical Diffusion as a Chemically-Motivated Generalization of Discrete Diffusion

The Hierarchical Discrete Diffusion Model (HDDM) is a theoretical contribution that generalizes discrete diffusion from a flat state space to a multi-level hierarchy while preserving the ELBO guarantee. What makes this distinctive is that the generalization is chemically motivated rather than mathematically arbitrary: the intermediate states correspond to natural chemical groupings (halogens, chalcogens, aromatics), and the forward process is designed to traverse these groupings in a semantically meaningful order.

What makes this distinctive at the idea level. Standard discrete diffusion (Austin et al., 2021) treats all categories as equidistant in the diffusion space—there is no notion that some categories are more similar than others. HDDM introduces a formal mechanism for encoding pairwise similarity through shared intermediate states. The key insight is that the diffusion trajectory itself can encode domain knowledge. Rather than adding chemical priors as auxiliary losses or architectural constraints (as geometric priors do in DiGress), HDDM embeds chemical knowledge directly into the noising process. This is a fundamentally different way of injecting domain expertise: instead of telling the model "these atoms should have similar embeddings" (a representational prior), HDDM tells the model "these atoms should be confused with each other during noising" (a procedural prior). The model learns to recover from chemically-plausible corruptions rather than arbitrary ones.

Comparison to prior work. The closest prior work is Zhou et al. (2025), which independently developed a hierarchical discrete diffusion approach for language modeling using the CTMC framework (Campbell et al., 2022). The paper explicitly acknowledges this and distinguishes HDDM along two axes: (1) HDDM is derived from a semigroup-consistent family of closed-form transition kernels parameterized by explicit diffusion schedules α_t and β_t, while Zhou et al. (2025) operates in the CTMC framework; (2) HDDM supports arbitrary row-stochastic projections Φ, which generalizes the deterministic hierarchical mapping used in the prior work. More importantly, the motivation differs: Zhou et al. (2025) applies hierarchy to language tokens (words → semantic categories), while HDDM applies it to atom types with groupings derived from chemical principles. The chemical motivation—that halogens should share an intermediate state because they share chemical behavior—is what makes HDDM a contribution to molecular generation specifically, not just to diffusion theory.

The paper's Lemma 3.1 and Theorem 3.2 provide mathematical guarantees (Chapman-Kolmogorov consistency, valid ELBO) that elevate HDDM from a heuristic to a principled framework. The proof that the transition kernels form a valid Markov chain relies on the algebraic property that the group transition Q^(1) and masking transition Q^(2) are projection operators satisfying Q^(i) Q^(j) = Q^(max(i,j)). This property is not obvious a priori—it depends on the specific block structure of the transition matrices—and the paper's demonstration that it holds is a genuine theoretical contribution.

Significance beyond performance gains. HDDM's theoretical framework (Proposition C.1) shows that the approach generalizes to arbitrary numbers of hierarchy levels with n transition matrices and n diffusion schedules. This means HDDM is not just a two-level trick for molecules—it is a general framework for encoding hierarchical domain structure into discrete diffusion models. One could imagine applying it to protein design (amino acids → residue categories → secondary structure types), materials generation (elements → periodic table groups → metal/nonmetal), or any domain with natural taxonomies. The paper opens a design space: what hierarchies are most useful for a given domain, and how do you choose the number of levels and the grouping granularity?

Is this incremental or fundamental? HDDM is a fundamental theoretical contribution to discrete diffusion methodology. It extends the framework in a way that preserves its mathematical guarantees while adding a new degree of freedom (hierarchical structure) that previous formulations (uniform, marginal, absorbing) did not possess. The fact that it reduces to standard absorbing diffusion when the hierarchy is removed (β_t = α_t) confirms that it is a strict generalization, not an alternative approach. However, the paper's empirical finding that the full NELBO weighting doesn't improve over simple cross-entropy (Section 3.3) suggests that the practical value of the theoretical guarantee is modest—the framework works, but the loss weighting derived from the ELBO is not necessary for good performance. This is an honest admission that slightly tempers the theoretical contribution: the framework is mathematically sound, but the practical gains come more from the hierarchical structure itself than from the optimal loss weighting.

---

Innovation 3: The Asymmetric Treatment of Atoms and Bonds as a Principle

A subtler but important conceptual contribution is the paper's deliberate asymmetry in how it treats atom and bond diffusion. MolHIT uses HDDM for atoms but standard uniform diffusion for bonds. This is not presented as a major theoretical claim, but it embodies an important design principle: different aspects of a structured object may require different diffusion dynamics, and the choice should be guided by the nature of the data rather than methodological uniformity.

What makes this distinctive at the idea level. The default approach in prior graph diffusion work (DiGress, Cometh, DeFoG) was to apply the same transition type to both nodes and edges—typically marginal or uniform transitions for both. This symmetry is mathematically clean and easier to implement, but it implicitly assumes that nodes and edges have the same statistical structure. MolHIT's authors recognized that this assumption is false for molecular graphs: atom types have natural cluster structure (halogens vs. chalcogens vs. aromatics) that hierarchical diffusion can exploit, while bond types are ordinal (none < single < double < triple < aromatic) and lack a similarly meaningful clustering. Uniform diffusion is more appropriate for bonds because it doesn't impose spurious similarity relationships.

The principle here is that diffusion dynamics should match the structure of the data manifold, not the other way around. This is an instance of a broader design philosophy: when modeling complex structured objects, apply different inductive biases to different components based on what is known about their domains. The paper does not elevate this to a theorem or a named principle, but the empirical decision—and its success—represents a methodological contribution that other structured generation tasks (scene graphs, program ASTs, circuit designs) could adopt.

Comparison to prior work. Prior work in discrete diffusion has generally applied a single transition type to all variables. Austin et al. (2021) explored different transition types (uniform, absorbing, discretized Gaussian) but applied them uniformly to all dimensions. Vignac et al. (2022) used the same marginal transition for both atoms and bonds. The idea of mixing transition types within a single model based on variable semantics is, to my knowledge, novel in the discrete diffusion literature. It is analogous to how continuous diffusion models might use different noise schedules for different modalities in multi-modal generation, but the discrete case is cleaner because the transition types are categorical rather than continuous.

Significance beyond performance gains. This asymmetry is a transferable design lesson: when generating structured objects with heterogeneous components, do not assume that a single noising process is optimal for all components. Identify the natural structure of each component type and design the diffusion process accordingly. For molecular graphs specifically, the finding that "a uniform transition kernel is essential for edge generation" (Section 3.3) is a strong empirical signal that bond types lack the cluster structure that makes hierarchical diffusion beneficial.

Is this incremental or fundamental? This is an incremental but practically important design principle. It does not introduce new mathematics or a new framework, but it corrects an unexamined assumption in prior work and provides empirical evidence for a more nuanced approach. The fact that the paper doesn't make a big theoretical deal of this choice—it's presented as an empirical finding in one paragraph of Section 3.3—suggests the authors view it as a practical engineering decision rather than a conceptual breakthrough. That assessment seems fair, but the principle deserves recognition because it represents a type of domain-aware model design that is underutilized in the graph generation literature.

---

Innovation 4: Verifier-Free Quality Control via Temperature and Top-p Sampling

MolHIT introduces temperature and nucleus (top-p) sampling to molecular graph generation and demonstrates that these simple mechanisms provide effective control over the quality-diversity tradeoff without requiring an external verifier or property predictor. This is a conceptually clean approach to a problem that prior work had either ignored or addressed with more complex machinery.

What makes this distinctive at the idea level. The quality-diversity tradeoff in molecular generation is well-recognized: models that produce more novel structures tend to produce more invalid ones, and pushing for high validity often reduces structural exploration. Prior approaches to managing this tradeoff fell into two categories: (1) post-hoc filtering (generate many molecules, discard invalid ones, measure what remains) or (2) property-guided generation (train a property predictor and use it to steer generation toward desired regions of chemical space). MolHIT's approach is different: it uses the model's own predictive uncertainty, modulated by temperature and top-p truncation, to control exploration during generation itself. The key conceptual move is recognizing that the model's predicted probability distribution over atom types already encodes a quality-diversity spectrum—high-probability tokens correspond to chemically conservative choices, while the tail of the distribution contains more adventurous but riskier alternatives. Temperature and top-p sampling provide a continuous knob for moving along this spectrum.

Figure 4 demonstrates this cleanly: as top-p decreases from 1.0 to 0.8, quality improves and validity rises to 99.4% while scaffold novelty decreases. The tradeoff is smooth and controllable. This is significant because it means MolHIT does not require an external verifier, a separately trained property predictor, or a reinforcement learning loop to achieve high validity. The control mechanism is built into the sampling procedure itself.

Comparison to prior work. In language generation, temperature and top-p sampling (Holtzman et al., 2019) are standard techniques. Their application to molecular graph generation, however, is "largely unexplored" (Section 3.3). Prior graph diffusion models used standard posterior sampling without these controls, accepting whatever validity their models produced. MolHIT is the first to show that these NLP-inspired techniques transfer effectively to structured discrete generation and provide the same type of quality-diversity control. The finding that temperature sampling should be applied only to atom predictions (not bond predictions) is an important domain-specific refinement: bond types are more constrained by valence rules, so stochastic exploration of bond types is more likely to produce invalid structures than stochastic exploration of atom types.

Significance beyond performance gains. The practical implication is substantial: MolHIT's 99.1% validity (Table 1) is achieved with a simple sampling hyperparameter (top-p = 0.8) rather than with architectural complexity or additional training objectives. This means practitioners can tune the validity-novelty tradeoff at inference time without retraining, simply by adjusting the top-p threshold. For drug discovery applications where validity is paramount, a low top-p value produces highly reliable outputs; for exploratory applications where diversity matters more, a higher top-p value can be used. The fact that this control mechanism is external to the model (it does not require modifying the training procedure) makes it easy to adopt and adapt.

Is this incremental or fundamental? This is an incremental but practically impactful contribution. The underlying techniques (temperature sampling, nucleus sampling) are well-established in other domains; the innovation is recognizing their applicability to molecular graph generation and demonstrating their effectiveness. The paper does not claim novelty for the sampling methods themselves, but rather for their application and the empirical characterization of the tradeoff they enable. The near-perfect validity (99.4%) achieved with top-p = 0.8 (Figure 4) is a strong result that validates this approach.

---

Innovation 5: The Co-Design of Representation and Diffusion Dynamics as a Complete Solution

The paper's most important meta-contribution is demonstrating that DAE and HDDM are not independent improvements but synergistic components that solve complementary problems. DAE fixes the representation (making the vocabulary chemically precise), while HDDM fixes the learning dynamics (making the expanded vocabulary learnable through hierarchical structure). Neither alone would achieve the full performance of MolHIT.

What makes this distinctive at the idea level. The standard ML workflow treats representation (tokenization, feature engineering) and learning algorithm (architecture, training objective) as separate concerns. MolHIT's authors recognized that in molecular generation, these are coupled: DAE's expanded vocabulary (12–56 tokens) would be substantially harder to learn with standard flat diffusion, as evidenced by the DiGress+DAE result on GuacaMol (Table 2) where validity drops to 65.2%. HDDM's hierarchical structure addresses exactly the learning difficulty that DAE creates, while DAE provides the chemical precision that makes HDDM's groups meaningful. This is co-design: the representation and the learning dynamics are designed together, each compensating for the other's limitations.

The ablation in Table 5 tells this story clearly. DAE alone: +9.1 validity, but quality only reaches 87.6. Add PN sampler: validity jumps to 99.4, but FCD degrades (less distributional fidelity). Add HDDM: quality reaches 94.2, FCD recovers to 1.03, validity stays at 99.1. Each component addresses a different failure mode, and the full system benefits from their combination. The paper doesn't just present a new model; it presents a decomposition of the validity-novelty problem into sub-problems (representation ambiguity, learning difficulty, sampling diversity) and provides targeted solutions for each.

Comparison to prior work. Prior graph diffusion models (DiGress, Cometh, DeFoG) made incremental improvements to the diffusion process while leaving the encoding unchanged. They implicitly assumed that the representation was adequate and that the problem was in the generative dynamics. MolHIT's contribution is showing that this assumption was wrong—the representation was the primary bottleneck—and that fixing it requires complementary changes to the diffusion process to handle the resulting vocabulary expansion. This is a more holistic approach to model design than the typical "improve one component at a time" methodology.

Significance beyond performance gains. The co-design principle transfers to other structured generation domains. When a standard representation is found to be lossy or ambiguous, expanding it will likely create learning difficulties that the original training procedure was not designed to handle. The solution is not just to expand the vocabulary, but to simultaneously adapt the learning dynamics to the expanded vocabulary's structure. This is a design pattern that could apply to protein design (expanding amino acid tokens to include post-translational modifications), materials generation (expanding element tokens to include oxidation states), or any domain where fine-grained tokenization reveals latent structure that coarse tokenization hid.

Is this incremental or fundamental? This is a fundamental contribution to the methodology of structured generative modeling, though it is presented implicitly through the design of MolHIT rather than as an explicit design principle. The paper does not claim "co-design of representation and dynamics" as a named contribution, but the logic of the ablation study and the interdependence of DAE and HDDM make this the natural interpretation of why MolHIT succeeds where prior approaches failed. It represents a shift from "fix the model" to "fix the representation, then fix the model to handle the fixed representation"—a more sophisticated approach to improving generative systems.

5. Experimental Analysis

Evaluation Methodology

• Dataset. All experiments use the MOSES benchmark (Polykovskiy et al., 2020) and the GuacaMol benchmark (Brown et al., 2019). MOSES contains approximately 1.9M drug-like molecules filtered from the ZINC database with 7 heavy atom types in the standard encoding, which DAE expands to 12 tokens. The paper uses the standard MOSES training/test split. GuacaMol contains a broader chemical space including charged species, with 12 elemental types expanding to 56 DAE tokens; the paper uses the full unfiltered GuacaMol dataset rather than the filtered subset used by prior graph diffusion work. For conditional generation, the MOSES dataset is augmented with four molecular properties (QED, SA, logP, MW) computed via RDKit and the sascorer module, with min-max normalization using training split statistics (Table 13).

• Base model(s). All experiments use the graph transformer architecture from DiGress (Vignac et al., 2022) with 12 transformer blocks and approximately 16.2M parameters, maintained without architectural modifications (same internal dimensions, same node-edge dual prediction). The authors explicitly state they "utilize the original graph transformer from DiGress, maintaining the same model size" (Section 4). The conditional modeling variant adds Adaptive Layer Normalization (AdaLN) for node features only. The total trainable parameter count is approximately 16.2M, with DAE and HDDM token additions adding "negligible overhead where representing a variance of less than 0.01% in total parameters" (Appendix D.5). This architectural consistency is deliberate: by keeping the architecture fixed, the paper isolates the contributions of HDDM and DAE from confounding architectural innovations.

• Metrics. The MOSES evaluation follows the standardized protocol of Polykovskiy et al. (2020) with seven metrics: Validity (fraction of generated molecules passing RDKit sanitization and basic valency rules), Uniqueness (proportion of valid molecules that are not duplicates, measuring mode collapse), Novelty (fraction of valid unique molecules absent from the training set), Filters (percentage passing medicinal chemistry filters including MCULE, BRENK, and PAINS), Fréchet ChemNet Distance (FCD, Preuer et al., 2018; distance between generated and test molecule distributions in ChemNet feature space), Similarity to Nearest Neighbor (SNN, average Tanimoto similarity to closest test-set neighbor), and Scaffold Similarity (Scaf, cosine similarity between Bemis-Murcko scaffold frequencies in generated and test sets). Additionally, the paper introduces Quality (Lee et al., 2025), defined as the proportion of molecules that are valid, unique, have synthetic accessibility SA ≤ 4, and drug-likeness QED ≥ 0.6. Two new structure novelty metrics are introduced: Scaffold Novelty = |𝒮_gen \ 𝒮_train| / n_total (fraction of generated molecules with scaffolds absent from training), and Scaffold Retrieval = |𝒮_gen ∩ 𝒮_test| / n_total (fraction of generated molecules whose scaffolds appear in the test set). For GuacaMol, metrics include Validity, Uniqueness, Novelty, KL Divergence, and FCD. For conditional generation, Mean Absolute Error (MAE) and Pearson correlation coefficient (r) between conditioned targets and generated molecule properties are used, along with validity. For scaffold extension, Validity, Diversity (1 minus average pairwise Tanimoto similarity among unique valid molecules), Hit@1, and Hit@5 (probability that the ground-truth extension is recovered within the top k samples).

• Baselines. The paper compares against both 1D sequence models and 2D graph models. 1D baselines include: VAE (Kingma and Welling, 2013), CharRNN (Segler et al., 2018), SAFE-GPT (Noutahi et al., 2024), and GenMol (Lee et al., 2025). The paper notes that "the original GenMol model was trained on a much larger molecule dataset, so we train the model on the MOSES dataset for a fair comparison" (Appendix D.4). 2D graph baselines include: DiGress (Vignac et al., 2022), DisCo (Xu et al., 2024), Cometh (Siraudin et al., 2024), and DeFoG (Qin et al., 2024), representing the prior state-of-the-art in atom-level graph diffusion. For multi-property guided generation, baselines are Marginal transition (effectively DiGress without geometric priors) and Marginal transition + DAE (incorporating decoupled atom encoding into the marginal transition baseline). The paper uses released checkpoints when available and otherwise trains models using official codebases with reported hyperparameters.

• Generation budget / compute accounting. The paper does not use a standardized "generation budget" or FLOPs accounting in the manner typical of scaling law studies. Instead, fairness is maintained through: (1) equal architecture (identical graph transformer, same parameter count) across all ablations and baselines where reimplemented, (2) comparable training epochs (MolHIT trains for 100 epochs on MOSES versus 300 epochs for prior graph diffusion baselines, making MolHIT's compute budget during training lower), and (3) equal sample counts for evaluation (25,000 samples per run on MOSES, 10,000 samples for GuacaMol and conditional tasks, all averaged over 3 independent runs). The paper does not report inference time or FLOPs per generated molecule, focusing instead on quality metrics at fixed sample counts.

• Cross-validation / statistical protocol. All reported results represent the average of three independent runs with standard deviations provided in Appendix D.3 (Tables 9–12). For MOSES unconditional generation, each run generates 25,000 molecules; for GuacaMol and conditional tasks, each run generates 10,000 molecules. For baselines, the paper uses publicly available checkpoints when released; otherwise it trains models using official codebases. Training details include gradient clipping at threshold 2.0, Exponential Moving Average (EMA) rate 0.999, and early stopping (100 epochs for MOSES, 40–50 for GuacaMol). The paper does not use cross-validation for strategy selection (unlike the compute-optimal test-time scaling example); performance is reported directly on the standard test splits. The authors note that for GuacaMol, MolHIT is trained for only 40 epochs versus DiGress's 1,000 epochs, and state that "further training will improve the metrics" (Section 4.2), which is an honest acknowledgment that results are not at convergence.

Main Quantitative Results

Unconditional Generation on MOSES

Headline result. MolHIT achieves 99.1% validity, 94.2% Quality, and 0.39 Scaffold Novelty on the MOSES benchmark (Table 1), establishing new state-of-the-art across multiple metrics and achieving near-perfect validity for the first time among graph diffusion models. The Quality score of 94.2% approaches the training set's 95.4%, indicating MolHIT generates molecules that are nearly indistinguishable from real drug-like compounds by this composite metric.

Validity comparison across model categories. Table 1 shows a clear hierarchy. 1D sequence models achieve high validity: SAFE-GPT reaches 99.8%, GenMol 99.7%, but pay a price in scaffold novelty (0.12 and 0.05 respectively) and novelty overall (43.7% and 68.9%). Prior graph diffusion models cluster in the 87–93% validity range: DiGress (87.1%), DisCo (88.3%), Cometh (87.2%), DeFoG (92.8%). MolHIT's 99.1% validity represents a substantial jump over the next-best graph model (DeFoG at 92.8%) and approaches the sequence-model ceiling, but without the memorization penalty: MolHIT's scaffold novelty of 0.39 is the highest among all compared models, substantially above Cometh's 0.36 and far above SAFE-GPT's 0.12. This directly supports the paper's central claim of bridging the validity-novelty gap.

Quality and FCD. MolHIT's Quality of 94.2% (Table 1) is the highest among all compared models—above DeFoG's 88.5%, DiGress's 82.5%, and all 1D baselines (SAFE-GPT: 92.8%, CharRNN: 92.6%, VAE: 92.8%). FCD, which measures distributional fidelity, improves from DiGress's 1.25 to MolHIT's 1.03 (lower is better), indicating that the generated molecules are closer to the test set distribution in ChemNet feature space. The training set's FCD is 0.48 (since FCD measures distance to test set, the training set has the minimum possible value for an in-distribution baseline); MolHIT's 1.03 is substantially lower than all other graph models (DeFoG: 1.95, Cometh: 1.44) and competitive with 1D models (SAFE-GPT: 0.72, CharRNN: 0.52).

Scaffold-level metrics. Scaffold Novelty (Table 1) quantifies the fraction of generated molecules whose Bemis-Murcko scaffold is absent from the training set—a direct measure of structural extrapolation. MolHIT achieves 0.39, meaning 39% of generated molecules have novel scaffolds, versus Cometh's 0.36 and DeFoG/DiGress's 0.26. The 1D models are substantially lower: SAFE-GPT (0.12), GenMol (0.05), indicating their memorization tendency. Scaffold Retrieval—the fraction of generated molecules whose scaffold appears in the held-out test set—is 0.033 for MolHIT, matching DiGress and DeFoG (both 0.031) and exceeding Cometh (0.023). This suggests MolHIT captures the test distribution's scaffold patterns as well as prior graph models while simultaneously exploring more novel structures.

Standard deviations. Appendix Table 9 provides full statistics. MolHIT's Quality has standard deviation ±0.2 across 3 runs (94.2 ± 0.2), Validity is 99.1 ± 0.0, and Scaffold Novelty is 0.39 ± 0.00. These tight variances indicate robust performance. The training set's Quality is 95.4% (computed once, no variance reported).

Unconditional Generation on GuacaMol

Headline result. On the full (unfiltered) GuacaMol dataset, MolHIT achieves 87.1% validity with 96.7% KL divergence and 54.9 FCD (Table 2), substantially outperforming DiGress trained on the same unfiltered data (74.7% validity, 92.4 KL, 61.1 FCD). The KL divergence of 96.7 (training set: 99.9) indicates strong distributional coverage.

The significance of the unfiltered comparison. Prior graph diffusion work (DiGress, Cometh, DeFoG) trained and evaluated on a filtered GuacaMol subset where molecules containing atoms that failed reconstruction were removed. This filtering biases the training distribution toward "easy" molecules and artificially inflates validity metrics because the model never sees the challenging cases during training. The paper reimplements DiGress on the full unfiltered dataset (DiGress "full" in Table 2) and finds validity drops from the reported 85.2% (filtered) to 74.7% (full)—a 10.5 percentage point degradation. This confirms that prior results were partially an artifact of dataset curation. MolHIT's 87.1% validity on the full dataset is higher than DiGress's 85.2% on the easier filtered dataset, demonstrating genuine improvement.

DAE's effect on GuacaMol. Adding DAE to DiGress (DiGress + DAE in Table 2) actually reduces validity from 74.7% to 65.2% and worsens FCD from 61.1 to 49.2. This is a critical finding: the expanded 56-token vocabulary makes the learning problem substantially harder when using flat (marginal) diffusion, because the model must distinguish 56 categories with no intermediate structure. This motivates HDDM—the hierarchical diffusion provides the structure needed to learn the expanded vocabulary. MolHIT (DiGress + DAE + HDDM) achieves 87.1% validity, demonstrating that HDDM is what makes DAE practical on larger vocabularies. The KL divergence of 96.7 (versus 87.0 for DiGress+DAE and 92.4 for DiGress full) further shows that HDDM improves distributional coverage.

FCD tradeoff. MolHIT's FCD of 54.9 is worse than DiGress full's 61.1 (lower is worse for FCD). The paper acknowledges this explicitly: "For FCD, the strong performance of original DiGress without DAE indicates that using DAE does not always lead to the generative task being easier since it can be hard to model with differentiate extended atom vocabulary." In other words, the expanded vocabulary improves validity and distribution coverage (KL) but makes it harder to exactly match the ChemNet feature distribution of the test set. The paper also notes that MolHIT was trained for only 40 epochs versus DiGress's 1,000 epochs, suggesting this gap might close with further training.

Standard deviations. Appendix Table 10 shows MolHIT's validity is 87.1 ± 0.5, DiGress full is 74.7 ± 0.4, and DiGress+DAE is 65.2 ± 0.4 across 3 runs. The variances are small relative to the performance gaps, confirming statistical significance.

Multi-Property Guided Generation

Headline result. MolHIT achieves a macro-averaged MAE of 0.058 and Pearson r of 0.807 across four properties (QED, SA, logP, MW), with 96.31% validity (Table 3). This represents a 52.4% reduction in MAE over the Marginal+DAE baseline (0.122) and a 59.4% reduction over pure Marginal (0.143), while simultaneously improving validity by 8.46 percentage points over Marginal+DAE (87.85%).

Per-property breakdown (Table 3). The improvements are consistent across properties:

• QED: MAE 0.061 (MolHIT) vs. 0.107 (Marginal+DAE) vs. 0.117 (Marginal). Pearson r: 0.804 vs. 0.565 vs. 0.489.
• SA: MAE 0.040 vs. 0.094 vs. 0.115. Pearson r: 0.790 vs. 0.559 vs. 0.570.
• logP: MAE 0.049 vs. 0.061 vs. 0.067. Pearson r: 0.950 vs. 0.836 vs. 0.802. The near-perfect Pearson r of 0.950 for logP is notable—the model's conditional generation is almost perfectly linearly correlated with the target.
• MW: MAE 0.081 vs. 0.227 vs. 0.272. Pearson r: 0.685 vs. 0.437 vs. 0.396. Molecular weight is the hardest property to condition on (highest MAE, lowest Pearson r across all methods), but MolHIT still achieves a 64.4% MAE reduction over Marginal+DAE.

Validity under conditioning. An important finding is that conditioning does not degrade validity—MolHIT's 96.31% validity under multi-property conditioning is comparable to its unconditional 99.1%. In contrast, Marginal+DAE drops from 96.2% unconditional validity (Table 5) to 87.85% under conditioning (Table 3), and pure Marginal drops to 75.03%. This suggests that HDDM's hierarchical structure provides robustness to conditioning that flat diffusion lacks.

Guidance strength. The paper uses classifier-free guidance with weight w = 1.0 and reports that "increasing the guidance weight beyond unity did not consistently yield better property alignment" (Appendix D.8). This is a notable negative result—unlike continuous diffusion for images where higher guidance weights (w = 3–7.5) typically improve conditioning, discrete graph diffusion saturates at w = 1. The authors identify better sampler/model designs for higher guidance strength as future work.

Standard deviations. Appendix Table 11 shows that all reported differences are statistically robust. For example, MolHIT's QED MAE is 0.061 ± 0.001 versus Marginal+DAE's 0.107 ± 0.001—the gap is over 40 standard deviations.

Scaffold Extension

Headline result. MolHIT achieves 83.9% validity, 57.4% Diversity, and 3.92% Hit@1 on scaffold extension (Table 4), representing a 33.1 percentage point validity improvement over DiGress (50.8%) and a 19.1 percentage point improvement over Marginal+DAE (64.8%). Hit@1 nearly doubles from DiGress's 2.07% to 3.92%, and Hit@5 increases from 6.41% to 9.79%.

Validity improvement. The jump from 50.8% (DiGress) to 83.9% (MolHIT) is the most dramatic result in this task. More than half of DiGress's scaffold completions are chemically invalid—the model cannot reliably complete a given substructure into a valid molecule. MolHIT reduces the failure rate from 49.2% to 16.1%. This is directly relevant for practical drug discovery: given a core scaffold, a chemist needs the model to produce completions that are at least chemically plausible before evaluating their properties.

Diversity and hit rate tradeoff. An interesting pattern emerges when DAE is added to the marginal baseline: Marginal+DAE achieves 64.8% validity and 58.0% Diversity (higher than both DiGress's 44.8% and MolHIT's 57.4%), but Hit@1 drops from DiGress's 2.07% to 1.67%. The paper interprets this as "the extended expressivity of the model" leading to higher diversity at the cost of less precise recovery of the ground-truth extension. However, Marginal+DAE's Hit@5 (6.37%) nearly matches DiGress's (6.41%), suggesting the model is exploring a broader range of valid completions—the ground truth is still in the top 5, just not at rank 1 as often. MolHIT achieves both high validity (83.9%) and the highest hit rates (3.92% Hit@1, 9.79% Hit@5), demonstrating that HDDM adds precision on top of DAE's diversity.

Standard deviations. Appendix Table 12 shows MolHIT's validity of 83.9 ± 0.4, Diversity 57.4 ± 0.6, Hit@1 3.92 ± 0.23, and Hit@5 9.79 ± 0.09 over 3 runs of 10,000 targets each. The tight standard deviations (especially Hit@5 at ±0.09) indicate consistent performance.

Ablation Studies and Robustness Checks

• Component ablation on MOSES (Table 5). The incremental integration of components shows each contributes independently:

  • DiGress baseline: Quality 82.5, FCD 1.25, Validity 87.1%.
  • + DAE only: Quality jumps to 87.6 (+5.1), FCD improves to 0.89 (-0.36), Validity leaps to 96.2% (+9.1). This is the single largest jump in validity, confirming DAE alone addresses the primary bottleneck (ill-posed reconstruction).
  • + PN Sampler: Quality further rises to 92.9 (+5.3), Validity reaches 99.4% (+3.2), but FCD degrades from 0.89 to 1.65 (+0.76). The PN sampler adds exploration that improves quality and validity but at the cost of distributional fidelity—the sampling variance pushes generated molecules slightly off the training manifold.
  • + HDDM (full MolHIT): Quality reaches 94.2 (+1.3), FCD recovers to 1.03 (-0.62), Validity settles at 99.1% (-0.3). HDDM adds back distributional fidelity by constraining generation to chemically meaningful trajectories through coarse-to-fine denoising, partially correcting the FCD degradation from the PN sampler while preserving the quality gains.
  • The non-obvious finding is that the PN sampler and HDDM have countervailing effects on FCD—the sampler hurts distributional fidelity while HDDM restores it—and the final FCD of 1.03 is better than DiGress's original 1.25.

• Effect of temperature and top-p sampling (Figure 4). The tradeoff between sample quality and scaffold novelty is characterized across top-p values from 0.4 to 1.0:

  • Top-p = 1.0 (no nucleus sampling): Scaffold Novelty is highest (~0.39) but Quality is lower (~93%) and Validity is ~98.5%.
  • Top-p = 0.8: Quality peaks at 95.1%, Validity reaches 99.4%, but Scaffold Novelty drops to ~0.32.
  • Top-p = 0.6 and below: Both Quality and Novelty degrade sharply—the sampling becomes too deterministic, and the model collapses to a narrow set of high-probability (but potentially less drug-like) outputs.
  • The finding that top-p = 0.8 is optimal represents a sweet spot where validity is near-perfect (99.4%) while maintaining reasonable novelty. The paper does not report results for top-p < 0.4, so the behavior in the extreme deterministic regime is unknown.

• Atom vs. bond temperature sampling (Algorithm 1, line 7). The paper states that "doing temperature sampling only for the atom prediction results in the best performance" without providing a comparative ablation table. This is an implicit ablation: the bond predictions use raw model probabilities without temperature scaling because bond types are more constrained by valence rules, and stochastic exploration of bond types is more likely to produce invalid structures than stochastic exploration of atom types. A formal ablation comparing temperature sampling on both vs. atoms-only would have strengthened this claim.

• HDDM vs. uniform/baseline diffusion for atoms (Table 3, comparing MolHIT to Marginal). In the conditional generation setting, the Marginal baseline achieves only 75.03% validity and 0.143 MAE, while MolHIT achieves 96.31% validity and 0.058 MAE. The Marginal+DAE intermediate shows that DAE alone improves validity to 87.85% and MAE to 0.122, but the full MolHIT (adding HDDM) provides the remaining gains. This confirms that hierarchical diffusion is particularly beneficial in conditional settings where the model must simultaneously satisfy property constraints and maintain chemical validity.

• DAE reconstruction on GuacaMol (Figure 5, Figure 6). The reconstruction experiments quantify DAE's impact on charged species:

  • Standard coarse encoding achieves only 1.88% reconstruction success rate for [nH] groups and negligible 0.09% identity preservation rate for charged species.
  • DAE achieves 100% success rate and over 99.98% identity preservation rate for all atom types on GuacaMol.
  • In generative performance (Figure 6), MolHIT generates molecules with formal charges at approximately the same proportion as the training set (~6%), while models with coarse encoding "barely generate the charged atoms." This demonstrates that DAE eliminates the systematic under-generation bias documented in Figure 3 for MOSES.

• DAE on GuacaMol without HDDM (Table 2, DiGress+DAE row). This serves as an implicit ablation of HDDM on a challenging dataset: adding DAE to DiGress without HDDM reduces validity from 74.7% to 65.2% and worsens KL divergence from 92.4 to 87.0. The 56-token vocabulary is too large for flat marginal diffusion to learn effectively, confirming that HDDM is necessary to make DAE practical when the vocabulary expands significantly beyond the original 7–12 tokens.

• Training epochs on GuacaMol. The paper reports MolHIT trained for 40 epochs versus DiGress's 1,000 epochs. This is framed as a limitation: "we have not fully trained the model until performance saturation on GuacaMol dataset and we believe performance improvement with further training." However, it also means MolHIT's reported performance (87.1% validity, 54.9 FCD) represents a lower bound—these numbers would likely improve with additional training. The paper does not provide learning curves or convergence analysis to estimate asymptotic performance.

• Guidance weight for CFG. The paper reports using w = 1.0 and finding that higher guidance weights did not improve property alignment. This is presented as a negative result: "increasing the guidance weight beyond unity did not consistently yield better property alignment" (Appendix D.8). No ablation table is provided showing performance at w = 0.5, 1.0, 2.0, 3.0, etc., so the claim cannot be quantitatively evaluated. This is a missing experiment: a formal CFG weight sweep would clarify whether w = 1.0 is a true saturation point or if improved sampler design could unlock benefits at higher guidance.

Critical Assessment

Claim 1: MolHIT achieves state-of-the-art performance on MOSES, including near-perfect validity for the first time in graph diffusion.

What the experiments demonstrate: Table 1 clearly shows MolHIT outperforms all graph diffusion baselines (DiGress, DisCo, Cometh, DeFoG) on Quality (94.2 vs. next-best 88.5), Validity (99.1 vs. next-best 92.8), and Scaffold Novelty (0.39 vs. next-best 0.36). The near-perfect validity does represent a first for graph diffusion models on MOSES. What is not demonstrated: The claim of "state-of-the-art" is relative to the specific baselines compared. The paper does not compare against every recent molecular generation method—notably absent are 3D-aware generation methods (e.g., EDM by Hoogeboom et al., 2022; GeoLDM by Xu et al., 2023) and autoregressive graph methods. The title "state-of-the-art" is justified within the scope of 2D graph diffusion models, but the paper could be more precise about this scope limitation.

The claim of near-perfect validity (99.1%) is supported but with an important methodological note: this is achieved with top-p = 0.8 (Figure 4), which represents a specific operating point on the quality-diversity tradeoff. At top-p = 1.0 (no nucleus sampling), validity is lower (~98.5% from Figure 4). The 99.1% figure is thus a result of inference-time hyperparameter tuning, not an inherent property of the trained model. This is not a weakness—temperature and top-p sampling are standard and legitimate—but it means the validity is partly attributable to the sampling strategy, not just the model and diffusion process.

Claim 2: MolHIT bridges the gap between 1D sequence models (high validity, low novelty) and 2D graph models (low validity, high novelty).

What the experiments demonstrate: Figure 1 (bottom) visualizes this claim effectively: MolHIT occupies a region of the quality-novelty space that neither sequence models nor prior graph models reach. Table 1 provides the quantitative backing: SAFE-GPT (a 1D model) achieves 99.8% validity but 0.12 Scaffold Novelty; DiGress achieves 87.1% validity but 0.26 Scaffold Novelty; MolHIT achieves 99.1% validity and 0.39 Scaffold Novelty. The claim holds on the MOSES dataset. What is not demonstrated: The claim is tested on only one dataset (MOSES). GuacaMol results (Table 2) do not include Scaffold Novelty metrics and do not compare against 1D sequence baselines, so the bridging claim cannot be verified on the more chemically diverse GuacaMol benchmark. Additionally, the 1D baselines on MOSES include GenMol, which was "trained on the MOSES dataset for a fair comparison" (Appendix D.4) rather than on the larger dataset used in the original paper. This re-training may put GenMol at a disadvantage relative to its reported performance, potentially understating the 1D baseline ceiling.

Claim 3: DAE resolves a fundamental reconstruction failure that made prior graph models' generation task ill-posed.

What the experiments demonstrate: Figure 3 (left) shows reconstruction success rates: DAE achieves near-perfect reconstruction while standard encoding fails dramatically on molecules containing pyrrolic nitrogen. Figure 3 (right) shows that prior models systematically under-generate [nH] motifs compared to the training distribution, while MolHIT matches the training distribution. Figure 5 and Figure 6 show analogous results for charged species on GuacaMol. The ablation in Table 5 shows DAE alone provides a +9.1 percentage point validity improvement. What is demonstrated but could be stronger: The reconstruction experiment (Figure 3 left) would be more informative if it broke down reconstruction failure by atom type systematically (e.g., what fraction of failures are due to [nH] vs. other ambiguous tokens, and whether DAE achieves 100% reconstruction for all token types or only most). The claim that the task becomes "well-posed" would be strengthened by showing that with DAE, the model learns a mapping that is provably invertible (i.e., encoding and decoding are exact inverses). The paper shows near-perfect empirical reconstruction rates but does not prove that DAE makes the problem theoretically well-posed—there could still be edge cases where ambiguity persists.

Claim 4: HDDM's hierarchical structure enables learning on expanded vocabularies that flat diffusion cannot handle.

What the experiments demonstrate: Table 2 provides the key evidence: DiGress+DAE on GuacaMol (56-token vocabulary, flat marginal diffusion) achieves only 65.2% validity, while MolHIT (56-token vocabulary, hierarchical diffusion) achieves 87.1% validity. The ablation in Table 5 shows that HDDM provides an incremental benefit even on the smaller 12-token MOSES vocabulary (Quality improves from 92.9 to 94.2, FCD from 1.65 to 1.03). What is not demonstrated: The paper does not show learning curves comparing HDDM vs. flat diffusion on the expanded vocabulary as a function of training time. It is possible that flat diffusion could eventually catch up to HDDM with longer training, and that HDDM's advantage is in sample efficiency rather than asymptotic performance. The GuacaMol results for MolHIT are after only 40 epochs, and the paper acknowledges training is not to convergence—this could mean HDDM's advantage relative to flat diffusion might shrink (or grow) with full training. Additionally, the paper does not ablate the specific choice of groupings in HDDM—would different groupings (e.g., grouping by periodic table row rather than chemical behavior) yield different results? The sensitivity of HDDM to grouping quality is unexplored.

Claim 5: MolHIT achieves strong performance on downstream tasks including multi-property guided generation and scaffold extension.

What the experiments demonstrate: Table 3 shows MolHIT substantially outperforms baselines on multi-property conditioning (MAE 0.058 vs. 0.122, validity 96.31% vs. 87.85%). Table 4 shows MolHIT dramatically improves scaffold extension (validity 83.9% vs. 50.8% for DiGress, Hit@1 3.92% vs. 2.07%). What is not demonstrated: For multi-property guided generation, the paper compares only against its own internal baselines (Marginal, Marginal+DAE) rather than against published conditional molecular generation methods (e.g., Graph Diffusion Transformer by Liu et al., 2024, which is cited in the paper but not used as a baseline). The lack of external conditional baselines makes it difficult to assess how MolHIT's conditioning performance compares to the broader state of the art. The scaffold extension task uses a simple substructure-fixing approach during diffusion; the paper does not compare against scaffold-based generation methods like JT-VAE (Jin et al., 2018) or other scaffold-constrained approaches. The guidance weight for CFG is fixed at w = 1.0 without an ablation across w values, and the finding that higher w doesn't help is asserted without evidence (no table, no figure). This is a missing experiment that would clarify whether the CFG implementation is correct or whether discrete graph diffusion has a fundamental limitation at higher guidance strengths.

Genuine weaknesses in the experimental design:

1. Single architecture, single parameter count. All experiments use the DiGress graph transformer at ~16.2M parameters. There is no scaling analysis—does the benefit of HDDM and DAE increase, decrease, or stay constant as model size grows? For GuacaMol where the 56-token vocabulary is challenging, a larger model might close the gap between DiGress+DAE and MolHIT even without HDDM. The paper acknowledges this as a limitation: "we have not tested with the model size increase or architectural improvement, in which we believe have further room for better performance."

2. Training budget not equalized with baselines. MolHIT trains for 100 epochs on MOSES versus 300 epochs for DiGress, Cometh, and DeFoG. This means MolHIT achieves better results with less training compute—which is impressive—but it also means the prior models might not be at their best if they were early-stopped. The paper does not report whether the baselines were trained to convergence. On GuacaMol, MolHIT is trained for 40 epochs versus DiGress's 1,000—a 25× difference—and the paper acknowledges that "further training will improve the metrics." This is an honest admission, but it means the GuacaMol results likely understate MolHIT's ultimate performance and may not represent a fair comparison at equal training compute.

3. No statistical significance testing beyond standard deviations. The paper reports means and standard deviations over 3 runs, which is adequate, but does not perform formal hypothesis tests (t-tests, bootstrap confidence intervals) for the key comparisons (e.g., is MolHIT's Quality of 94.2 statistically significantly higher than DeFoG's 88.5?). The standard deviations are small enough that the gaps are clearly significant by informal inspection, but formal testing would strengthen the claims.

4. Missing ablation of grouping strategy. The paper uses chemically motivated groupings for HDDM (halogens together, chalcogens together, aromatics together) but does not ablate the choice of groupings. Would a random grouping perform similarly? Would finer groupings (each atom type in its own group, collapsing HDDM to standard diffusion) or coarser groupings (all atoms in one group) perform better or worse? The sensitivity of HDDM's benefit to grouping quality is unknown, which limits the generalizability of the approach to other domains where optimal groupings may not be obvious.

5. Missing learning curves and convergence analysis. The paper provides no training curves showing loss, validity, or FCD as a function of training epoch for any experiment. This makes it impossible to assess whether the reported gains are asymptotic or if further training would change the relative ordering of methods. The GuacaMol experiments in particular are explicitly not at convergence, but without learning curves, the reader cannot estimate how much performance might improve.

6. Limited evaluation of DAE on GuacaMol without HDDM. The DiGress+DAE result on GuacaMol (Table 2) shows validity dropping to 65.2%. But this is with marginal diffusion, not with absorbing or uniform diffusion. The paper does not test whether DAE with absorbing diffusion (without hierarchy) would perform better than DAE with marginal diffusion. The failure of DAE on GuacaMol is attributed to vocabulary size, but it could also be due to the mismatch between DAE's expanded discrete structure and the marginal transition's design.

7. Scaffold extension evaluation is limited. The scaffold extension task evaluates Hit@1 and Hit@5 against the ground-truth test-set completion. But in practice, scaffold extension is valuable when there are multiple valid completions of a scaffold, not just the one that happened to appear in the test set. The paper does not measure whether generated extensions are chemically valid and drug-like (beyond basic validity), which would be a better measure of practical utility than exact-match recovery of the test-set completion. The diversity metric (57.4%) captures this partially, but a more thorough evaluation would include property distributions of generated extensions compared to the test set.

Experiments that would have strengthened the paper:

• A direct comparison of HDDM with the hierarchical discrete diffusion method of Zhou et al. (2025) on a shared task. The paper cites this as concurrent work but does not empirically compare against it.
• A formal ablation of the grouping strategy (random groups, chemical groups, periodic-table groups, single-group collapsing to standard diffusion) to quantify how much chemical knowledge in the grouping matters.
• Scaling curves showing performance as a function of model size (e.g., 4M, 8M, 16M, 32M parameters) for DiGress, DiGress+DAE, and MolHIT to test whether the benefits of DAE and HDDM persist at larger scales.
• A CFG guidance weight sweep (w = 0, 0.5, 1.0, 2.0, 4.0) for the conditional generation task with a figure analogous to Figure 4, to characterize the guidance strength vs. property alignment tradeoff.
• Comparison against external conditional baselines (e.g., Graph Diffusion Transformer by Liu et al., 2024) on multi-property guided generation, and against scaffold-constrained methods (e.g., JT-VAE) on scaffold extension.
• Learning curves for all major experiments, showing validation metrics as a function of training epoch, with early stopping rationale.
• Inference time / FLOPs analysis comparing MolHIT's per-molecule generation cost to baselines. The paper uses equal architecture, so inference cost should be similar, but the PN sampler's re-noising step might add overhead that should be quantified.

Conditional nature of the claims:

The paper's central claims hold strongly on the MOSES dataset (clean drug-like molecules, 7 heavy atom types expanding to 12 DAE tokens). On GuacaMol (broader chemical space, charged species, 12 elemental types expanding to 56 DAE tokens), the validity claim holds (87.1% vs. 74.7%), but the distributional fidelity claim (FCD) does not (54.9 vs. 61.1). The paper acknowledges this tradeoff and attributes it partly to undertraining. The practical implication is that MolHIT's benefits are most pronounced on datasets where DAE's vocabulary expansion is moderate (2–3×) and where HDDM's group structure captures meaningful chemical relationships. When the vocabulary expands dramatically (4–5×) and the chemical space becomes more heterogeneous (charged species, hypervalent atoms), the distributional fidelity cost becomes non-trivial. The bridging of the validity-novelty gap—the paper's primary narrative—is well-supported on MOSES but untested on GuacaMol (where scaffold novelty metrics are not reported and 1D baselines are not compared). Future work would need to validate whether MolHIT occupies the same Pareto-optimal frontier position on more diverse benchmarks.

6. Limitations and Trade-offs

Limitation 1: The Benefits of DAE and HDDM Are Conditional on Moderate Vocabulary Expansion and Clean Chemical Data

The assumption or constraint. DAE's core mechanism is to expand the atom vocabulary by making explicit the chemical state (aromaticity, formal charge, hydrogen saturation) that standard encoding collapses into ambiguous tokens. This expansion works cleanly on MOSES, where the vocabulary grows from 7 to 12 tokens and the chemical space consists of neutral, drug-like molecules without formal charges. On GuacaMol, where the vocabulary expands from 12 to 56 tokens and the chemical space includes charged and hypervalent species, the learning problem becomes substantially harder. The paper acknowledges explicitly (Section 4.2):

"For FCD, the strong performance of original DiGress without DAE indicates that using DAE does not always lead to the generative task being easier since it can be hard to model with differentiate extended atom vocabulary."

The consequence. The expanded vocabulary changes from an unambiguous benefit to a potential liability as chemical complexity increases. On GuacaMol, DiGress+DAE (flat diffusion with expanded vocabulary) actually reduces validity from 74.7% to 65.2% compared to DiGress with standard encoding (Table 2). Even with HDDM, MolHIT's FCD on GuacaMol (54.9) is worse than original DiGress (68.0 on filtered data, 61.1 on full data), indicating that distributional fidelity degrades when the vocabulary becomes large and the chemical space heterogeneous. The practical implication is that DAE's benefits are not universal—they depend on the dataset's chemical diversity relative to the model's capacity to learn the expanded vocabulary. For very complex datasets (e.g., full ChEMBL with transition metals, radicals, isotopes), the vocabulary could explode to hundreds of tokens, making the learning problem intractable even with hierarchical diffusion.

What evidence exists in the paper. Table 2 provides the critical data point: DiGress+DAE on GuacaMol achieves only 65.2% validity (versus 74.7% for DiGress full), and MolHIT's FCD of 54.9 is worse than DiGress full's 61.1. The paper also notes (Section 4.2) that MolHIT is trained for only 40 epochs versus DiGress's 1,000 epochs, meaning the 87.1% validity figure is not at convergence and the FCD gap might narrow—but might also not. Figure 5 and Figure 6 show that DAE enables generation of charged species that coarse encoding models fail to produce, but this comes at the distributional fidelity cost captured by FCD. The paper does not measure how performance scales with vocabulary size—there is no experiment testing, for example, DAE at 20 tokens versus 50 tokens versus 100 tokens to characterize the difficulty curve.

Mitigation status. The paper is transparent about this limitation. It explicitly states (Appendix A, Limitations): "we have not fully trained the model until performance saturation on GuacaMol dataset and we believe performance improvement with further training." It also suggests scaling model size as a potential remedy: "we have not tested with the model size increase or architectural improvement, in which we believe have further room for better performance." HDDM partially mitigates the vocabulary expansion problem by providing hierarchical structure—MolHIT achieves 87.1% validity versus DiGress+DAE's 65.2% on GuacaMol—but the FCD gap suggests that hierarchical diffusion alone does not fully solve the distributional fidelity challenge on complex chemical spaces. The paper does not propose a principled way to determine how large a vocabulary DAE can support for a given model capacity, leaving this as an empirical question for practitioners to tune per dataset.

---

Limitation 2: The Hierarchical Grouping Strategy Is Chemically Motivated but Not Empirically Validated or Ablated

The assumption or constraint. HDDM's hierarchical structure depends on the grouping kernel Φ, which maps clean atom types to intermediate group states. The paper designs these groupings based on chemical domain knowledge: halogens together, chalcogens and pnictogens together, aromatics together, charged species clustered by charge sign (Tables 6, 7, 8). The implicit assumption is that these chemically motivated groupings are optimal or at least good enough that the choice doesn't matter much. However, the paper never tests this assumption. The authors state (Section 3.1):

"We implement a deterministic grouping kernel that clusters atom elements based on their intrinsic chemical properties and aromaticity."

but provide no ablation comparing this grouping against alternatives.

The consequence. Without an ablation of the grouping strategy, it is unknown whether HDDM's benefit comes from hierarchical structure per se or from the specific chemically informed groupings used. If a random grouping were equally effective, then HDDM's contribution would be purely about vocabulary factorization, and the chemical motivation would be post-hoc rationalization. If some groupings are substantially better than others, then HDDM requires chemical expertise to configure and may not transfer to domains where natural groupings are less obvious. The paper also does not test the sensitivity to the number of groups: MOSES uses 4 groups, GuacaMol uses 6 groups. Would 3 groups or 8 groups perform differently? Is there a risk of over-grouping (too many groups, losing the coarse-to-fine benefit) or under-grouping (too few groups, insufficient structure)? A practitioner applying HDDM to a new domain—say, protein residue generation or materials composition—would need guidance on how to design groups, but the paper provides only chemical examples, not general principles.

What evidence exists in the paper. The paper provides extensive detail on the groupings used (Tables 6, 7, 8 in Appendix D.2) but no experiments that vary them. The ablation in Table 5 shows HDDM adds +1.3 Quality and recovers FCD from 1.65 to 1.03 over the DAE+PN Sampler baseline, but this measures HDDM's contribution as a whole—it does not isolate the grouping choice. The paper also provides no learning curves comparing different grouping strategies, no analysis of which groups are most predictive or most frequently confused, and no measurement of whether the model actually learns the group structure (e.g., by analyzing whether group-level predictions are more accurate than atom-level predictions at intermediate timesteps, as the hierarchical theory would predict).

Mitigation status. The paper does not acknowledge this as a limitation. It presents the groupings as a natural design choice without discussing alternatives or validating the choice empirically. Future work could systematically explore: (1) random groupings versus chemically informed groupings, (2) data-driven groupings (clustering atoms by co-occurrence patterns in the training set), (3) sensitivity to the number of hierarchy levels (the paper generalizes HDDM to arbitrary levels in Proposition C.1 but uses only one intermediate level), and (4) whether the optimal grouping is dataset-specific. The paper's theoretical framework (Proposition C.1) supports arbitrary hierarchies, providing the mathematical scaffolding for such exploration, but the empirical validation is missing.

---

Limitation 3: Single Architecture, Single Parameter Count, No Scaling Analysis

The assumption or constraint. All experiments in the paper use the graph transformer architecture from DiGress (Vignac et al., 2022) with exactly 12 transformer blocks and approximately 16.2M parameters. The authors state this explicitly (Section 4): "For the model architecture, we utilize the original graph transformer from DiGress, maintaining the same model size." The implicit assumption is that the benefits of DAE and HDDM are architecture-agnostic and will transfer to larger or differently structured models. The paper also does not vary model size within the DiGress architecture—there are no experiments at, say, 4M, 8M, 16M, or 32M parameters to characterize how the benefits scale with capacity.

The consequence. Without scaling analysis, several critical questions remain unanswered. First, do the relative improvements from DAE and HDDM persist at larger scales? It is possible that a larger model with standard encoding and flat diffusion could match or exceed MolHIT's performance simply through increased capacity, making DAE and HDDM unnecessary at scale. The GuacaMol results hint at this: the expanded 56-token vocabulary is challenging at 16.2M parameters, but a 100M-parameter model might learn it effectively with flat diffusion, eliminating HDDM's advantage. Second, are DAE and HDDM complementary to architectural scaling, or do they substitute for it? If DAE+HDDM provide benefits equivalent to a certain amount of model scaling, practitioners need to know this tradeoff to make informed resource allocation decisions. Third, does the gap between MolHIT and baselines widen or narrow with more training? MolHIT on GuacaMol is explicitly undertrained (40 epochs vs. 1,000 for DiGress), and without scaling curves, the asymptotic performance is unknown.

What evidence exists in the paper. The paper provides no scaling experiments whatsoever. The GuacaMol results (Table 2) contain an implicit hint: DiGress full (74.7% validity, 1,000 epochs) versus MolHIT (87.1% validity, 40 epochs) suggests that architectural innovations can dramatically improve sample efficiency, but the ultimate performance ceiling is unknown. The ablation in Table 5 shows component contributions at a single model size. The paper acknowledges this limitation in Appendix A: "we have not tested with the model size increase or architectural improvement, in which we believe have further room for better performance." The reliance on a single architecture from 2022 (DiGress) also means the results are tied to a specific transformer design that may not represent the best available graph architecture as of the paper's publication.

Mitigation status. The paper explicitly identifies this as a limitation and future work direction (Appendix A). The authors frame it as an opportunity for further improvement rather than a fundamental weakness, which is reasonable—scaling studies are expensive and the paper's contribution is primarily methodological. However, a practitioner deciding whether to adopt MolHIT for a large-scale deployment would need to know whether the relative gains hold at production model sizes (potentially 100M–1B parameters). The paper provides no evidence either way. A minimal scaling experiment—say, training DiGress and MolHIT at 4M, 8M, and 16M parameters on MOSES—would have substantially strengthened the generalizability claims.

---

Limitation 4: Distributional Fidelity Degrades on Complex Chemical Spaces, Creating a Validity-FCD Tradeoff

The assumption or constraint. The paper's primary narrative is that MolHIT bridges the validity-novelty gap—achieving high validity like 1D sequence models while maintaining the structural exploration of graph models. The MOSES results (Table 1, Figure 1) support this narrative strongly. However, the GuacaMol results reveal a tension that the paper does not fully resolve: while validity improves dramatically (87.1% vs. 74.7%), distributional fidelity as measured by FCD worsens (54.9 vs. 61.1, where lower is worse for FCD). The paper acknowledges this (Section 4.2):

"For FCD, the strong performance of original DiGress without DAE indicates that using DAE does not always lead to the generative task being easier."

The consequence. The validity-FCD tradeoff means that MolHIT generates a higher fraction of chemically valid molecules, but the overall distribution of generated molecules—in the feature space of ChemNet, which captures both chemical and biological similarity—is less faithful to the test set distribution than DiGress. This has practical implications for drug discovery: validity ensures that generated molecules are chemically possible, but FCD captures whether they "look like" real drug-like molecules in a more holistic sense. A model with high validity but poor FCD might generate valid but chemically unusual or biologically irrelevant structures that would fail downstream filters or synthesis attempts even though they pass basic valency checks. The tradeoff is particularly concerning because MolHIT adds two mechanisms that could cause distributional drift: DAE expands the vocabulary to include rare chemical states that the model may not learn to generate at the correct frequencies, and the PN sampler adds variance during generation that can push samples off the training manifold (as evidenced by the FCD degradation from 0.89 to 1.65 when the PN sampler is added in Table 5).

What evidence exists in the paper. Table 5 provides the clearest decomposition of this tradeoff on MOSES. DiGress+DAE achieves FCD 0.89 (better than DiGress's 1.25). Adding the PN sampler degrades FCD to 1.65 (worse than DiGress). HDDM partially recovers to 1.03 (better than DiGress but worse than DAE alone). This shows that the PN sampler—which is critical for MolHIT's high validity (99.4%) and quality (92.9%)—is also the primary source of distributional drift. HDDM partially compensates, but does not fully restore the FCD achieved by DAE alone. On GuacaMol, the pattern is starker: MolHIT's FCD of 54.9 is worse than both DiGress full (61.1) and DiGress original on filtered data (68.0). The paper does not break down whether this FCD degradation comes primarily from the expanded vocabulary (some tokens generated at wrong frequencies), the PN sampler (off-manifold exploration), or the hierarchical structure (coarse-to-fine trajectory biases the generative path).

Mitigation status. The paper acknowledges the FCD issue on GuacaMol but attributes it partly to undertraining (40 epochs vs. 1,000). It does not investigate whether the FCD gap would close with full training or whether it represents a fundamental tradeoff. The temperature sampling analysis in Figure 4 shows that lowering top-p improves quality metrics (validity, drug-likeness) at the cost of scaffold novelty, but does not measure FCD as a function of top-p—this would directly characterize whether the PN sampler's FCD cost can be tuned via the temperature parameter. The paper does not propose a mechanism for controlling distributional fidelity independently of validity, leaving practitioners to choose between higher validity (MolHIT with PN sampler) and better distribution matching (DiGress+DAE without PN sampler).

---

Limitation 5: Training and Evaluation Are Limited to Two Datasets, Both from Medicinal Chemistry

The assumption or constraint. All experiments in the paper use either MOSES (1.9M drug-like molecules from ZINC) or GuacaMol (drug-like molecules from ChEMBL). Both datasets are drawn from medicinal chemistry and share similar chemical property distributions: moderate molecular weight (250–350 Da for MOSES), predominance of C, N, O, S, F, Cl, Br, and the absence of metals, organometallics, or highly reactive species. The paper's claims about MolHIT's performance are implicitly conditioned on this chemical domain.

The consequence. The paper does not demonstrate that DAE and HDDM generalize to chemically distinct domains. Several practically important molecular generation tasks involve chemistries that differ substantially from drug-like small molecules: polymers and materials (repeating units, elements like Si, P, B in non-drug-like contexts), catalysts and organometallics (transition metals with variable oxidation states and coordination geometries), natural products (complex fused ring systems, unusual heteroatom combinations), and peptides and biologics (repeating amide backbones, large graphs). In these domains, DAE's vocabulary expansion might need to encode different chemical attributes (coordination number, oxidation state, polymerization degree) that have different statistical structure than aromaticity and formal charge. HDDM's groupings—designed for medicinal chemistry elements—would need to be redesigned, and the optimal grouping strategy might differ substantially. The paper provides no evidence that the framework transfers.

What evidence exists in the paper. The paper's experiments are exclusively on MOSES and GuacaMol. The GuacaMol dataset pushes slightly beyond MOSES (including charged species, broader element set including I, Se, Si, B, P), but still falls within the medicinal chemistry domain. The paper does not test on materials-focused datasets (e.g., QM9, which contains small inorganic molecules), polymer datasets, or protein-related benchmarks. The related work section cites protein design (Gruver et al., 2023) and 3D molecular generation (Hoogeboom et al., 2022; Xu et al., 2023) as future directions, but does not evaluate on these tasks.

Mitigation status. The authors acknowledge this limitation implicitly by listing "apply into the 3D molecular generation and proteins" as future work (Appendix A). However, they do not discuss whether DAE and HDDM would need modification for these domains. A practitioner working on, say, catalyst design would need to determine: (1) what chemical attributes to encode in DAE for transition metals (oxidation state? spin state? coordination number?), (2) how to group elements when chemical similarity is more nuanced (Pt and Pd are chemically similar but have different catalytic properties; grouping them vs. separating them could have large consequences), and (3) whether the graph transformer architecture is appropriate for inorganic bonding patterns (coordination bonds, metal-metal bonds, delocalized charge). The paper's framework provides a template but no guidance for these domain-specific choices.

---

Limitation 6: Conditional Generation Shows Limited Benefit from Classifier-Free Guidance Beyond Unity Weight

The assumption or constraint. For multi-property guided generation (Section 4.3, Table 3) and scaffold extension (Section 4.4, Table 4), MolHIT uses classifier-free guidance (CFG) with a guidance weight of w = 1.0. The paper reports (Appendix D.8):

"We observe that in our discrete graph-diffusion framework, increasing the guidance weight beyond unity did not consistently yield better property alignment."

This is a negative result: CFG, which is a primary mechanism for improving conditioning quality in continuous diffusion models (where typical guidance weights range from 2.0 to 7.5), appears to saturate at w = 1.0 in MolHIT's discrete graph setting.

The consequence. The limited effectiveness of CFG above w = 1.0 means that MolHIT cannot further improve property conditioning by "pushing" the model toward the conditioned distribution. This caps the achievable property alignment accuracy, which may be insufficient for applications requiring precise molecular property control. The paper's best property correlation (Pearson r = 0.950 for logP) is strong, but other properties (MW: r = 0.685, SA: r = 0.790) show room for improvement that higher guidance weights might address if the mechanism worked. The failure of higher guidance weights also suggests a fundamental difference between discrete and continuous diffusion in how CFG interacts with the denoising process—possibly because the discrete state space and the PN sampler's projection step create discontinuities that CFG's linear interpolation cannot navigate effectively. If this is a fundamental limitation of discrete diffusion rather than an implementation issue, it would affect all discrete diffusion models seeking to use CFG for conditional generation.

What evidence exists in the paper. The paper provides no quantitative evidence for the claim that higher guidance weights don't help. There is no table or figure showing performance at w = 0.5, 1.0, 2.0, 3.0, 5.0, etc. The statement appears only in Appendix D.8 without supporting data. This is a significant omission because it means the reader cannot assess: (1) whether the CFG implementation is correct (perhaps the unconditional model was not properly trained with conditioning dropout, making CFG ineffective at any w), (2) whether there is a narrow optimal range around w = 1.0 that a finer sweep might reveal, or (3) whether the saturation is gradual (performance plateaus) or sharp (performance degrades above w = 1.0). The absence of this data also prevents comparison with other discrete diffusion conditional generation methods (e.g., Schiff et al., 2024, which the paper cites) that may achieve better guidance.

Mitigation status. The paper identifies this as an area for future work: "We leave the better design the sampler or models to be effective in higher guidance strength w as a promising avenue for future research" (Appendix D.8). However, without characterizing the current failure mode—does guidance above w = 1.0 produce invalid molecules, molecules that ignore the condition, or molecules that collapse to a narrow mode?—it is difficult to design fixes. The paper does not investigate whether the issue is specific to the PN sampler (which bypasses the analytical posterior and may not preserve the conditional structure that CFG relies on), to the use of AdaLN for conditioning (versus other conditioning mechanisms like FiLM or cross-attention), or to discrete diffusion generally. A practitioner needing strong property control would need to either (1) accept the limitation and work at w = 1.0, (2) experiment with alternative conditioning mechanisms not explored in the paper, or (3) use MolHIT for unconditional generation and apply post-hoc property prediction and filtering—all of which are untested alternatives.

7. Implications and Future Directions

How This Work Changes the Landscape

MolHIT does not introduce a new architecture, a new training paradigm, or a new generative modeling framework. It does something more subtle and arguably more broadly applicable: it demonstrates that careful representational engineering—fixing how atoms are encoded and how noise is structured during diffusion—can close a performance gap that the field had attributed to fundamental limitations of graph-based generation. The conceptual shift is from "graph models struggle with validity because molecules are hard to represent as graphs" to "graph models struggled with validity because we gave them an impoverished chemical vocabulary and a chemically ignorant noising process."

This is a diagnostic reframing, not a paradigm shift, but its implications ripple outward. The paper's decomposition of the validity problem into two root causes—an ill-posed reconstruction task from ambiguous encoding, and a flat diffusion process that ignores chemical group structure—provides a template for diagnosing other struggling generative models. When a model underperforms, ask: can it even reconstruct its training data perfectly? If not, the representation is the bottleneck, and no amount of architectural tuning will fix it. The reconstruction experiment in Figure 3 is the paper's most transferable methodological contribution: a simple, cheap diagnostic that isolates encoding failures from generation failures.

The paper also resolves a standing contradiction in the molecular generation literature: the tension between 1D sequence models (high validity, low scaffold novelty) and 2D graph models (low validity, high scaffold novelty). Prior work implicitly accepted this as an inherent tradeoff—that models must choose between chemical correctness and structural exploration. MolHIT shows that both limitations stemmed from the same underlying cause (inadequate representation) and that fixing it improves both axes simultaneously, shifting the Pareto frontier outward (Figure 1 bottom) rather than sliding along it. This reframes the validity-novelty tradeoff from a fundamental tension to an artifact of poor encoding—a reframing that, if it holds on other benchmarks and architectures, would change how researchers approach molecular generative model design.

The identification of verifier-free quality control via temperature and top-p sampling for molecular graphs (Figure 4, Section 3.3) is a smaller but practically significant shift. Prior graph diffusion models used standard posterior sampling without these controls, accepting whatever validity they produced. MolHIT demonstrates that simple inference-time hyperparameters can push validity from ~98.5% to 99.4% without architectural changes or additional training—a finding that transfers immediately to any discrete graph diffusion model, regardless of whether it adopts HDDM or DAE.

Research directions that become more attractive after this work:

• Representation-first diagnosis for generative models. The paper's reconstruction experiment provides a clear, low-cost test that should become standard practice. If a graph generative model underperforms, measure reconstruction success by atom type (not just aggregate validity) before tuning architecture or training hyperparameters.
• Hierarchical and domain-structured diffusion processes. HDDM's mathematical framework (Proposition C.1) generalizes to arbitrary hierarchy levels and arbitrary row-stochastic projections. This opens a design space: what natural taxonomies exist in a given domain, and how should they be encoded into the diffusion trajectory? Protein design (residue → secondary structure → domain → fold), materials generation (element → periodic table group → metal/nonmetal → crystal system), and program synthesis (token → AST node type → expression/statement → function) all have natural hierarchies that HDDM could exploit.
• Discrete diffusion with heterogeneous transition types. The paper's asymmetric treatment of atoms (HDDM) and bonds (uniform)—chosen empirically, not theoretically derived—suggests a design principle that might generalize: when modeling structured objects with heterogeneous components, apply different diffusion dynamics to each component type based on its statistical structure. This is underexplored in the discrete diffusion literature.

Research directions that become less attractive:

• Incremental improvements to flat discrete diffusion for molecular graphs. The paper shows that a chemically informed hierarchical process (HDDM) and expanded encoding (DAE) together provide gains that dwarf recent innovations in flat diffusion (marginal transitions in Cometh, flow matching in DeFoG). The 9.1 percentage point validity improvement from DAE alone (Table 5) exceeds the cumulative gains from prior diffusion process refinements. This suggests that further work on flat discrete diffusion transitions for molecular graphs is unlikely to yield substantial improvements unless it also addresses the encoding problem.
• Adding geometric or structural priors as auxiliary losses to improve validity. DiGress used geometric features (bond angles, cycle counts) as conditional information to improve validity. MolHIT removes these priors entirely ("we also remove calculating geometric prior originally used in Vignac et al., where they use extra graph features as conditional information. In our experiments, this has negligible effects on the performance"—Appendix D.5) and achieves dramatically better validity through encoding and diffusion structure alone. This suggests that auxiliary structural losses were compensating for representational deficiencies, and that fixing the representation makes them unnecessary.

Follow-Up Research This Work Enables

Systematic ablation of HDDM grouping strategies with a shared evaluation protocol. The paper designs chemically motivated groupings (halogens together, chalcogens together, aromatics together) but never tests whether these groupings are optimal or even beneficial compared to random groupings. A direct follow-up would train MolHIT with the same architecture, DAE encoding, and training budget on MOSES, varying only the grouping kernel Φ: (a) chemical groups (the paper's choice), (b) random groups of equal size, (c) groups based on periodic table rows/columns, (d) groups learned from atom co-occurrence statistics in the training set via clustering, (e) a single group (collapsing HDDM to standard absorbing diffusion), and (f) each atom in its own group (also collapsing to standard diffusion). The experiment would measure validity, FCD, and scaffold novelty across these conditions, answering whether HDDM's benefit comes from hierarchical structure generally or from chemically informed structure specifically. A negative result—random groupings performing as well as chemical groupings—would refine our understanding by showing that vocabulary factorization alone, not chemical knowledge, drives the improvement.

Scaling laws for DAE vocabulary size versus model capacity on GuacaMol. The paper shows that DAE's 56-token vocabulary on GuacaMol is too large for flat marginal diffusion to learn effectively (DiGress+DAE validity: 65.2%, Table 2) but manageable with HDDM (MolHIT validity: 87.1%). However, all experiments use a fixed ~16.2M parameter model, and MolHIT is undertrained at 40 epochs. A follow-up would train DiGress, DiGress+DAE, and MolHIT on GuacaMol at model sizes of 4M, 8M, 16M, 32M, and 64M parameters, each to convergence (measured by validation loss plateau), and measure validity, FCD, and KL divergence. The resulting curves would characterize: (1) whether DAE's expanded vocabulary becomes tractable with flat diffusion at sufficient model scale, (2) whether HDDM's benefit is primarily about sample efficiency (larger gap at small models and few epochs) or asymptotic performance (persistent gap at convergence), and (3) the compute-optimal allocation between model scaling and representation/diffusion design. This experiment directly addresses the paper's acknowledged limitation that "we have not tested with the model size increase or architectural improvement."

CFG guidance weight sweep with diagnostic sampling analysis. The paper claims higher CFG weights don't improve conditioning but provides no quantitative evidence. A follow-up would train the conditional MolHIT model on MOSES with four properties (as in Section 4.3) and evaluate at guidance weights w ∈ {0.0, 0.5, 1.0, 1.5, 2.0, 3.0, 5.0, 7.5, 10.0}, measuring MAE, Pearson r, validity, and uniqueness for each property at each weight. To diagnose why (or whether) higher weights fail, the experiment should include qualitative analysis: for w = 1.0 versus w = 5.0, sample 100 molecules conditioned on specific property targets and manually inspect whether failures are due to (a) valid molecules ignoring the condition, (b) invalid molecules that the PN sampler produces under strong guidance, or (c) mode collapse to a narrow set of molecules that satisfy the condition at the cost of diversity. If the failure mode is (b), the fix might involve constraining the PN sampler to stay closer to the posterior at high guidance; if (c), the fix might involve annealing the guidance weight during sampling. This experiment tests whether the CFG limitation is fundamental to discrete graph diffusion or specific to MolHIT's implementation.

Extension of DAE and HDDM to polymer and materials generation. The paper's chemical domain is exclusively drug-like small molecules (MOSES, GuacaMol). A natural stress test is to apply MolHIT to a materials-focused dataset like QM9 (small organic/inorganic molecules with up to 9 heavy atoms including diverse elements and bonding patterns) or a polymer dataset with repeating units. This would require: (1) designing DAE encodings for the new chemical space (what attributes matter for materials? oxidation state, coordination number, polymerization degree?), (2) designing HDDM groupings for elements not present in drug-like molecules (transition metals, metalloids, lanthanides), and (3) evaluating whether validity improvements transfer to chemistries where bonding rules differ (coordination bonds, metal-metal bonds, delocalized charge). A strong experiment would compare DiGress with standard encoding, DiGress+DAE, and MolHIT on QM9 at equal model size and training budget, measuring validity and distributional metrics. A negative result—DAE and HDDM providing no benefit or even degrading performance on inorganic molecules—would scope the paper's claims to medicinal chemistry and motivate domain-specific adaptations.

Combining HDDM with continuous-time discrete diffusion frameworks. The paper derives HDDM within a discrete-time, closed-form transition kernel framework (Lemma 3.1) but cites the continuous-time Markov chain (CTMC) framework (Campbell et al., 2022) as an alternative used by Cometh and DisCo. A natural integration would be to formulate HDDM's hierarchical forward process as a CTMC with rate matrices that encode the group structure, then derive the corresponding reverse-time SDE and training objective. This would: (1) enable HDDM to benefit from CTMC innovations like tau-leaping samplers and score-based training objectives, (2) allow direct comparison with Cometh and DisCo within their preferred framework, and (3) potentially resolve the CFG guidance limitation if the continuous-time formulation permits more stable guidance. The experiment would reimplement MolHIT's two-level hierarchy as a CTMC, train on MOSES, and compare validity, FCD, and conditioning quality against the discrete-time version at equal compute budget.

Training-free difficulty estimation via PRM score distribution for adaptive sampling. This direction is inspired by the compute-optimal test-time scaling literature but applied to MolHIT's generation process. The PN sampler already shows that temperature and top-p control a quality-diversity tradeoff (Figure 4). A follow-up could develop an adaptive sampler that estimates generation difficulty on-the-fly: start generating with top-p = 1.0 (most exploratory), measure the entropy of the model's predicted atom distributions at early denoising steps as a signal of uncertainty, and if uncertainty is high (hard molecule), reduce top-p to 0.8 for the remaining steps to improve validity. The evaluation would measure whether such adaptive sampling achieves the validity of top-p = 0.8 (99.4%) while maintaining the scaffold novelty of top-p = 1.0 (0.39), compared to fixed sampling at either extreme. This would require no retraining—it is purely an inference-time strategy—and would test whether MolHIT's generation uncertainty correlates with output validity.

Practical Applications and Downstream Use Cases

High-throughput virtual screening with validity guarantees. In a drug discovery pipeline that generates millions of candidate molecules for virtual screening, MolHIT's 99.1% validity (Table 1) versus DiGress's 87.1% means roughly 120,000 fewer invalid molecules per million generated—molecules that would otherwise waste downstream computation on docking simulations, synthetic accessibility assessment, and property prediction. For an organization running such a pipeline at scale, this directly translates to a ~13% reduction in wasted compute. Furthermore, MolHIT's scaffold novelty of 0.39 (versus SAFE-GPT's 0.12) means that among the valid molecules, a substantially larger fraction contain genuinely new structural frameworks not present in the training set, increasing the chance of discovering novel chemical matter. The temperature sampling knob (Figure 4) provides a practical mechanism for pipeline operators to dial the validity-novelty tradeoff without retraining: top-p = 0.8 for conservative screening (99.4% validity), top-p = 1.0 for exploratory campaigns.

Scaffold extension for lead optimization. In medicinal chemistry, lead optimization often starts from a core scaffold (a molecular framework with known activity) and explores chemical modifications that improve potency, selectivity, or pharmacokinetic properties. MolHIT's scaffold extension validity of 83.9% (Table 4) versus DiGress's 50.8% is a practical game-changer: more than 8 in 10 scaffold completions are chemically valid, versus roughly 1 in 2 for prior graph models. This means a medicinal chemist reviewing model outputs sees mostly chemically plausible suggestions rather than wasting time on impossible structures. The Hit@5 of 9.79% means that for roughly 1 in 10 scaffolds, the model recovers the known ground-truth completion within its top 5 suggestions—a useful sanity check that the model understands the scaffold's chemical context. For practical deployment, a chemist would input a scaffold of interest, generate 20–50 completions with MolHIT, filter to the valid set (~17–42 molecules), and evaluate them for synthetic feasibility and predicted properties—a workflow that becomes viable at 83.9% validity where it would be frustrating at 50.8%.

Property-conditioned library generation for multi-parameter optimization. Many drug discovery programs require simultaneously optimizing multiple properties: high drug-likeness (QED), good synthetic accessibility (SA), appropriate lipophilicity (logP), and molecular weight within a target range. MolHIT's multi-property guided generation achieves a macro-averaged MAE of 0.058 and near-perfect logP correlation (Pearson r = 0.950) with 96.31% validity (Table 3). A practical workflow would: (1) define target property ranges for a project (e.g., QED > 0.6, SA < 4, 2 < logP < 4, MW < 400), (2) generate 10,000 molecules conditioned on random targets within these ranges, (3) filter to valid molecules (~9,631 expected at 96.31% validity), and (4) select the subset meeting all property criteria. The 52.4% reduction in MAE versus the Marginal+DAE baseline means the generated molecules are substantially closer to their intended property targets, reducing the need for post-hoc filtering and increasing the usable fraction of generated output. The key practical advantage over sequence-based conditional models is MolHIT's higher scaffold novelty (0.39, Table 1)—the generated molecules are more likely to be structurally distinct from known compounds, which is valuable when seeking novel intellectual property.

Data generation for self-improving molecular models. The paper's framework can generate high-quality molecular graphs that serve as training data for downstream predictive models (property predictors, toxicity classifiers, retrosynthesis planners). Because MolHIT achieves 99.1% validity and high scaffold novelty simultaneously, generated data is both chemically reliable and structurally diverse—two properties that prior graph models could not deliver together. A self-improvement loop could: (1) train a property predictor on available labeled data, (2) use MolHIT with conditional generation to propose molecules with desirable predicted properties, (3) synthesize and test a subset of the most promising candidates, (4) add the experimental results to the training set, and (5) retrain the predictor. MolHIT's scaffold novelty ensures that each iteration explores genuinely new structural territory, while its validity ensures that computational resources are not wasted on impossible molecules. The importance of this application is amplified by the paper's finding (Figure 3 right) that prior models systematically under-generate specific motifs like pyrrolic nitrogen—a self-improvement loop using DiGress would inherit this bias and miss important regions of chemical space. MolHIT's corrected motif distribution means the training data for downstream models is more representative of true drug-like chemical space.

When to Prefer This Method

The paper articulates a clear tradeoff against prior graph diffusion models (DiGress, Cometh, DeFoG) and against 1D sequence models (SAFE-GPT, CharRNN), but does not frame an explicit decision rule against a named alternative with precisely quantified conditions. Instead, the tradeoff emerges implicitly from the experimental results:

• MolHIT should be preferred over prior graph diffusion models whenever chemical validity matters, which is essentially all practical applications. The 99.1% validity on MOSES (versus 87–93% for DiGress/Cometh/DeFoG) represents a categorical improvement—the failure rate drops by more than 6× (from 12.9% to 0.9% invalid), and the Quality score of 94.2% exceeds all published graph diffusion models by a wide margin (next-best: DeFoG at 88.5%). The only scenario where a prior graph model might be preferred is when distributional fidelity (FCD) is the sole metric of interest on chemically complex datasets—on GuacaMol, DiGress achieves better FCD (68.0 on filtered data) than MolHIT (54.9), though with substantially worse validity (85.2% vs. 87.1%). This is a narrow edge case that the paper itself suggests may close with further training.

• MolHIT should be preferred over 1D sequence models when structural novelty is a priority. SAFE-GPT achieves 99.8% validity on MOSES (slightly higher than MolHIT's 99.1%) but scaffold novelty of only 0.12 (versus MolHIT's 0.39). If the application requires generating molecules with scaffolds absent from the training set—as in exploratory drug discovery or novel materials design—MolHIT is the clear choice. If the application prioritizes validity above all else and novelty is irrelevant (e.g., generating slight variants of known drugs), a well-optimized 1D model might suffice at lower implementation complexity. The paper does not compare inference speed or implementation difficulty, which are practical considerations that might favor simpler sequence models in resource-constrained settings.

• When deploying MolHIT for practical generation, the temperature and top-p parameters (Figure 4) provide a continuous spectrum: top-p = 0.8 for maximum validity (99.4%) and quality (95.1%) at the cost of scaffold novelty (~0.32), top-p = 1.0 for maximum scaffold novelty (~0.39) at slightly reduced validity (~98.5%). The appropriate operating point depends on the application's tolerance for invalid outputs versus its appetite for structural exploration—and this choice can be made at inference time without retraining.