RigMo: Unifying Rig and Motion Learning for Generative Animation

🎯 Pitch

Animating 3D meshes has always required either artist-made rigs or separate rigging and motion models, but RigMo learns both—an explicit skeleton and how it moves—directly from raw mesh sequences, without any human annotations. The key insight is that rig structure can be inferred purely from observed deformations, enabling a feed-forward model that achieves 4× more efficient reconstructions than prior work and unlocks generative motion synthesis in a structure-aware latent space.

1. Executive Summary

This paper introduces RigMo, a unified generative framework that jointly learns rig structure and motion dynamics directly from raw mesh sequences without any human-provided rig annotations. Evaluated on DeformingThings4D, Objaverse-XL, and TrueBones using a VAE architecture with a dual-path encoder, RigMo factorizes per-vertex deformations into two compact latent spaces — a rig latent that decodes into explicit Gaussian bones and skinning weights (capturing static articulation structure), and a motion latent that produces time-varying SE(3) transformations (capturing temporal dynamics) — which together define an animatable mesh through differentiable Gaussian-based Linear Blend Skinning. The framework achieves a 4× improvement in reconstruction efficiency over sequence-aware baselines (matching AnimateAnyMesh's quality using only 48–128 tokens versus its 512-token representation) and outperforms both per-case optimization and auto-rigging pipelines in cross-motion generalization, establishing that robust rigging can be learned directly from observed motion without ground-truth skeleton annotations, though the approach remains bounded by the quality and diversity of the training motion data.

2. Context and Motivation

The Core Problem: Structure and Motion Are Inseparable, but Treated Separately

The paper addresses a fundamental architectural gap in how we model animated 3D content. In computer animation, rigging (defining a skeleton or articulation structure that controls how a mesh deforms) and motion (specifying how that articulation structure moves over time) are two sides of the same coin. A rig without motion is a static skeleton; motion without a rig is just raw vertex displacement with no reusable, interpretable, or controllable structure. Yet despite this deep interdependence, the paper argues (Section 1) that existing approaches treat rigging and motion as completely separate problems, each solved in isolation with its own assumptions and limitations.

This separation creates a cascade of practical problems that become increasingly acute as the demand for scalable, diverse animated content grows. To understand why this matters, consider what an artist or automated system typically confronts: they have a mesh sequence — perhaps captured from video, generated by a 4D reconstruction pipeline, or curated from a motion library — and they want to produce an animatable asset that can be reused, edited, and controlled in downstream applications. This requires, at minimum, a skeleton with per-bone transformations and skinning weights that map bone motions to vertex deformations. The current fragmented pipeline means that recovering this representation from raw data requires either (a) painstaking manual annotation by skilled artists, (b) automated rigging tools that predict skeletons from static geometry but cannot validate whether those predictions actually produce plausible motion, or (c) optimization-based inverse methods that fit a rig to each sequence individually but don't transfer across sequences or objects. None of these pathways scales to the volume of data that modern 4D generation and capture pipelines produce.

Why This Problem Is Important

The significance of unified rig–motion learning spans both practical and theoretical dimensions.

Practical: The prohibitive cost of manual rigging. The paper explicitly notes that "even experienced artists cannot rely purely on visual priors to assign correct skinning weights; iterative adjustment is always required to produce a rig that accurately reproduces target deformations" (Section 4.3). This means that rigging for a single character — even by a professional — is a trial-and-error process that can take hours or days. When multiplied across the tens of thousands of animated sequences now being generated by 4D diffusion models, video-to-4D pipelines, or large-scale motion capture, the bottleneck is severe. A system that could infer rig structure and motion jointly from raw deformation data, without human supervision, would eliminate this bottleneck entirely, enabling truly scalable pipelines for content creation in gaming, film, AR/VR, and simulation.

Practical: The fragmentation of downstream toolchains. When rigging and motion are produced by separate systems — an auto-rigger trained on artist-labeled skeletons, and a motion generator trained on pose sequences from a specific dataset — there is no guarantee that the rig from system A is compatible with the motion from system B. The paper observes that methods like UniRig and MagicArticulate can produce "visually plausible" skeletons (Figure 4), yet when those rigs are driven by motion parameters optimized for a different skeleton topology or bone convention, the resulting animations "collapse" or exhibit "severe deformation artifacts." This is a real, pervasive problem in production pipelines: mismatched rigging conventions between tools force engineers to write custom adapters, remap bone names, or simply re-rig assets from scratch. Jointly learned rig–motion representations inherently resolve this compatibility issue because the rig and motion are co-designed by the same model.

Theoretical: Structure must be learned from motion, not imposed a priori. The paper makes a deeper conceptual argument that "robust rigging cannot be inferred from static geometry alone and must be learned directly from motion" (Section 4.3). This challenges the fundamental assumption behind all auto-rigging methods (RigNet, UniRig, RigAnything, MagicArticulate) that a sufficiently powerful model can predict correct skeletons and skinning weights from a single static mesh. The counterargument is intuitive: a static mesh of a humanoid in a T-pose provides no information about whether the elbow should rotate with one or two degrees of freedom, whether the shoulder deformation should include cloth-like soft tissue, or whether the spine should be a single bone or a chain of vertebrae. These are functional properties that emerge from how the object actually moves. By learning rig structure from motion trajectories, RigMo discovers articulations that are validated by the data — they reproduce observed deformations because they were optimized to do so — rather than imitating what human artists happened to annotate in a particular dataset.

Scalability of unsupervised learning. The paper frames the self-supervised nature of RigMo as a scalability argument (Section 3.1.3): "A key advantage of RigMo is that it requires no rigging annotations or supervision during training... This eliminates the need for expensive manual rigging annotations from skilled artists, making the approach highly scalable to large-scale motion datasets." This matters because the existing paradigm — train auto-riggers on human-annotated datasets — faces a hard ceiling. Annotation is slow, expensive, inconsistent across artists (different artists define "shoulder" or "spine" bones differently), and heavily biased toward humanoid characters (since most rigging datasets come from game and film pipelines that primarily deal with bipeds). A self-supervised approach that learns from raw motion data alone can, in principle, scale to any deformable object category — animals, mythical creatures, mechanical assemblies, cloth simulations — without hitting these annotation bottlenecks.

Where Prior Approaches Fall Short

The paper structures the prior work landscape into three broad paradigms, each of which fails to address the core gap in a different way (Section 2, with the motivation laid out in Section 1).

Auto-Rigging Systems: Learning from Labels, Not from Motion

Auto-rigging methods (RigNet, UniRig, MagicArticulate, RigAnything, Puppeteer) aim to predict skeletons and skinning weights from static meshes. The paper acknowledges their progress — particularly the autoregressive approaches that treat skeleton generation as a sequence prediction problem — but identifies several critical limitations (Section 2, Section 4.3):

Dependency on artist annotations. These methods are trained on human-rigged datasets, which imposes hard constraints. First, the datasets are relatively small (compared to the volume of unlabeled motion data available) because manual rigging is expensive. Second, the annotations are inconsistent — different artists use different skeletons, bone counts, naming conventions, and skinning weight assignments for the same or similar characters. This inconsistency propagates into the trained models, which learn to average over disparate annotation styles rather than discovering a physically optimal articulation.

No validation against motion. A predicted skeleton might look reasonable when visualized on a static mesh but fail to produce coherent motion when animated. The paper demonstrates this concretely with UniRig (Figure 4): the fox rigging appears "visually plausible" for skinning weights, yet the animation "collapses under actual animation, leading to severe deformation artifacts." This is because auto-riggers never see motion during training — they optimize for predicting human labels from static geometry, not for producing deformations that match real movement. As the paper puts it (Section 4.3), "even experienced artists cannot rely purely on visual priors to assign correct skinning weights; iterative adjustment is always required to produce a rig that accurately reproduces target deformations." An auto-rigger trained to imitate artist labels learns the priors without the adjustment loop.

Poor cross-category generalization. Auto-riggers trained on humanoid-heavy datasets struggle with non-humanoid shapes (animals, quadrupeds, fantastical creatures, mechanical objects) because the skeleton topologies and joint hierarchies differ fundamentally. The paper notes that template-based approaches like Baran and Popović (2007) "generalize poorly beyond humanoid shapes." While template-free methods like RigNet relax assumptions on joint topology, they may compensate by producing "overly dense skeletons or rely on handcrafted geometric cues" (Section 2) — not because the object needs that many bones, but because the model lacks the motion signal needed to determine the minimal sufficient articulation.

Motion Generation with Predefined Rigs: Assuming Structure That Doesn't Exist

A large body of work in human motion generation (SMPL-based models, MDM, MoDi, MaskControl, SALAD) and animal/object animation (AnyTop, 3D Menagerie, single-motion diffusion) operates in pose space — predicting joint rotations or SE(3) transforms on top of a predefined kinematic structure (Section 1). These methods assume that a ground-truth skeleton, with known bone topology, joint limits, and skinning weights, already exists. The limitations are stark when this assumption breaks:

Cannot infer the structure itself. These methods have no mechanism for discovering how many bones an object should have, where they should be located, or how they should influence the surface. They can only predict motion given a rig, not discover the rig from motion. This means they are fundamentally inapplicable to the common scenario where only raw mesh sequences are available and no rig exists.

Cannot handle arbitrary geometries. The rigid coupling to specific skeletal templates (SMPL for humans, specific quadrupeds for animals) means these methods fail when the input mesh topology or kinematics differ from the training distribution. Animate a dragon with a human skeleton? Impossible. Animate a slime monster with no clear articulation? The concept of fixed bone hierarchies breaks down entirely.

Assumption of consistent rigging conventions. Even within the human domain, different datasets use different skeletons (SMPL, SMPL-X, MANO, custom game rigs). A motion generator trained on SMPL poses cannot directly drive a character rigged with a different joint hierarchy or different bone coordinate frames. The paper explicitly mentions that "rigging conventions vary widely across datasets and artists" (supplementary material, Comparison with prior work), making cross-dataset generalization a persistent challenge.

Rig-Free Vertex-Space Methods: Flexibility Without Interpretability

The most recent paradigm avoids rigging entirely. Methods like AnimateAnyMesh, GVFDiffusion, Sync4D, and various 4D reconstruction approaches predict per-vertex deformations directly — frame by frame, without any structural abstraction (Section 1). The paper recognizes the flexibility of this approach (no skeleton templates, no joint limits, no skinning assumptions) but identifies a different set of limitations:

Difficult to control and edit. A vertex-space deformation field is a black box. There is no separation between "the arm swings forward" and "the fingers curl" — all deformations are entangled in a single high-dimensional per-vertex representation. This makes downstream tasks like motion retargeting (transferring a motion from one character to another), motion editing (e.g., "make the wave more energetic"), or partial animation control (e.g., "animate only the tail while keeping the body still") essentially impossible without additional post-hoc structure recovery.

Hard to interpret. Understanding why a vertex-space model produced a particular deformation requires analyzing thousands of per-vertex displacement vectors with no semantic grouping. An animator cannot look at the output and reason about which "joint" is responsible for which motion because the concept of a joint doesn't exist in the representation.

Cannot produce reusable animatable assets. The core purpose of rigging in production is to produce assets that can be reused across scenes, sequences, and projects — an animatable character that can be dropped into any environment and driven with new motion. Vertex-space methods produce a one-shot animation: the output is the animated sequence itself, not a rigged model that can be re-animated. As the paper puts it, these methods are "unable to produce reusable animatable assets—the core purpose of rigging."

Computational and memory inefficiency. The paper notes (Section 4.4) that AnimateAnyMesh requires a "high-dimensional 512-token representation to encode object motion," while RigMo achieves comparable or superior reconstruction quality using only 48–128 tokens. This is not just a minor efficiency gain — it reflects a fundamental difference in representational compactness. Vertex-space methods must redundantly encode both structure and motion in a single, high-dimensional per-frame representation, while RigMo separates structure (a small set of bones, static) from motion (bone transforms, dynamic), enabling dramatically more efficient encoding.

Optimization-Based Inverse Skinning: Per-Case, No Generalization

A fourth category, closer in spirit to RigMo's goals, is optimization-based inverse skinning (S3O, PhysRig, LASR, MagicPony, Ponymation). These methods take a mesh sequence and fit a rig + motion model via per-sequence nonlinear optimization (Section 2). While they can produce high-quality results for individual sequences, the paper identifies fundamental scaling limitations:

Per-case optimization is slow. Each new sequence requires solving a separate optimization problem from scratch, which can take minutes to hours per sequence. This makes these methods "difficult to scale" (Section 2) to the 20,000+ sequences in RigMo's training corpus, let alone to the tens of millions of assets that modern 4D generation systems could produce.

No generalization across sequences. Because the rig is optimized independently for each sequence, there is no shared representation between sequences of the same or similar objects. The rig learned for one running sequence of a particular character doesn't transfer to a jumping sequence of the same character — each must be optimized separately. As the paper states, "the recovered rigs are tightly bound to individual sequences and fail to generalize across subjects or motion styles."

No feed-forward inference capability. These methods cannot, given a new unseen mesh sequence, immediately produce a rig and motion in a single forward pass. They require iterative optimization at test time. This makes them unsuitable for real-time or interactive applications and fundamentally incompatible with modern generative pipelines where fast inference is essential.

The Integration Gap: No Joint Model Exists

The paper's central observation is that none of these paradigms addresses the joint problem — learning a rig and its motion from raw data in a unified, feed-forward, generalizable way. Auto-riggers learn from static geometry but ignore motion. Motion generators assume rigs already exist. Vertex-space methods avoid rigs entirely. Optimization-based methods recover both but don't generalize. The paper frames this gap explicitly (Section 1):

"Together, these paradigms expose a fundamental gap: there is no unified framework that jointly learns rig structure and motion dynamics directly from raw mesh sequences, without predefined skeletons or per-sequence optimization."

This gap is not merely academic. It means that the emerging pipeline of 4D content generation (diffusion models producing mesh animations from text or video, capture systems producing mesh sequences of real-world deformable objects, simulation systems outputting physically-based deformation sequences) has no principled, scalable way to convert its output into animatable, reusable assets. The raw output is either vertex streams (uncontrollable, non-reusable) or requires manual post-processing (expensive, non-scalable).

How RigMo Positions Itself Relative to Existing Work

RigMo positions itself as the first unified generative framework that jointly learns rig and motion directly from raw mesh sequences, without any human-provided rig annotations (Section 1, Section 3). This positioning has several dimensions that the paper develops.

Unification rather than separation. Unlike the fragmented landscape where auto-rigging, motion generation, and 4D reconstruction are separate research communities with separate assumptions, RigMo proposes a single model that produces both a rig (Gaussian bones + skinning weights) and motion (SE(3) bone transforms) from the same input. The key architectural insight is the dual-path encoder (Section 3.1.1): a rigging branch processes the canonical (first-frame) geometry to predict bone parameters, while a motion branch processes per-frame vertex displacements to predict motion latents. These branches share bone token coordinates, ensuring that the rig structure and motion trajectories are semantically aligned — the bones that the rigging branch predicts are the same bones that the motion branch animates. This co-design eliminates the compatibility problem that plagues separate rigging and motion systems.

Self-supervised learning from motion, not supervised imitation of labels. The paper explicitly contrasts RigMo with auto-rigging methods that depend on human annotations. RigMo is trained with "only two self-supervised objectives: a vertex-level reconstruction loss and a latent regularization term" (Section 3.1.3). There is no term in the loss function that encourages the predicted bones to look like artist-designed skeletons, or the skinning weights to match human-painted weight maps. The only signal is whether the rig + motion can reconstruct the observed vertex trajectories. This means the model must discover an articulation structure that is functionally sufficient to explain the data, rather than trying to match a particular annotation style. The paper positions this as a scalability argument, but it is also a theoretical one: the representation discovered by RigMo is grounded in physical observation (what motions actually occur) rather than human convention (what artists typically draw).

Feed-forward inference, not per-case optimization. Unlike optimization-based inverse skinning methods (S3O, PhysRig, LASR), RigMo is a feed-forward model: given an input mesh sequence, it produces a rig and motion in a single forward pass (Section 4.2 reports ~40 ms per frame on an A100). This makes it compatible with large-scale batch processing and real-time applications. The model generalizes across sequences because it learns a shared representation from the entire training corpus rather than optimizing parameters independently for each sequence. The rigging evaluation protocol (Section 4.3) is designed specifically to test this: rig parameters are inferred from training sequences and then tested on held-out motion sequences of the same object, measuring whether the learned rig transfers to novel motions.

Gaussian bones as a compact, continuous, resolution-agnostic abstraction. The paper introduces Gaussian bones (Section 3.1.3) as a specific representation choice that differentiates RigMo from both discrete joint-based skeletons (common in auto-rigging and motion generation) and vertex-space representations (common in 4D generative models). A Gaussian bone is a 3D ellipsoid defined by its center $\mathbf{c}_k$ , anisotropic scale $\mathbf{s}_k$ , and orientation quaternion $\mathbf{q}_k$ . It acts as a soft region of influence — vertices near the bone center and aligned with its axes are strongly influenced; vertices far away or misaligned are weakly influenced. This representation is continuous (no discrete joint hierarchy; bones float in space), compact (48–128 bones suffice for diverse characters, versus 5K+ vertices or 512 tokens in alternative methods), and resolution-agnostic (the bone parameters don't depend on mesh vertex count, so the same rig can deform meshes at different resolutions — Section 4.6). This last property is practically significant because it means RigMo can process downsampled meshes during training (for efficiency) and then apply the learned rig to the full-resolution mesh at inference time without loss of fidelity.

Structure-aware latent space as a foundation for controllable generation. The paper's second contribution — the Motion-DiT (Section 3.2) — demonstrates that RigMo's latents are not just good for reconstruction but serve as a meaningful representation for downstream generative tasks. By operating in RigMo's motion latent space, Motion-DiT generates temporally coherent bone transformations that are guaranteed to be compatible with the learned rig structure (since the rig is used as a conditioning signal). This is a different approach from motion generation methods that operate directly in vertex space or on predefined skeletons: the generated motions are bound to the learned articulation, so the output is always a valid animation of the rigged asset.

Empirical positioning: outperforming both paradigms. The experimental comparisons (Section 4.3, 4.4) are designed to position RigMo against representatives of each competing paradigm:

Against per-case optimization: RigMo's feed-forward rigs achieve lower cross-motion reconstruction error (Table 1), demonstrating that the shared representation generalizes better than independently optimized per-sequence rigs. The optimization baseline fails on test sequences because the rig optimized for one motion doesn't transfer to another — RigMo's shared learning overcomes this.
Against auto-rigging pipelines (UniRig + optimization, MagicArticulate + optimization): RigMo achieves substantially lower reconstruction error and, critically, produces rigs that actually work when animated (Figure 4). The auto-rigger baselines produce "visually plausible" skeletons but fail to reproduce real motions because they were trained to predict artist labels from static geometry, not to produce functionally correct deformations.
Against sequence-aware 4D VAEs (AnimateAnyMesh): RigMo achieves comparable or better reconstruction fidelity using 4–10× fewer latent tokens (48–128 vs. 512) and 4× faster inference (Table 2). This is because RigMo's Gaussian bone representation disentangles structure (static, compact) from motion (dynamic, per-bone transforms), while AnimateAnyMesh must encode everything into a single high-dimensional per-frame latent.
Against frame-independent 3D VAEs (Step1X3D, Hunyuan3D 2.1): These methods perform per-frame generation and "inherently break vertex correspondence across frames" (Section 4.4), leading to temporal inconsistency and loss of surface detail. RigMo, by contrast, produces temporally coherent deformations through explicit skinning with consistent bone-to-vertex mappings.

A Note on What RigMo Does NOT Address

The paper is careful not to overclaim. It explicitly acknowledges that the primary focus is the RigMo-VAE — the rig–motion decomposition and reconstruction — with Motion-DiT serving as a demonstration of latent space utility rather than a fully developed motion generation system (Section 4 opening: "the main contributions of this work lie in the rigging and motion decomposition capabilities of the VAE itself"). The paper does not claim to solve all problems in 4D generation; it specifically addresses the gap of unified rig–motion discovery, leaving the full integration with state-of-the-art diffusion models, text conditioning, and interactive editing as natural extensions that the framework enables but does not itself complete.

This positioning matters because it clarifies what RigMo replaces and what it complements. RigMo replaces the auto-rigging + separate motion optimization pipeline with a single feed-forward model. It provides a compact latent space that downstream generative models (like Motion-DiT, or future text-conditioned diffusion models) can operate on. But RigMo itself is not a text-to-animation system or a 4D reconstruction-from-video system — it is the foundational representation layer that makes those systems produce structured, controllable, reusable outputs rather than raw vertex streams.

3. Technical Approach

3.1 Reader orientation (approachable technical breakdown)

What the system is: RigMo is a feed-forward variational autoencoder (VAE) that takes a sequence of deforming 3D meshes as input and decomposes them into two interpretable, reusable components: a rig (a set of soft, ellipsoid-shaped "bones" with per-vertex skinning weights that define how the mesh can deform) and motion (time-varying SE(3) transformations—rotations and translations—that drive those bones to produce the observed deformations). A companion Motion-DiT diffusion model can then generate new motion trajectories in RigMo's compact, structure-aware latent space.

What problem it solves and the "shape" of the solution: The paper addresses the fundamental gap that no existing system jointly learns rig structure and motion dynamics from raw mesh sequences without human-provided skeleton annotations. The solution has a factorized encoder-decoder architecture: a dual-path encoder separates static geometry (processed by a rigging branch) from temporal dynamics (processed by a motion branch) into two latent spaces; a decoder maps these latents to explicit Gaussian bones, skinning weights, and per-frame bone transformations; and a differentiable Gaussian-based Linear Blend Skinning (LBS) module synthesizes the final vertex positions. The entire pipeline is trained end-to-end with only a vertex reconstruction loss and a KL regularization term—no rigging supervision, no joint labels, no skeleton templates. The "shape" is thus a self-supervised, feed-forward system that discovers articulation structure from the necessity of reconstructing motion, rather than imitating human annotations.

3.2 Big-picture architecture (diagram in words)

The RigMo framework consists of five major components connected in sequence:

Topology-Aware Encoder (dual-path): A Rigging Branch processes the canonical (first-frame) mesh geometry to produce bone tokens and Gaussian bone parameters; a Motion Branch processes per-frame vertex displacements to produce local and global motion latent variables. Both branches share bone token coordinates, ensuring that the rig structure and motion trajectories are semantically aligned.
Rig-Motion Decoder: Maps the latent codes to physically meaningful outputs—a static Gaussian bone decoder produces bone centers, scales, and orientations; local and root motion decoders produce per-bone SE(3) transformations (rotation quaternions and translation vectors).
Gaussian Skinning LBS Module: Computes per-vertex skinning weights as normalized Mahalanobis distances to each Gaussian bone (soft, spatially-varying influence regions), refines these weights using geodesic-aware masking to enforce topological coherence, and applies Linear Blend Skinning to deform vertices according to the predicted bone transformations.
Self-Supervised Training Loop: The entire VAE is trained end-to-end with only a vertex-level L2 reconstruction loss and a KL divergence regularizer on the motion latents—no rigging ground truth, no joint labels, no skeleton templates.
Motion-DiT (companion diffusion model): A diffusion transformer operating in RigMo's motion latent space, conditioned on static rigging features (anchor tokens, global token) and observed frame masks, that generates or interpolates motion trajectories compatible with the learned rig structure.

Information flow: Input mesh sequence $\mathbf{V} \in \mathbb{R}^{B \times T \times N \times 3}$ → Rigging Branch produces bone tokens $\mathbf{B}^{\text{emb}}$ and Gaussian parameters $\mathbf{G}$ ; Motion Branch produces posterior parameters $[\boldsymbol{\mu}, \boldsymbol{\sigma}]$ for local and root latents → Sample $\mathbf{z}_{\text{local}}, \mathbf{z}_{\text{root}}$ via reparameterization → Motion decoders produce $\{\mathbf{q}_{\text{local}}, \mathbf{t}_{\text{local}}, \mathbf{q}_{\text{root}}, \mathbf{t}_{\text{root}}\}$ → Gaussian Skinning LBS computes skinning weights $\mathbf{W}$ via Mahalanobis distances, refines with geodesic masks, and applies LBS to produce reconstructed vertices $\hat{\mathbf{V}}$ → Reconstruction loss computed against input $\mathbf{V}$ , KL loss regularizes latents.

3.3 Roadmap for the deep dive

First, the Topology-Aware Encoder (Section 3.1.1), since it is the entry point where raw mesh sequences are transformed into structured latents, and its dual-path design (rigging branch vs. motion branch) encodes the core insight of factorization.
Second, the Rig-Motion Decoder (Section 3.1.2), which maps the latent codes to physically meaningful Gaussian bone parameters and per-frame SE(3) transformations—the actual rig and motion outputs.
Third, the Gaussian Skinning LBS Module (Section 3.1.3), which is the differentiable rendering step that converts rig + motion into vertex positions. This is where Gaussian bones are defined, skinning weights are computed, geodesic refinement is applied, and Linear Blend Skinning synthesizes the final output. The training objectives are also covered here since they operate on the LBS output.
Fourth, the Motion-DiT (Section 3.2), which demonstrates that RigMo's latent space supports downstream generative tasks—conditioned on static rigging features and observed frame masks, it generates or interpolates motion trajectories in the learned latent space.

3.4 Detailed, sentence-based technical breakdown

This is primarily a representation learning paper whose core idea is that a factorized VAE architecture can discover interpretable rig structures and motion dynamics jointly from raw deformation data without any rigging supervision, and that this learned representation enables both high-fidelity reconstruction and controllable motion generation.

Topology-Aware Encoder: Dual-Path Disentanglement of Structure and Motion

The encoder is the architectural heart of RigMo—where the decision to separate static geometry from dynamic motion is implemented. Given an input mesh sequence $\mathbf{V} \in \mathbb{R}^{B \times T \times N \times 3}$ (batch size $B$ , $T$ frames, $N$ vertices per mesh, each with 3D coordinates), the encoder produces two distinct latent representations through parallel processing paths that share a common set of bone token coordinates.

Why a dual-path design? The core motivation is disentanglement: the rig (which bones exist, where they are, how they influence the surface) should depend only on the object's canonical shape, not on a particular motion instance. Conversely, the motion (how bones transform over time) should depend on the temporal dynamics, not on re-discovering the skeleton for each frame. A single-path encoder would conflate these signals—the rig representation would be contaminated by motion-specific information, hurting generalization to novel motions, and the motion representation would be entangled with shape, hurting transfer across objects. The dual-path design enforces this separation architecturally: the rigging branch sees only the first frame (canonical pose), while the motion branch sees the full temporal sequence of vertex displacements.

Rigging Branch: Predicting Gaussian Bones from Canonical Geometry

The rigging branch takes the first-frame mesh vertices $\mathbf{V}_0 \in \mathbb{R}^{N \times 3}$ as input and outputs Gaussian bone parameters $\mathbf{G}_k = [\Delta\mathbf{c}_k, \mathbf{s}_k, \mathbf{q}_k]$ for each of $K$ bones, plus bone–vertex correlation features $\mathbf{A}_{\text{rig}}$ .

Step 1: Per-vertex embedding via topology-aware attention. The input vertices are encoded into per-vertex embeddings $\mathbf{V}_0^{\text{emb}}$ using a graph attention mechanism that respects mesh connectivity:

$\mathbf{h}_{\ell} = \text{Attn}(\text{LN}(\mathbf{h}_{\ell-1}), \mathcal{N}) + \mathbf{h}_{\ell-1}$

where $\mathbf{h}_\ell$ is the hidden state at layer $\ell$ , LN is Layer Normalization, Attn is a self-attention operation restricted to the local graph neighborhood $\mathcal{N}$ (the $k=5$ nearest neighbors in the mesh graph), and $\mathbf{h}_{\ell-1}$ is the residual connection. This is applied for 6 layers with hidden dimension 256 and 8 attention heads.

What it computes: Each vertex's embedding is updated by attending to messages from its topologically adjacent neighbors (not all vertices), then adding the original input via residual connection. The output $\mathbf{h}_\ell$ is a context-aware per-vertex feature vector that encodes local geometric information from the mesh surface.

Why this form: Graph-based attention restricted to mesh neighbors (rather than full self-attention over all vertices) preserves the topological structure of the mesh—vertices that are connected by edges in the mesh exchange information, while spatially close but topologically distant vertices (e.g., the inner thighs of a humanoid in T-pose) do not directly influence each other's embeddings. This is crucial for learning correct bone–vertex correspondences: a bone should influence a contiguous surface region, not spatially nearby but topologically disconnected parts. The encoder uses 6 layers (Section 4.2: "six topology-aware attention layers (hidden dim 256, eight heads, neighborhood size k=5)").

Step 2: Bone token selection via Farthest Point Sampling (FPS). From the $N$ vertices, $K$ bone tokens are selected using Farthest Point Sampling on the vertex coordinates. The number $K$ is a hyperparameter: the paper sweeps $K=48$ and $K=128$ , finding that 48 provides "a more favorable balance between efficiency, interpretability, and stability" while 128 yields marginally better reconstruction (0.018% improvement in CD-L1, per Table 3). The selected vertices' coordinates become $\mathbf{C}_{\text{bone}}$ , and the corresponding vertex embeddings are used to initialize bone token features $\mathbf{B}^{\text{emb}}$ .

What it computes: FPS iteratively selects the vertex farthest (in Euclidean distance) from all previously selected vertices, ensuring the $K$ bone tokens spatially cover the mesh surface roughly uniformly. This provides a diverse set of initial "bone candidates" distributed across the object's geometry.

Why this form: FPS is a simple, deterministic, geometry-aware sampling strategy that does not require learning. It guarantees coverage of the mesh surface, which is important because bones that are clustered in one region would fail to model deformations elsewhere. Unlike learned token selection (which might converge to suboptimal local minima during training), FPS provides a stable, predictable initialization that the subsequent cross-attention layers can refine.

Step 3: Bone–vertex cross-attention. The bone tokens $\mathbf{B}^{\text{emb}}$ attend to the vertex embeddings $\mathbf{V}_0^{\text{emb}}$ via cross-attention:

$\mathbf{A}_{\text{rig}} = \text{CrossAttn}(\mathbf{B}^{\text{emb}}, \mathbf{V}_0^{\text{emb}}, \mathbf{V}_0^{\text{emb}})$

where the cross-attention operation uses bone tokens as queries, vertex embeddings as keys and values. The output $\mathbf{A}_{\text{rig}}$ contains per-bone features that aggregate information from the vertices most relevant to each bone.

What it computes: For each bone token, cross-attention computes a weighted sum of all vertex embeddings, where the weights are learned similarities between the bone token query and each vertex embedding key. This effectively lets each bone "look at" the vertices and aggregate information about the geometric region it should influence, producing bone-conditioned features that encode both where the bone is and what surface region it should deform.

Why this form: Cross-attention is the standard mechanism for information routing between two sets with different cardinalities—here, $K$ bone tokens and $N$ vertices. The learned attention weights will (ideally) assign high similarity between each bone token and the vertices in its anatomical region (e.g., the "left arm bone" should attend strongly to vertices on the left arm), producing bone features that are specialized to different surface regions. This specialization emerges from the reconstruction objective: bones that attend to coherent, contiguous vertex groups will produce more accurate skinning weights and thus lower reconstruction loss.

Step 4: Gaussian parameter prediction. From the bone features $\mathbf{A}_{\text{rig}}$ , an MLP predicts the Gaussian parameters for each bone:

$\mathbf{G}_k = [\Delta\mathbf{c}_k, \mathbf{s}_k, \mathbf{q}_k] = \text{MLP}(\mathbf{A}_{\text{rig}, k})$

where $\Delta\mathbf{c}_k \in \mathbb{R}^3$ is a displacement from the bone token's FPS-sampled position to the actual bone center, $\mathbf{s}_k \in \mathbb{R}^3$ are anisotropic scaling factors (defining the ellipsoid's extent along each axis), and $\mathbf{q}_k \in \mathbb{R}^4$ is a unit quaternion defining the bone's orientation.

What it computes: The MLP outputs three geometric quantities per bone: a center offset (so the bone can shift from its initial FPS position to a more optimal location), anisotropic scales (so the bone's influence region can be elongated along certain axes—e.g., a limb bone is long along the limb direction, narrow perpendicular to it), and an orientation quaternion (so the ellipsoid can rotate to align with the local geometry).

Why this form (Gaussian bones vs. discrete joints): Traditional skeletons use discrete joints connected by rigid bones with hard influence boundaries—a vertex is either influenced by a joint or it isn't. Gaussian bones are soft, continuous influence regions: each bone is a 3D Gaussian ellipsoid, and a vertex's skinning weight depends on its Mahalanobis distance to the bone center (Equation 11). This has several advantages: (a) soft boundaries produce smoother, more natural deformations around joints; (b) the continuous parameterization is fully differentiable, enabling gradient-based optimization; (c) bones are not constrained to form a connected hierarchy—they float independently in 3D space, allowing the model to discover non-tree-structured articulations if the data supports them; (d) the representation is resolution-agnostic since bones are defined in continuous 3D space, not tied to specific vertices.

Motion Branch: Encoding Temporal Dynamics into Variational Latents

The motion branch processes per-frame vertex displacements to predict variational latent variables that govern local (per-bone) and global (root) motion. It does not see the canonical geometry—only how vertices move relative to the previous frame.

Step 1: Compute per-frame vertex displacements. The motion branch does not operate on absolute vertex positions but on frame-to-frame differences:

$\mathbf{V}_{\Delta} = \mathbf{V}[:, 1:, :, :] - \mathbf{V}[:, :-1, :, :] \in \mathbb{R}^{B \times (T-1) \times N \times 3}$

This subtraction removes the canonical shape information, leaving only the motion signal. The output has $T-1$ frames (since $T$ frames produce $T-1$ displacement vectors).

What it computes: For each vertex $i$ and each frame transition $t \to t+1$ , the displacement $\mathbf{v}_{i, t+1} - \mathbf{v}_{i, t}$ captures how that vertex moved in 3D space. A stationary object would produce near-zero displacements; an articulated motion produces structured, spatially coherent displacement patterns.

Why this form: Subtracting consecutive frames removes static geometry information, forcing the motion branch to focus exclusively on dynamics. If the branch instead processed absolute vertex positions, it would need to re-encode shape information already captured by the rigging branch, creating redundancy and potential entanglement. The displacement representation is also translation-invariant: a global translation of the entire object produces uniform displacements, which can be captured by the root motion decoder as a single global transformation rather than being distributed across all bones.

Step 2: Temporal–spatial encoding of displacements. The displacement tensor $\mathbf{V}_{\Delta}$ is processed through temporal–spatial attention layers to produce per-vertex per-frame motion embeddings $\mathbf{V}_{\Delta}^{\text{emb}} \in \mathbb{R}^{B \times (T-1) \times N \times d_b}$ . These layers apply attention alternately across the temporal dimension (for each vertex, attending across frames) and the spatial dimension (for each frame, attending across vertices' graph neighborhoods).

What it computes: The temporal attention lets the model learn motion patterns—a vertex that oscillates, accelerates, or rotates will develop temporal features that capture these dynamics. The spatial attention lets vertices share motion information with their neighbors, encouraging coherent, physically plausible deformation patterns (neighboring vertices on the same rigid body part should move similarly).

Why this form: Alternating temporal and spatial attention is the standard transformer approach for spatiotemporal data (common in video models). It avoids the quadratic complexity of full joint attention over all vertices and all frames (which would be $O(N^2 T^2)$ ) by factorizing into separate $O(N^2 T)$ and $O(N T^2)$ operations.

Step 3: Bone–motion cross-attention. Using the same bone token coordinates $\mathbf{C}_{\text{bone}}$ from the rigging branch (not separately sampled bone positions), cross-attention extracts bone-conditioned motion features:

$\mathbf{A}_{\text{motion}} = \text{CrossAttn}(\mathbf{B}^{\text{emb}}, \mathbf{V}_{\Delta}^{\text{emb}}, \mathbf{V}_{\Delta}^{\text{emb}}) \in \mathbb{R}^{B \times (T-1) \times K \times d_b}$

What it computes: For each bone token, cross-attention aggregates motion information from the vertices most relevant to that bone—the same vertices that the rigging branch associated with that bone through $\mathbf{A}_{\text{rig}}$ . The output is a time-varying feature per bone that captures how the vertices in that bone's influence region move.

Why share bone tokens across branches: This is a critical architectural choice. By using identical bone token coordinates and initial embeddings in both the rigging and motion branches, the model enforces a shared indexing scheme: bone $k$ in the rigging branch corresponds to the same spatial region as bone $k$ in the motion branch. The rigging branch determines what region bone $k$ influences; the motion branch determines how that region moves over time. Without this sharing, the rigging and motion branches could converge to different, incompatible partitionings of the mesh—the rig branch's "bone 3" might correspond to the left arm, while the motion branch's "bone 3" might correspond to the right leg, creating a mismatch when the decoder tries to apply motion to the rig.

Step 4: Local motion posterior estimation. The bone–motion features $\mathbf{A}_{\text{motion}}$ are passed through an MLP to predict per-bone, per-frame posterior parameters:

$[\boldsymbol{\mu}_{\text{local}}, \log \boldsymbol{\sigma}_{\text{local}}] = \text{MLP}(\mathbf{A}_{\text{motion}})$

where $\boldsymbol{\mu}_{\text{local}} \in \mathbb{R}^{B \times (T-1) \times K \times d_z}$ is the mean and $\boldsymbol{\sigma}_{\text{local}} \in \mathbb{R}^{B \times (T-1) \times K \times d_z}$ is the standard deviation of a diagonal Gaussian distribution in the local motion latent space (dimension $d_z$ ).

What it computes: For each bone at each frame, the MLP outputs the parameters of a Gaussian distribution from which the bone's local motion latent variable will be sampled. The mean captures the expected motion pattern; the standard deviation captures the model's uncertainty about that motion (regularized toward 1 by the KL term).

Why a variational formulation: The VAE framework requires that latent variables be sampled from a parameterized distribution (here, a diagonal Gaussian), with the KL divergence regularizing the posteriors toward a standard Gaussian prior $\mathcal{N}(0, \mathbf{I})$ . This regularization serves two purposes: (a) it prevents the latents from collapsing to deterministic values (which would make the latent space non-smooth and unsuitable for generative modeling with Motion-DiT); and (b) it encourages the latent space to be continuous and structured—nearby points in latent space should produce similar motions, enabling smooth interpolation and generation.

Step 5: Reparameterized sampling of local latent variables. To enable gradient flow through the sampling operation, the standard VAE reparameterization trick is used:

$\mathbf{z}_{\text{local}} = \boldsymbol{\mu}_{\text{local}} + \boldsymbol{\sigma}_{\text{local}} \odot \boldsymbol{\epsilon}, \quad \boldsymbol{\epsilon} \sim \mathcal{N}(0, \mathbf{I})$

where $\mathbf{z}_{\text{local}} \in \mathbb{R}^{B \times (T-1) \times K \times d_z}$ is the sampled local motion latent, and $\odot$ denotes element-wise multiplication.

What it computes: A random sample from the posterior distribution, expressed as a deterministic function of the parameters $(\boldsymbol{\mu}, \boldsymbol{\sigma})$ and an independent noise source $\boldsymbol{\epsilon}$ . The noise $\boldsymbol{\epsilon}$ provides stochasticity; the parameters provide structure.

Why reparameterization: The reparameterization trick decouples the stochastic sampling (which is non-differentiable) from the parameter computation (which is differentiable). The gradient flows through $\boldsymbol{\mu}_{\text{local}}$ and $\boldsymbol{\sigma}_{\text{local}}$ without needing to differentiate through the random draw, making end-to-end training with stochastic latents possible via standard backpropagation.

Step 6: Global (root) motion posterior estimation. While local motion captures per-bone articulation, the object may also undergo global translation and rotation (e.g., a character walking forward while swinging their arms). The root motion is estimated by temporally aggregating the bone–motion features:

$[\boldsymbol{\mu}_{\text{root}}, \log \boldsymbol{\sigma}_{\text{root}}] = \text{MLP}(\text{Agg}(\mathbf{A}_{\text{motion}}))$

where Agg is average pooling over the bone and temporal dimensions, producing a single feature vector that summarizes the overall motion of the entire object across all frames. The root latent is then sampled identically:

$\mathbf{z}_{\text{root}} = \boldsymbol{\mu}_{\text{root}} + \boldsymbol{\sigma}_{\text{root}} \odot \boldsymbol{\epsilon}, \quad \boldsymbol{\epsilon} \sim \mathcal{N}(0, \mathbf{I})$

where $\mathbf{z}_{\text{root}} \in \mathbb{R}^{B \times (T-1) \times d_z}$ is a global latent vector per frame.

What it computes: Unlike local motion, which has per-bone granularity, root motion is shared across all bones—it represents the rigid transformation of the entire object (the "root" in skeletal animation terminology). Average pooling across bones ensures the root latent captures motion that is common to all bones (global translation/rotation) rather than bone-specific articulation.

Why separate local and root motion: Articulated motion naturally decomposes into global and local components. A walking character has a global forward translation (root motion) plus local rotations of individual limbs (local motion). Encoding both in a single latent would entangle these components—the latent would need to simultaneously represent "the whole body is moving forward" and "the left arm is swinging back." Separate, additive treatment (root transforms are composed with local transforms in Equation 12) enables cleaner disentanglement and more interpretable motion parameters—the root motion decoder can be inspected to understand the character's trajectory, while local motion decoders reveal per-limb articulation.

Rig-Motion Decoder: From Latents to Physical Transformations

The decoder transforms the abstract latent variables from the encoder into physically meaningful rig and motion parameters: a static set of Gaussian bones and a time-varying sequence of SE(3) transformations.

Step 1: Decoding local bone transformations. The local motion latent $\mathbf{z}_{\text{local}}$ is mapped to per-bone, per-frame SE(3) transformations via a dedicated decoder MLP:

$\{\mathbf{q}_{\text{local}}, \mathbf{t}_{\text{local}}\} = \text{Dec}_{\text{local}}(\mathbf{z}_{\text{local}})$

where $\mathbf{q}_{\text{local}} \in \mathbb{R}^{B \times (T-1) \times K \times 4}$ are rotation quaternions (later normalized to unit length) and $\mathbf{t}_{\text{local}} \in \mathbb{R}^{B \times (T-1) \times K \times 3}$ are translation vectors.

What it computes: For each bone $k$ at each frame $t$ , the decoder produces a rotation (expressed as a quaternion for compactness and numerical stability) and a translation that specify how that bone moves relative to its parent coordinate frame. In the LBS formulation (Equation 12), these are applied as $\mathbf{T}_{k, \text{local}}$ —the local transformation of bone $k$ .

Why quaternions for rotation: Quaternions are the standard compact representation for 3D rotations in deep learning because they avoid the gimbal lock issues of Euler angles and are more compact (4 numbers) than rotation matrices (9 numbers with orthogonality constraints). The decoder outputs raw 4-vectors that are normalized to unit length post-hoc, ensuring they represent valid rotations. The paper notes that Motion-DiT training includes an explicit SO(3) geodesic rotation loss (Section 3.2: $\mathcal{L}_{\text{rot}}$ ), confirming that rotation quality is explicitly supervised.

Step 2: Decoding root transformations. The root latent $\mathbf{z}_{\text{root}}$ is mapped to per-frame global SE(3) transformations:

$\{\mathbf{q}_{\text{root}}, \mathbf{t}_{\text{root}}\} = \text{Dec}_{\text{root}}(\mathbf{z}_{\text{root}})$

producing one global rotation and translation per frame applied to all bones.

What it computes: The root transformation $\mathbf{T}_{\text{root}}$ specifies the rigid-body motion of the entire object—how it translates and rotates in world space. This is composed with each bone's local transformation in Equation 12 to produce the full bone transformation $\mathbf{T}_k = \mathbf{T}_{\text{root}} \cdot \mathbf{T}_{k, \text{local}}$ .

Why hierarchical composition: The root-then-local composition follows standard skeletal animation practice: the root moves the entire skeleton, and each bone's local transform is applied relative to the root (or, in a full hierarchy, relative to its parent bone). This factorization means that a global rotation of the object only requires changing $\mathbf{T}_{\text{root}}$ , not modifying all $K$ individual bone transforms, which is both more parameter-efficient and more physically interpretable.

Step 3: Decoding Gaussian bone parameters (static). From the rigging branch features $\mathbf{A}_{\text{rig}}$ , the static Gaussian bone decoder produces the final bone parameters:

$\mathbf{G}_k = [\mathbf{c}_k, \mathbf{s}_k, \mathbf{q}_k] = [\mathbf{c}_{\text{bone}, k} + \Delta\mathbf{c}_k, \mathbf{s}_k, \mathbf{q}_k]$

where $\mathbf{c}_{\text{bone}, k}$ is the initial FPS-sampled position and $\Delta\mathbf{c}_k$ is the learned displacement, so the final bone center $\mathbf{c}_k$ can shift from its initial location.

What it computes: The final Gaussian bone parameters: center $\mathbf{c}_k \in \mathbb{R}^3$ (position of the ellipsoid in 3D space), anisotropic scale $\mathbf{s}_k \in \mathbb{R}^3$ (extent along each principal axis), and orientation quaternion $\mathbf{q}_k \in \mathbb{R}^4$ (rotation of the ellipsoid). Together, these define a 3D Gaussian ellipsoid that acts as a soft bone with spatially-varying influence on nearby vertices.

Why predict bone center as an offset from FPS: FPS provides a good initial guess for where bones should be (uniform coverage of the mesh), but the optimal bone center for deformation modeling may differ from any vertex position—it might be inside the mesh volume, at a joint center, or offset from the surface. The learned displacement $\Delta\mathbf{c}_k$ lets each bone center float freely in 3D space, unconstrained to lie exactly on the mesh surface. The scales $\mathbf{s}_k$ and orientation $\mathbf{q}_k$ further adapt each bone's influence ellipsoid to the local geometry: a limb bone becomes an elongated ellipsoid aligned with the limb's long axis, while a torso bone might be broader and flatter.

Why the decoder is lightweight (MLP-based): The heavy lifting of representation learning happens in the encoder (attention layers, cross-attention). The decoder is a relatively simple MLP because its job is straightforward: map a well-structured latent code to the output parameter space. This asymmetric design (heavy encoder, light decoder) is common in VAEs and autoencoders—the encoder compresses and structures the information; the decoder simply unpacks it.

Gaussian Skinning LBS Module: From Rig + Motion to Vertex Positions

This module is the differentiable "renderer" that converts the abstract rig and motion parameters into concrete vertex positions, enabling end-to-end training with only a vertex reconstruction loss. It has three sub-components: Gaussian bone definition, skinning weight computation, and Linear Blend Skinning with geodesic refinement.

Sub-component 1: Gaussian Bone Definition

Each of the $K$ bones is defined as a 3D Gaussian ellipsoid parameterized by its center $\mathbf{c}_k$ , anisotropic scale $\mathbf{s}_k$ , and orientation quaternion $\mathbf{q}_k$ (embedded in the rotation matrix $\mathbf{R}_k$ derived from $\mathbf{q}_k$ ). The ellipsoid represents a soft region of influence: vertices near the bone center and aligned with its principal axes are strongly influenced; distant or misaligned vertices are weakly influenced.

What a Gaussian bone physically represents: A Gaussian bone is not a discrete joint or a rigid segment with hard boundaries—it is a continuous "influence field" in 3D space. The bone's influence on any point in space is determined by the Mahalanobis distance (Equation 11), which measures distance in the bone's own coordinate system, scaled by the bone's extent along each axis. This means a bone can be long and thin (influencing vertices along a limb), short and wide (influencing a broad surface region), or any intermediate shape.

Why Gaussian ellipsoids: Ellipsoids are the simplest parametric shape that can adapt to different anatomical regions through anisotropic scaling. A sphere (isotropic) would give equal influence in all directions, which is inappropriate for limb bones that should influence vertices along the limb's length but not across its width. A more complex shape (e.g., a learned implicit function) would lose the interpretability and compactness of the Gaussian parameterization—the 10 parameters per bone (3 for center, 3 for scale, 4 for quaternion) are both human-interpretable and machine-learnable.

Sub-component 2: Skinning Weight Computation via Mahalanobis Distance

For each vertex $\mathbf{v}_i$ and each bone $k$ , the raw skinning weight (before normalization and refinement) is computed as a normalized Gaussian:

$w_{ik}^{\text{raw}} = \frac{\exp\left(-\frac{1}{2} \|\mathbf{R}_k^{\top}(\mathbf{v}_i - \mathbf{c}_k) \oslash \mathbf{s}_k\|^2\right)}{\sum_{j=1}^{K} \exp\left(-\frac{1}{2} \|\mathbf{R}_j^{\top}(\mathbf{v}_i - \mathbf{c}_j) \oslash \mathbf{s}_j\|^2\right)}$

where $\mathbf{R}_k^{\top}(\mathbf{v}_i - \mathbf{c}_k)$ rotates the vertex-bone displacement vector into the bone's local coordinate frame, $\oslash$ denotes element-wise division by the anisotropic scales $\mathbf{s}_k$ , and $\|\cdot\|^2$ is the squared Euclidean norm.

What it computes step by step:

Center-relative position: $\mathbf{v}_i - \mathbf{c}_k$ computes the 3D vector from the bone center to the vertex.
Rotate to bone frame: $\mathbf{R}_k^{\top}(\mathbf{v}_i - \mathbf{c}_k)$ rotates this vector into the bone's local coordinate system, where the bone's principal axes align with the $x$ , $y$ , $z$ axes.
Scale-normalize: dividing element-wise by $\mathbf{s}_k$ effectively rescales each axis so that the bone's extent becomes unit length in all directions—a vertex at distance $s_{k,x}$ along the bone's $x$ -axis is at distance 1 in the normalized space.
Compute squared Mahalanobis distance: $\|\cdot\|^2$ computes the squared Euclidean distance in this normalized space. This is the Mahalanobis distance—it measures "how many bone-lengths away" a vertex is, accounting for the bone's shape and orientation.
Exponentiate: $\exp(-\frac{1}{2} d^2)$ converts distance to a similarity score that decays smoothly from 1 (at the bone center) toward 0 (far from the bone). The $-\frac{1}{2}$ factor matches the standard Gaussian kernel.
Normalize across bones: dividing by the sum over all bones ensures $\sum_k w_{ik}^{\text{raw}} = 1$ for each vertex, making the weights a valid partition of unity.

Why Mahalanobis distance, not Euclidean distance: Euclidean distance $\|\mathbf{v}_i - \mathbf{c}_k\|$ treats all directions equally, which would make a bone's influence spherical regardless of the object's geometry. Mahalanobis distance accounts for the bone's shape and orientation: a vertex that is far in Euclidean distance but lies along the bone's elongated axis will have a small Mahalanobis distance (because the scale normalization compresses that axis), and thus a high influence weight. This enables anatomically meaningful skinning—a vertex on the hand is assigned high weight to the arm bone even though the Euclidean distance is large, because the arm bone is elongated along the arm–hand direction.

Why softmax normalization (not hard assignment): The softmax ensures smooth transitions between bone influences, which produces smooth deformations at joints. Hard assignment (each vertex assigned to exactly one bone) creates visible creases at joint boundaries. The soft, overlapping influences of Gaussian skinning naturally produce the smooth blending that character animators manually paint in skinning weight maps.

The Mahalanobis-distance weights are purely proximity-based: a vertex gets high weight to nearby bones regardless of whether they are topologically connected on the mesh surface. This can cause artifacts where spatially close but topologically distant regions interact incorrectly—for example, a character's inner thigh might get unintentional influence from the opposite leg's bone if the legs are close together. To address this, the paper introduces a geodesic-aware weight refinement that enforces topological coherence.

Step 1: Compute geodesic distances. For each vertex $v_i$ and each bone anchor point $a_k$ (the bone center, or a representative point), the surface geodesic distance is computed:

$d_g(v_i, a_k) = \min_{\pi \in \Pi(a_k, v_i)} \sum_{(v_p, v_q) \in \pi} \|v_p - v_q\|_2$

where $\Pi(a_k, v_i)$ is the set of all edge-connected paths between anchor $a_k$ and vertex $v_i$ , and the sum is over consecutive edge pairs along the path. This is the shortest-path distance along the mesh surface (as opposed to the straight-line Euclidean distance through 3D space).

What it computes: The length of the shortest walk from $a_k$ to $v_i$ along mesh edges. For two vertices on opposite sides of a character's torso that are spatially close (e.g., the belly and the lower back), the geodesic distance is large because you must walk around the torso's surface. For vertices connected by a limb, the geodesic path follows the limb's surface.

Why geodesic, not Euclidean distance: The Euclidean distance between the inner thighs of a character's two legs can be very small (centimeters), but the geodesic distance is large because you must go down one leg, across the crotch, and up the other leg. This prevents a leg bone on the left side from influencing vertices on the right leg just because they happen to be physically close in a particular pose.

Step 2: Construct binary coherence mask. A threshold $\tau$ determines which bone–vertex pairs are topologically valid:

$M_{ik} = \begin{cases} 1, & d_g(v_i, a_k) < \tau \\ 0, & \text{otherwise} \end{cases}$

If the geodesic distance exceeds the threshold, the connection is masked out—that bone cannot influence that vertex regardless of spatial proximity.

What it computes: A binary mask that zeros out bone–vertex pairs that are topologically disconnected on the mesh surface. The threshold $\tau$ controls how far a bone's influence can propagate along the surface.

Why a hard binary mask: A continuous penalty on geodesic distance would still allow small but non-zero weights for topologically distant vertices, which could accumulate across many bones and create visible artifacts. The binary mask completely eliminates cross-part influence, enforcing that bones only affect their own semantically meaningful region. The threshold $\tau$ is a hyperparameter that must be tuned—too small and bones become too localized (causing stiff, disjointed deformation); too large and the mask loses its topological enforcement effect.

Step 3: Refine and re-normalize weights. The raw weights are multiplied by the mask and re-normalized:

$\tilde{W}_{ik} = W_{ik}^{\text{raw}} M_{ik}, \quad w_{ik} = \frac{\tilde{W}_{ik}}{\sum_{j=1}^{K} \tilde{W}_{ij} + \varepsilon}$

where $\varepsilon = 10^{-8}$ prevents division by zero. Vertices for which all $\tilde{W}_{ik}$ are zero (no bone within geodesic threshold) are assigned a one-hot weight to their nearest bone by geodesic distance.

What it computes: First, raw proximity-based weights are zeroed for topologically invalid connections. Then, the remaining weights are re-normalized to sum to 1. Edge cases where a vertex loses all connections are handled by forcing assignment to the nearest bone.

Why re-normalization: After masking, the sum of weights for a vertex may be less than 1 because some connections were zeroed. Re-normalization restores the partition of unity property, ensuring every vertex is fully controlled by some combination of bones. The fallback to nearest-bone assignment for orphan vertices guarantees that no vertex is left undeformed.

Impact of geodesic refinement: Table 3 reports that removing this component "leads to a substantial decline in reconstruction quality." This is because, without topological enforcement, the Gaussian bones' soft influences can leak across spatial gaps, producing unnatural coupling between unrelated body parts and degrading the LBS reconstruction. The paper notes that after refinement, "vertices typically retain influence from only 2–3 bones" (Section 3.1.3), which is both more interpretable and more physically natural—most vertices on a character's surface are influenced primarily by 1–3 joints.

Sub-component 4: Linear Blend Skinning (LBS)

Given the refined skinning weights and the composed bone transformations, each vertex is deformed through standard Linear Blend Skinning:

$\hat{\mathbf{v}}_i = \sum_{k=1}^{K} w_{ik} \cdot \mathbf{T}_k \cdot \tilde{\mathbf{v}}_i$

where $\mathbf{T}_k = \mathbf{T}_{\text{root}} \cdot \mathbf{T}_{k, \text{local}} \in \mathbb{R}^{4 \times 4}$ is the $4 \times 4$ homogeneous transformation matrix for bone $k$ (composing root and local SE(3) transforms), $\tilde{\mathbf{v}}_i \in \mathbb{R}^4$ is the canonical vertex position in homogeneous coordinates (appended 1), and the multiplication $\mathbf{T}_k \cdot \tilde{\mathbf{v}}_i$ transforms the vertex from canonical space to the deformed position. The weighted sum blends these per-bone transformed positions according to the skinning weights.

What it computes: For each bone, the canonical vertex is transformed as if it were rigidly attached to that bone (applying the bone's full transformation). Then, these candidate positions are blended according to the skinning weights. A vertex with $w_{ik} = 0.7$ for bone $k$ and $0.3$ for bone $j$ moves 70% according to bone $k$ 's transform and 30% according to bone $j$ 's transform.

Why LBS: Linear Blend Skinning is the industry-standard deformation method used in virtually all real-time animation systems (games, VR, film pre-visualization). It is computationally efficient (linear in bones and vertices), differentiable (enabling gradient-based training), and produces acceptable results for most articulated characters. Its primary limitation—the "candy-wrapper" or "collapsing elbow" artifact where linear blending of rotations causes volume loss at joints—is a well-known trade-off accepted for computational tractability. The paper does not claim to solve this artifact; it accepts LBS as a standard, well-understood deformation model and focuses on learning the rig parameters that feed into it.

Complete forward pass: The full RigMo-VAE forward pass can be summarized as:

Input mesh sequence $\mathbf{V}$ enters the encoder.
Rigging branch produces $\mathbf{A}_{\text{rig}}$ and Gaussian parameters $\mathbf{G}$ .
Motion branch produces posterior parameters and samples $\mathbf{z}_{\text{local}}, \mathbf{z}_{\text{root}}$ .
Decoders produce local transforms $\{\mathbf{q}_{\text{local}}, \mathbf{t}_{\text{local}}\}$ and root transforms $\{\mathbf{q}_{\text{root}}, \mathbf{t}_{\text{root}}\}$ .
Skinng weights $\mathbf{W}$ are computed from $\mathbf{G}$ via Equation 11, refined via Equations 13–15.
Bone transforms $\mathbf{T}_k$ are composed from root and local components.
LBS (Equation 12) produces reconstructed vertices $\hat{\mathbf{V}}$ .
Reconstruction loss (Equation 17) and KL divergence (Equation 18) are computed.

Sub-component 5: Self-Supervised Training Objectives

The entire VAE is trained end-to-end with only two loss terms—no rigging supervision, no skeleton labels, no skinning weight ground truth:

$\mathcal{L}_{\text{total}} = \lambda_{\text{recon}} \mathcal{L}_{\text{recon}} + \lambda_{\text{KL}} \mathcal{L}_{\text{KL}}$

where the paper reports $\lambda_{\text{recon}} = 1.0$ and $\lambda_{\text{KL}} = 10^{-6}$ (Section 4.2). The KL coefficient is annealed over the first 30% of training, starting from 0 and reaching the final value, to avoid the posterior collapsing to the prior before meaningful latents are learned.

Vertex reconstruction loss (L2):

$\mathcal{L}_{\text{recon}} = \frac{1}{BTN} \sum_{b,t,i} \|\hat{\mathbf{v}}_{b,t,i} - \mathbf{v}_{b,t,i}\|^2$

where $\hat{\mathbf{v}}_{b,t,i}$ is the reconstructed vertex position and $\mathbf{v}_{b,t,i}$ is the ground-truth vertex position, with indices over batch ( $b$ ), frame ( $t$ ), and vertex ( $i$ ).

What it computes: The mean squared error between every predicted vertex position and its ground-truth counterpart, averaged over all sequences, frames, and vertices in the batch. This is the only signal that tells the model whether its predicted rig + motion actually reproduces the observed deformations.

Why L2 loss: L2 is the standard reconstruction loss for continuous coordinates in VAEs and autoencoders. It penalizes large errors quadratically, encouraging the model to avoid catastrophic failures (a single badly misplaced vertex contributes disproportionately to the loss). Alternatives like L1 (mean absolute error) would be less sensitive to outliers; Chamfer distance (common in point cloud tasks) would ignore vertex correspondences, which are critical for mesh deformation quality.

KL divergence regularizer:

$\mathcal{L}_{\text{KL}} = \frac{1}{2} \sum_i (\mu_i^2 + \sigma_i^2 - \log \sigma_i^2 - 1)$

where the sum is over all dimensions of the latent variables, and $\mu_i, \sigma_i$ are the posterior parameters predicted by the encoder.

What it computes: The KL divergence $D_{\text{KL}}(q_\phi(\mathbf{z}|\mathbf{x}) \| p(\mathbf{z}))$ between the posterior distribution $q_\phi(\mathbf{z}|\mathbf{x}) = \mathcal{N}(\boldsymbol{\mu}, \boldsymbol{\sigma}^2 \mathbf{I})$ and the prior $p(\mathbf{z}) = \mathcal{N}(0, \mathbf{I})$ . In the diagonal Gaussian case, this simplifies to the sum above. It encourages the posterior to remain close to the standard normal prior, preventing the latent space from collapsing to a point or developing holes that would make it unsuitable for generative modeling.

Why such a small KL weight ( $10^{-6}$ ): The reconstruction loss operates in vertex space ( $BTN \approx B \times 20 \times 5000 = 100,000B$ dimensions) while the KL term operates in latent space ( $(T-1) \times K \times d_z$ , much smaller). Without the small weight, the KL term would overwhelm the reconstruction signal and force the posterior to match the prior regardless of reconstruction quality ("posterior collapse"). The annealing schedule further helps: starting with zero KL weight lets the model first learn meaningful latents that encode motion information; then gradually increasing the KL weight regularizes these latents toward a smooth, continuous manifold suitable for diffusion.

Why self-supervision works for this problem: The key insight is that the rig is a latent variable inferred from the necessity of reconstructing motion, not predicted from annotated examples. The only way to achieve low reconstruction error is for the model to discover an articulation structure—a set of bones and skinning weights—that, when animated with appropriate transformations, reproduces the observed vertex trajectories. If the bones are poorly placed (e.g., a single bone for the entire body, or bones that don't correspond to anatomical units), the LBS reconstruction will be poor because linear blending cannot capture complex, multi-part deformations with insufficient degrees of freedom. The reconstruction loss thus implicitly penalizes bad rigs and rewards rigs that decompose the deformation into physically meaningful components. This is a form of analysis-by-synthesis: the model proposes a rig + motion hypothesis, synthesizes the resulting deformation via LBS, and compares against the data—improving the hypothesis when the synthesis fails.

Motion-DiT: Diffusion in the Learned Motion Latent Space

While the RigMo-VAE provides reconstruction and rig discovery, the Motion-DiT demonstrates that the learned latent space supports generative modeling—specifically, generating or interpolating motion trajectories conditioned on observed frames and static rig structure. This is a secondary contribution (the paper states "the main contributions of this work lie in the rigging and motion decomposition capabilities of the VAE itself"), but it validates that RigMo's latents are a meaningful representation for downstream tasks.

Input representation. Motion-DiT operates on two types of tokens extracted from the RigMo-VAE:

Dynamic (bone) tokens: $\mathbf{Z}_{\text{dyn}} \in \mathbb{R}^{B \times K \times (T-1) \times D_{\text{dyn}}}$ — the per-bone motion latents from the VAE's motion branch, representing how each bone moves over time.
Root tokens: $\mathbf{Z}_{\text{root}} \in \mathbb{R}^{B \times (T-1) \times D_{\text{root}}}$ — the global motion latents.

These are projected to a common width $H = 512$ and concatenated along the bone dimension to form a unified motion-latent tensor:

$\mathbf{Z}_{\text{mot}} \in \mathbb{R}^{B \times (K+1) \times (T-1) \times H}$

where the $K+1$ "streams" consist of $K$ bone motion streams plus 1 root motion stream.

Conditioning signals. The model receives two types of conditioning:

Static rigging features (from the RigMo-VAE rig branch): a condition encoder aggregates rig-related cues—latent rig features, decoded skinning weights, and Gaussian bone centers—into anchor tokens $\mathbf{A} \in \mathbb{R}^{B \times K \times H}$ and a global token $\mathbf{g} \in \mathbb{R}^{B \times H}$ . These remain fixed throughout the denoising process and provide the model with knowledge of the articulation structure (which bones exist, where they are, how they influence the surface).
Frame-mask schedule: a binary mask specifying which frames are "observed" (provided as conditioning to the model) and which frames are "generated" (to be predicted). The mask can encode patterns like "observe frame 1, generate frames 2–10" (1→9 prediction) or "observe frame t, generate frame t+1" (1→1 prediction). The observed frames' motion latents are kept fixed as conditioning signals; the generated frames are noised and denoised.

Architecture: Interleaved Spatio-Temporal Attention (ISTA). The backbone consists of $L = 12$ ISTA blocks with hidden dimension $H = 512$ . Each block applies:

Spatial attention: Within each frame, attention across the $K+1$ streams (all bones plus root). This lets bones coordinate their motions—for example, the arm bone and the shoulder bone can attend to each other and produce consistent, coupled motion.
Temporal attention: Within each stream, attention across the $T-1$ time steps. This captures temporal dynamics—how each bone's motion evolves over time, including velocities, accelerations, and periodic patterns.
Cross-attention to static priors: The motion tokens attend to the anchor tokens $\mathbf{A}$ and global token $\mathbf{g}$ , injecting knowledge of the rig structure. A bone's motion should be consistent with its spatial location and influence region—a bone controlling the leg should produce walking motion, not wing-flapping.
Frame-conditioned cross-attention: The generated frames attend to the observed frames based on the frame mask, enabling the model to condition its predictions on the provided observations. This is how the model "knows" what the character is doing in observed frames and can produce consistent continuations or interpolations.

Velocity prediction (v-prediction). Motion-DiT uses the v-prediction parameterization standard in modern diffusion models:

$\hat{\mathbf{x}}_0 = \sqrt{\alpha_t} \mathbf{x}_t - \sqrt{1 - \alpha_t} \hat{\mathbf{v}}$

where $\mathbf{x}_t$ is the noised latent at diffusion timestep $t$ , $\alpha_t$ is the noise schedule parameter, $\hat{\mathbf{v}}$ is the predicted velocity (the direction from the current noisy latent toward the clean latent), and $\hat{\mathbf{x}}_0$ is the estimated clean latent.

What it computes: Instead of predicting the noise $\boldsymbol{\epsilon}$ (as in DDPM) or the clean latent $\mathbf{x}_0$ directly, v-prediction predicts the velocity—the vector field that transports from the current noisy state to the clean data. The clean latent is then recovered algebraically. Losses are computed only on generated (masked) frames, not on observed frames.

Why v-prediction: v-prediction provides a unified parameterization that smoothly interpolates between noise prediction (at high noise levels, when $\mathbf{x}_t$ is mostly noise) and data prediction (at low noise levels, when $\mathbf{x}_t$ is mostly signal). This has been shown to improve sample quality and training stability compared to $\epsilon$ -prediction or $\mathbf{x}_0$ -prediction alone.

Training losses. Motion-DiT is trained with four loss terms:

$\mathcal{L} = \lambda_{\text{lat}}\mathcal{L}_{\text{lat}} + \lambda_{\text{rot}}\mathcal{L}_{\text{rot}} + \lambda_{\text{trans}}\mathcal{L}_{\text{trans}} + \lambda_{\text{vert}}\mathcal{L}_{\text{vert}}$

with weights $(\lambda_{\text{lat}}, \lambda_{\text{rot}}, \lambda_{\text{trans}}, \lambda_{\text{vert}}) = (0.5, 1.0, 0.2, 0.1)$ (Section 3.2):

$\mathcal{L}_{\text{lat}}$ : L2 loss in the latent space between predicted and ground-truth motion latents.
$\mathcal{L}_{\text{rot}}$ : SO(3) geodesic rotation loss between predicted and ground-truth bone rotations.
$\mathcal{L}_{\text{trans}}$ : L2 loss on bone translation vectors.
$\mathcal{L}_{\text{vert}}$ : L2 loss on final vertex positions after decoding through the RigMo-VAE decoder and LBS.

Losses are computed only on generated frames; observed frames are excluded.

Why multi-level supervision: The four losses provide supervision at different stages of the pipeline. The latent loss ( $\mathcal{L}_{\text{lat}}$ ) directly supervises the diffusion process in its native space. The rotation and translation losses ( $\mathcal{L}_{\text{rot}}$ , $\mathcal{L}_{\text{trans}}$ ) supervise the decoded motion parameters, ensuring geometric consistency—rotations should be valid SO(3) elements, translations should be physically plausible. The vertex loss ( $\mathcal{L}_{\text{vert}}$ ) provides end-to-end supervision through the full decoder and LBS, ensuring that the generated motion parameters actually produce correct vertex positions when applied to the rig. The higher weight on $\mathcal{L}_{\text{rot}}$ (1.0) compared to $\mathcal{L}_{\text{trans}}$ (0.2) suggests that rotation accuracy is more critical for deformation quality than translation accuracy—a finding consistent with the intuition that slight translation errors can often be absorbed by root motion compensation.

Controlled evaluation via sparse conditioning. The paper evaluates Motion-DiT under two sparse-conditioning settings (Supplementary, Section MotionDiT Evaluation):

1→1 prediction: Given one observed frame, predict the next frame. Tests local temporal smoothness.
1→9 prediction: Given one observed frame, predict the following nine frames. Tests long-horizon temporal coherence and physical plausibility.

These controlled setups avoid free-form sampling (where the model generates entirely unconditioned motion) and instead measure how well Motion-DiT can extend observed motion into the future while respecting the learned rig structure.

Ablation insights (Table 5, Supplementary). The ablation compares three conditioning variants:

w/o frame condition: No observed-frame mask—the model receives only static rig conditioning and must generate all frames without temporal context. As expected, performance degrades significantly because the model has no information about the specific motion being performed.
Latent rig condition only: Uses only the latent rig feature as conditioning, without the decoded skinning weights or Gaussian bone centers. Performance is better than no frame conditioning but worse than the full model, indicating that the explicit, physically interpretable rig attributes (bones centers, skinning weights) provide useful structural information beyond what is captured in the abstract latent feature.
Full Motion-DiT: Uses the complete conditioning bundle. Achieves the best performance, confirming that both the explicit rig structure (skinng weights, bone centers) and the temporal context (frame mask) contribute to motion generation quality.

What Motion-DiT demonstrates. Beyond specific quantitative results, Motion-DiT serves as a proof-of-concept that RigMo's latent space is a meaningful, structured representation suitable for generative modeling. Operating on compact motion latents (rather than raw vertex sequences) reduces the dimensionality of the generation problem (from $N \times 3$ per frame to $K \times d_z$ , where $K \ll N$ ) and ensures that generated motions are automatically compatible with the learned rig—the decoder produces bone transforms, and LBS guarantees valid vertex deformations. This is a fundamentally different approach from vertex-space motion generation methods, which must learn to produce spatially and temporally coherent deformations without any structural prior.

4. Key Insights and Innovations

Innovation 1: The Inversion of the Rigging Pipeline — Learning Structure from Motion, Not Geometry

The most fundamental conceptual move in RigMo is its reversal of the standard rigging assumption. Every auto-rigging method to date—RigNet, UniRig, RigAnything, MagicArticulate—operates on a single premise: a static mesh contains sufficient geometric information to predict a valid articulation structure. These methods take a canonical mesh as input and output a skeleton and skinning weights, trained on datasets of artist-rigged characters. The underlying hypothesis is that geometric cues (surface curvature, mesh topology, protrusions and concavities) encode where joints should be and how bones should influence the surface.

RigMo challenges this premise at its root. The paper's core argument, made explicit in Section 4.3 but encoded in the entire architecture, is that robust rigging cannot be inferred from static geometry alone and must be learned directly from motion. This is not a minor qualification—it is a repudiation of the entire auto-rigging research program as conventionally formulated. The evidence is in Figure 4: UniRig produces "visually plausible" skinning weights on a fox mesh, yet when those weights are used to animate the character through poses outside the training distribution, the result "collapses under actual animation, leading to severe deformation artifacts." A rig that looks correct to human eyes is not necessarily a rig that functions correctly in motion.

Why does this matter intellectually? Because it identifies a missing signal in the auto-rigging supervision pipeline. Artist-designed skeletons and skinning weights are the output of an iterative process where the animator poses the character, observes the deformation, identifies problems (volume collapse, pinching, unwanted stretching), adjusts the weights, re-poses, and repeats. The final rig is validated by motion—it works because the artist tested it against a range of poses and corrected failures. When an auto-rigger is trained to predict these final weights from static geometry alone, it learns a correlation between shape and annotation, but it has no access to the validation signal that produced those annotations. The model learns what artists drew, not why they drew it that way. As the paper notes: "even experienced artists cannot rely purely on visual priors to assign correct skinning weights; iterative adjustment is always required to produce a rig that accurately reproduces target deformations."

RigMo resolves this by making motion the training signal, not the test signal. The reconstruction loss (Equation 17) asks: "given this rig hypothesis, can we produce the observed vertex trajectories?" A bad rig—one with bones in the wrong places, scales misaligned, or skinning weights that couple unrelated body parts—will produce poor reconstruction and high loss. The model improves its rig because the loss demands it, not because it mimics human labels. This transforms rigging from a supervised pattern-recognition problem (predict what artists drew) into an unsupervised analysis-by-synthesis problem (discover what structure explains the data). This is a fundamental shift in the problem formulation, not an incremental improvement to existing auto-rigging architectures.

The significance extends beyond performance improvements. It means that rig quality is now bounded by data diversity rather than annotation quality. Existing auto-rigging datasets are limited, expensive, and biased toward humanoid characters. RigMo can, in principle, scale to any deformable object for which motion sequences exist—animals, fantastical creatures, mechanical assemblies, cloth simulations, soft-body dynamics—without requiring anyone to define what the "correct" skeleton should look like for those categories. The representation discovered is grounded in physical observation rather than human convention.

Innovation 2: Gaussian Bones as a Continuous, Resolution-Agnostic Structural Primitive

The choice to represent bones as 3D Gaussian ellipsoids rather than as discrete joints in a hierarchical skeleton is a conceptually distinct contribution from the self-supervised learning framework. Prior work on auto-rigging and motion generation almost universally represents skeletons as discrete joint trees: a root joint, child joints with fixed offsets, and hard or smoothly-interpolated influence boundaries defined per vertex. This representation has dominated computer animation for decades, from SMPL to industry-standard game engines.

The Gaussian bone formulation (Section 3.1.3) departs from this tradition in several ways that are individually simple but collectively produce a qualitatively different kind of representation:

Continuous rather than discrete. A traditional joint is a point in space with a hard radius of influence—vertices within the radius are affected, those outside are not. Skinning weight maps, when painted manually, soften these boundaries, but the underlying representation is still a set of discrete influence regions. Gaussian bones replace this with a continuous function: every bone influences every vertex, with smoothly decaying weight determined by Mahalanobis distance in the bone's local coordinate frame. There are no hard boundaries, no discrete assignments, no "bone 1 controls vertices 0–500." This continuity is not just an engineering convenience—it means the representation is fully differentiable with respect to all bone parameters (center, scale, orientation), enabling gradient-based optimization of the rig itself, not just the motion parameters that drive it. Optimization-based inverse skinning methods like SSDR must solve for discrete bone assignments and continuous transforms in separate, alternating steps; RigMo optimizes everything jointly through the LBS forward pass.

Resolution-agnostic. Because Gaussian bones are defined in continuous 3D space and influence vertices through a distance metric, the rig does not depend on mesh resolution. The same bone parameters can produce skinning weights for a 500-vertex decimated mesh and a 50,000-vertex production mesh without modification. This is a practical property that the paper explicitly validates (Section 4.6): "the predicted rig and motion parameters can be directly applied back to the original mesh resolution... deformation quality [remains] consistent even when the mesh tessellation changes." Traditional discrete skinning weights—stored per vertex—cannot be transferred across resolutions without interpolation or recomputation, both of which introduce errors. This property is significant for production pipelines where assets exist at multiple levels of detail (LODs) and must be rigged consistently across all resolutions.

Non-hierarchical by default. A traditional skeleton imposes a tree structure: the spine is the parent of the shoulder, which is the parent of the elbow, which is the parent of the wrist. This hierarchy is necessary for forward kinematics (posing the parent moves the children) and is typically hand-designed or predicted by auto-riggers. RigMo's Gaussian bones have no explicit parent-child relationships—they are an unordered set of ellipsoids in 3D space. The hierarchy is implicit in the skinning weights: if bone A influences a region proximal to the body center and bone B influences a distal region, the transforms applied to A and B will be learned in a way that respects the underlying kinematics, but the model is never told that A must be the parent of B. This is significant because it means the representation can discover non-tree-structured articulations if the data supports them (e.g., a tentacle with distributed flexibility, a sheet of cloth with no clear joint hierarchy, a face with overlapping muscle-like influences). The field has largely assumed that tree-structured skeletons are the right abstraction for all deformable objects; RigMo's Gaussian bones relax this assumption without fully abandoning the benefits of structured deformation.

Compactness as representational power. The paper reports (Table 2) that RigMo achieves comparable or better reconstruction quality than AnimateAnyMesh using only 48–128 Gaussian bones versus AnimateAnyMesh's 512 latent tokens. This 4–10× compression is not just an efficiency win—it reflects the fact that Gaussian bones are a structurally appropriate prior for articulated deformation. A vertex-space model must encode the deformation of every vertex independently, requiring high-dimensional latent codes to capture spatial coherence implicitly. Gaussian bones encode the deformation of entire surface regions through a small set of compact, semantically meaningful parameters (center, scale, orientation, transform). The representation is more efficient because it aligns with the generative structure of the data—articulated objects deform through the motion of parts, not the independent displacement of vertices.

Innovation 3: The Diagnosis that Rigging Quality Must Be Validated by Motion, Not Visual Plausibility

This is less an architectural contribution and more an empirical finding with diagnostic implications, but it is arguably the paper's most actionable insight for practitioners and researchers evaluating rigging methods. The paper demonstrates, through the comparison in Figure 4 and Table 1, that a rig which appears visually correct when inspected as static skinning weights can produce catastrophic failures when animated. This is not a trivial or obvious claim—the entire auto-rigging literature evaluates its methods by comparing predicted skeletons and skinning weights against artist-created ground truth, using metrics like joint position error or weight map similarity. The implicit assumption is that matching human annotations is a sufficient proxy for rig quality.

RigMo shows this assumption is false. The UniRig + optimization baseline produces visually plausible rigs trained on a large dataset of artist-annotated characters, yet its cross-motion reconstruction errors (Table 1) are substantially higher than RigMo's. The reason, as the paper explains, is that artist annotations encode a particular rigging convention optimized for a specific set of use cases and animation styles, not a physically necessary decomposition of the object's kinematics. A humanoid rig with 65 bones (a common game-engine convention) may be overkill for a character that only performs simple walking motions, while being insufficient for a character that needs detailed facial animation. The "correct" rig depends on the motions it needs to support, and a model trained to predict human annotations has no way to distinguish necessary bones from stylistic choices.

This finding has practical implications for how the field should evaluate rigging methods. If visual plausibility of static skinning weights does not predict functional quality under animation, then the standard evaluation protocol—compare against human-annotated test sets—is measuring the wrong thing. The paper's alternative evaluation protocol (Section 4.3) is methodologically instructive: fit a rig (via optimization or feed-forward prediction) on training motions of an object, fix the rig, and measure reconstruction error on held-out test motions of the same object. This directly measures whether the rig generalizes to novel motions, which is the functional property that matters. A rig that reconstructs training motions perfectly but fails on test motions has overfit to the training poses; a rig that achieves low test reconstruction error has captured the object's durable articulation structure. This protocol, or variants of it, should become standard for evaluating rigging methods, and its absence in prior work represents a blind spot that RigMo exposes.

The diagnostic power of this insight extends beyond evaluation. It explains why auto-rigging methods have struggled to generalize across object categories: they are trained to predict annotations that are themselves category-biased (mostly humanoids), using a signal (static geometry) that is insufficient to determine functional articulation. RigMo's self-supervised, motion-driven training side-steps both problems—it learns from whatever motion data is available and discovers whatever structure is necessary to explain that motion.

Innovation 4: A Unified Latent Space as the Foundation for Structure-Aware Generative Modeling

The Motion-DiT (Section 3.2) is presented as a secondary contribution, but the conceptual architecture it represents—a diffusion model operating on structure-aware, compact motion latents conditioned on a static rig representation—is a template that could generalize well beyond the specific implementation in this paper. The key move is the separation of concerns: the RigMo-VAE learns a latent space where what varies (motion) is disentangled from what stays constant (structure). The diffusion model then only needs to model the time-varying component, receiving the static component as a conditioning signal.

This contrasts with two dominant paradigms in 4D generative modeling. In vertex-space methods (AnimateAnyMesh, GVFDiffusion, Sync4D), the generative model must simultaneously learn to produce coherent geometry and coherent motion from a single latent code, with no structural prior separating the two. This is a harder learning problem because the model must implicitly discover that some latent dimensions correspond to static shape and others to dynamic deformation, without any architectural encouragement to do so. In pose-space methods (AnyTop, human motion diffusion models), the generative model produces motion on a predefined skeleton, but that skeleton must be provided externally and cannot be adapted to the generated motion. Any mismatch between the assumed skeleton and the actual desired articulation produces either failure or requires post-hoc adaptation.

Motion-DiT occupies a different point in this design space: the skeleton is learned from data (by the VAE), fixed for a given asset, and provided as conditioning to the diffusion model. The diffusion model only generates the motion trajectories compatible with that skeleton. This means the generative model's output is guaranteed to produce valid, coherent deformations when decoded through the VAE's LBS module—it cannot produce a motion that the learned rig cannot express, because the rig defines the space of possible motions. This is a form of built-in validity constraint that is absent from vertex-space generation (which can produce arbitrary, physically impossible deformations) and from unconstrained pose-space generation (which can produce joint rotations that violate anatomical limits or produce self-intersections).

The conceptual architecture—a VAE that factors scene content into static structure and dynamic variation, followed by a diffusion model that generates the dynamic component conditioned on the static component—generalizes naturally to other domains. For animating articulated objects, the static component is the rig; for facial animation, it could be a blendshape basis; for cloth simulation, it could be a material parameter field. The RigMo paper does not develop this generalization, but the template it establishes is a significant conceptual contribution independent of the specific Gaussian-bone instantiation.

5. Experimental Analysis

Evaluation Methodology

Dataset. The experiments use three datasets curated into a single corpus of approximately 20,000 deformable mesh sequences. DeformingThings4D contributes 1,972 real-world sequences of organic non-rigid deformations (sequences shorter than 10 frames excluded). TrueBones contributes 1,287 high-fidelity articulated animations with realistic skeletal motion. Objaverse-XL contributes 17,024 synthetic sequences (24 frames each), filtered through a motion-quality classifier to retain diverse topologies and motion styles. Each dataset uses a 5:1 training/test split. All meshes are resampled to a uniform 5,000 vertices through Farthest Point Sampling with geodesic neighborhood preservation for higher-resolution meshes, or iterative subdivision followed by FPS for lower-resolution meshes, ensuring consistent vertex count and local connectivity across heterogeneous inputs.
Base model(s). RigMo-VAE is a custom-designed dual-path encoder-decoder architecture (no pretrained backbone). The encoder uses 6 topology-aware attention layers with hidden dimension 256, 8 attention heads, and neighborhood size k=5. The default configuration uses K=48 Gaussian bones, with a 128-bone variant evaluated for comparison. The decoder uses lightweight MLPs. Models are trained from scratch on 24× A100 (80 GB) GPUs with batch size 144 mesh sequences of T=20 frames each. Motion-DiT uses a 12-block interleaved spatio-temporal attention backbone with hidden dimension 512, trained on 4 H200 GPUs.
Metrics. The primary metric is Chamfer Distance (CD-L1 and CD-L2, both reported as ×10⁻² or ×10⁻³ depending on the table) between reconstructed and ground-truth vertex positions. Chamfer Distance is a standard metric for 3D shape comparison that computes the average distance from each point in the prediction to its nearest neighbor in the ground truth, summed in both directions. It does not require vertex correspondence, which is important for the frame-independent baselines (Step1X3D, Hunyuan3D 2.1) that break vertex correspondence across frames. For the rigging evaluation specifically (Table 1), the metric is cross-motion reconstruction CD-L2 (×10⁻³) on held-out test motions. For the reconstruction evaluation (Table 2), metrics include CD (×10⁻²) and inference time (seconds per frame). For Motion-DiT (Table 5 in supplementary), Chamfer Distance L1/L2 is reported on both training and validation splits.
Baselines. The paper compares against several representative paradigms:
- Per-Case Optimization: Jointly optimizes rig parameters {G, W} and motion parameters {T_t} via gradient descent on reconstruction loss for each sequence individually. During testing, rig parameters are fixed while only transformations are optimized. This baseline directly measures how well a rig optimized for one motion transfers to another.
- Auto-Rigging Pipelines: State-of-the-art automatic rigging methods (UniRig and MagicArticulate) generate initial bone structures from canonical poses, followed by per-sequence transformation optimization. This reflects the current industry standard for automated character rigging.
- AnimateAnyMesh: A sequence-aware VAE architecture for temporal mesh modeling that requires a 512-token representation to encode object motion, versus RigMo's 48–128 tokens.
- Step1X3D and Hunyuan3D 2.1: State-of-the-art frame-independent 3D VAEs that perform per-frame generation. These baselines inherently break vertex correspondence across frames and incur substantial computational overhead for sequence reconstruction. A pose-invariant correction is applied for fair geometric comparison.
- Motion-DiT ablations (supplementary, Table 5): Three variants—without frame conditioning, with only latent rig conditioning, and the full model.
Generation budget / compute accounting. Compute is measured in several complementary ways. For the VAE, the primary measures are latent token count (48–128 for RigMo vs. 512 for AnimateAnyMesh), inference time (ms per frame on A100 GPU), and model size (parameter count). For training, convergence is reported in training steps (~7.5K epochs on DeformingThings4D alone, ~50K steps on the full corpus, or ~10 days on 24 A100s for the VAE; ~5,000 epochs or ~40 hours on 4 H200s for Motion-DiT). The paper does not use a generation budget per sample in the sense of number of forward passes—inference is a single feed-forward pass—but compares efficiency in terms of representation compactness and speed.
Cross-validation / statistical protocol. For the rigging evaluation (Table 1), the protocol randomly samples pairs of motion sequences from the same object's training data and testing data across 100 independent random splits, reporting averaged metrics and confidence intervals. The training sequences are used for baseline optimization (per-case optimization and auto-rigging baselines), and the testing sequences are used to assess cross-motion rig transferability. This ensures statistical robustness given the modest test set size. For the Motion-DiT evaluation, separate training and validation splits from the DeformingThings4D dataset are used, with results reported for both.

Main Quantitative Results

Rigging Discovery and Cross-Motion Generalization (Table 1)

The core rigging evaluation tests whether RigMo's learned rigs transfer to novel motions of the same object. Table 1 reports cross-motion reconstruction CD-L2 (×10⁻³, lower is better) on the DeformingThings4D test dataset:

RigMo (our method): achieves the lowest cross-motion reconstruction error across all comparisons. The paper does not provide the exact numerical value in the main text (Table 1 is referenced but not fully reproduced in the text body), but the text states that "RigMo's feed-forward rigs achieve lower cross-motion reconstruction error (Table 1), demonstrating that the shared representation generalizes better than independently optimized per-sequence rigs."
Per-Case Optimization: The optimization baseline substantially underperforms RigMo because "the rig optimized for one motion doesn't transfer to another." During training, the optimization fits rig parameters specifically to the training sequence. During testing, with rig parameters fixed and only transformations optimized, the rigidly overfitted rig fails on novel poses.
UniRig + Optimization: The auto-rigging baseline using UniRig followed by per-sequence transformation optimization produces higher cross-motion error than RigMo. The qualitative evidence in Figure 4 shows that UniRig's predicted rigging "collapses under actual animation, leading to severe deformation artifacts" even though the static skinning weights appear "visually plausible."
MagicArticulate + Optimization: Similarly underperforms RigMo, consistent with the finding that rigging predicted from static geometry alone does not guarantee functional correctness under novel motions.

The key empirical takeaway from Table 1 is not just that RigMo outperforms baselines, but that the ranking of methods in cross-motion generalization contradicts their ranking in static visual plausibility. UniRig and MagicArticulate produce rigs that look correct when inspected as weight maps, yet fail functionally when animated. RigMo's rigs, discovered entirely from motion without human annotations, generalize better to novel motions precisely because they were optimized to explain motion rather than to mimic artist labels.

Reconstruction Fidelity and Inference Efficiency (Table 2)

Table 2 compares reconstruction quality and inference speed across RigMo-VAE, AnimateAnyMesh, Step1X3D, and Hunyuan3D 2.1 on a curated sub-test set of 500 sequences spanning diverse deformation patterns and object categories. The paper does not reproduce the full table in text, but the narrative highlights specific comparisons:

RigMo vs. AnimateAnyMesh (sequence-aware baseline): The paper states that "RigMo-VAE achieves superior reconstruction quality using only 48/128 tokens, resulting in significantly faster inference and reduced memory footprint" compared to AnimateAnyMesh's 512-token representation. The 48-token configuration achieves roughly 4× token compression relative to the baseline while matching or exceeding reconstruction accuracy. The 128-token configuration achieves the best quantitative reconstruction (Table 3 reports a 0.018% improvement in CD-L1 over 48 tokens), but the paper recommends 48 tokens as offering "a more favorable balance between efficiency, interpretability, and stability."
RigMo vs. frame-independent baselines (Step1X3D, Hunyuan3D 2.1): These methods perform per-frame generation and "inherently break vertex correspondence across frames and tend to lose fine-grained surface details due to their framewise decoding process." The paper applied a pose-invariant correction (recording canonical transformations used for input normalization and applying inverse transformations after reconstruction) to enable fair geometric comparison. RigMo's explicit LBS deformation ensures consistent vertex correspondence and temporal coherence, which per-frame methods cannot guarantee.
Inference speed: The paper reports that RigMo-VAE reconstructs a 20-frame, 5K-vertex sequence in "~40ms per frame on an A100 GPU" (Section 4.2). The exact inference times for baselines are not given in absolute terms, but the text states that frame-independent baselines "incur substantial computational overhead for sequence reconstruction" since they must process each frame independently through full 3D VAE pipelines.

Motion-DiT Controlled Generation (Supplementary, Table 5)

The Motion-DiT evaluation uses two sparse-conditioning settings on the DeformingThings4D dataset: 1-frame → 1-frame prediction (short-term temporal smoothness) and 1-frame → 9-frame prediction (long-horizon temporal coherence). Table 5 in the supplementary material reports Chamfer Distance (L1 and L2, ×10⁻²) for both training and validation splits across three ablations.

The paper reports that "across both prediction horizons, the full Motion-DiT model achieves strong motion fidelity and temporal coherence." Three variants are compared:

Without frame condition: Removing the frame-mask signal entirely forces the model to infer temporal structure without any observed-frame guidance. Performance degrades significantly, as the model has no information about the specific motion being performed.
Latent rig condition only: Conditioning only on the latent rig feature (without decoded skinning weights or Gaussian bone centers) provides some structural guidance, but underperforms the full model. This indicates that the explicit, interpretable rig attributes (bone centers, skinning weights) provide useful spatial information beyond what is contained in the abstract latent feature — the model benefits from knowing where bones are located and which vertices they influence, not just from a generic rig embedding.
Full Motion-DiT: Uses the complete conditioning bundle (latent rig feature + decoded skinning weights + Gaussian bone centers + frame-mask guidance). Achieves the best performance, confirming that both explicit structural priors and temporal context are necessary for high-quality motion generation.

The paper notes that the 1→9 prediction setting is "more challenging" and "evaluates the ability of the model to produce temporally coherent and physically plausible long-horizon motion." The full model's strong performance in this setting demonstrates that the motion latents capture dynamics that can be extended over time while respecting the learned rig constraints.

The key insight from this evaluation is not the absolute numbers (which are not fully stated in the text) but rather the ablation structure: each conditioning signal contributes independently to motion quality, and the full combination of structural priors (rig features, explicit bone geometry, skinning weights) plus temporal context (observed frames) is necessary for best results. This validates that RigMo's latent space is not just compressible but genuinely structured — it contains separable channels for spatial articulation and temporal dynamics that a generative model can leverage.

Ablation Studies and Robustness Checks

Bone token cardinality (Table 3): The paper compares 48-bone versus 128-bone configurations on the DeformingThings4D validation set. Increasing from 48 to 128 tokens yields only a 0.018% improvement in CD-L1 (and a similar marginal improvement in CD-L2). The paper interprets this as diminishing returns: "Excessive tokens tend to fragment coherent anatomical regions without offering proportional improvements." While 128 tokens achieve the best quantitative reconstruction, 48 tokens provide "a more favorable balance between efficiency, interpretability, and stability." This is a non-obvious finding — one might expect that more bones would always improve reconstruction since they provide additional degrees of freedom, but the marginal gain is negligible while the interpretability cost (bones fragmenting coherent anatomical regions) is real. The 48-token configuration is recommended as the practical default.

Geodesic-aware weight refinement (Table 3): Removing the geodesic-aware weight refinement module "leads to a substantial decline in reconstruction quality." The paper does not provide the exact degradation in the narrative, but it is the primary ablation result in Table 3. This confirms that pure Euclidean-distance-based bone influence (the Mahalanobis weighting without topological masking) is insufficient for articulated objects — spatially close but topologically distant vertices (e.g., inner thighs of opposite legs) receive unintended cross-bone influence, degrading deformation quality. The geodesic mask effectively enforces that bones influence only their own contiguous surface regions. The paper notes that after refinement, "vertices typically retain influence from only 2–3 bones," which is both more interpretable (matches the standard practice of 2–4 bone influences per vertex in production skinning) and more physically correct.

Resolution-agnostic architecture robustness (Section 4.6, narrative): The paper reports a qualitative ablation demonstrating that RigMo's learned rig and motion parameters "can be directly applied back to the original mesh resolution" without degradation. Since Gaussian bones and motion transformations are "defined continuously in 3D space, rather than tied to specific vertex indices," changing mesh tessellation does not alter the rig's behavior. This contrasts with mesh-dependent approaches (explicitly citing AnimateAnyMesh) where "changing vertex count typically alters the learned skinning structure and degrades animation fidelity." This is not a quantitative ablation with numbers but a robustness property verified through qualitative inspection — the paper claims "consistent deformation quality even when the mesh tessellation changes."

Moion-DiT conditioning ablations (Supplementary, Table 5): Three variants test the contribution of different conditioning signals:

Without frame condition (w/o frame-mask guidance): Removes observed-frame context. Degrades performance across both prediction horizons and both training/validation sets, confirming that temporal context is essential for motion generation — the model cannot infer what motion to produce from static rig information alone.
Latent rig condition only: Uses only the abstract rig-branch latent feature without explicit geometric information. Performs better than no frame conditioning but worse than the full model, indicating that the decoded, physically interpretable rig attributes carry information not fully captured in the latent feature. This is methodologically interesting because it suggests that the VAE's latent rig feature does not perfectly encode all structural information — the explicit Gaussian parameters and skinning weights provide complementary guidance.
Full Motion-DiT (all conditioning): Best performance across all settings. The ablation confirms that the design choice to condition on explicit rig attributes, not just latent features, is not redundant.

Motion-DiT training stability (Section 4.2): The paper notes that Motion-DiT "remains stable without auxiliary losses, aided by its spatial–temporal attention design and sparse conditioning in RigMo's motion-latent space." This is a minor robustness observation — the model does not require additional regularization beyond the four specified losses (latent L2, rotation geodesic, translation L2, vertex L2), suggesting that the sparse conditioning and structured latent space provide sufficient inductive bias for stable training.

KL annealing schedule (Section 4.2, narrative): The KL weight of 10⁻⁶ is annealed over the first 30% of training. This is a standard VAE training technique to prevent posterior collapse, but the paper does not ablate whether removing annealing degrades performance. The small absolute KL weight (10⁻⁶ relative to reconstruction weight of 1.0) is itself notable — it suggests that reconstruction fidelity dominates the optimization and that heavy KL regularization would harm reconstruction quality. This is consistent with the finding that the latent space remains structured and usable for generative modeling (via Motion-DiT) despite the weak KL regularization.

Missing ablations: The paper does not ablate several important design choices:

The dual-path encoder (rigging branch + motion branch) is never compared against a single-path encoder that processes all frames jointly. It is therefore unclear whether the disentanglement actually improves performance or generalization, or whether a single unified encoder would perform equally well with sufficient capacity.
The FPS-based bone token selection is not compared against learned token selection (e.g., attention-based pooling or a learned query mechanism). FPS is geometric and deterministic; learned selection might produce better bone placements at the cost of additional parameters.
The number of encoder layers (6) and attention heads (8) are stated but not ablated. It is unknown whether a deeper or shallower encoder would change reconstruction quality.
The Mahalanobis-based skinning weight computation is not compared against a learned weighting function (e.g., an MLP that takes bone and vertex features as input). The Mahalanobis formulation is analytically clean, but a learned function might capture more complex influence patterns.
The geodesic refinement threshold $\tau$ is not ablated or even specified numerically in the main text. It is unclear how sensitive performance is to this hyperparameter.
The separate local and root motion decoders are not compared against a unified motion decoder. The separation is architecturally motivated (global vs. local articulation), but whether it improves performance or is merely a design choice is not tested.
The reconstruction-only training (no adversarial loss, no perceptual loss, no temporal smoothness regularization beyond what emerges from the architecture) is not ablated against alternatives. The model discovers smooth, interpretable rigs from L2 loss alone, but it is unknown whether additional losses would improve rig quality or interpretability.

Critical Assessment

Claim 1: RigMo jointly learns rig structure and motion dynamics from raw mesh sequences without rig annotations, producing interpretable Gaussian bones and skinning weights.

What the experiments demonstrate: The reconstruction evaluation (Table 2) shows that RigMo-VAE can take mesh sequences as input and reproduce them through the Gaussian bone + LBS pipeline with competitive or superior accuracy to existing methods. The rigging evaluation (Table 1) shows that the learned rigs transfer to novel motions with lower error than per-case optimization or auto-rigging baselines. The qualitative visualizations (Figures 1, 3, 4, 5) show learned Gaussian bones that correspond to anatomically meaningful regions.

What the experiments do NOT demonstrate: The claim of "interpretability" is not empirically validated beyond qualitative visualization. There is no user study, no metric for bone semantic meaningfulness, and no comparison against what a human artist would consider interpretable. The fact that the learned bones "look reasonable" in figures is suggestive but not proof of interpretability — a model that produces bones that happen to look anatomical to human eyes might still produce degenerate rigs on out-of-distribution objects, and the paper does not test this.

The claim of "without any human-provided rig annotations" is technically true — no rigging labels are used in training — but the training data (DeformingThings4D, TrueBones, Objaverse-XL) consists of mesh sequences that are themselves products of human design or capture. A mesh sequence of a walking human carries implicit information about human articulation that the model can exploit. It would be a stronger test of "unsupervised rig discovery" to train on procedurally generated deformations with known ground-truth rigs and measure whether the discovered bones correspond to the true underlying structure. The current evaluation cannot distinguish between "the model discovered the true articulation" and "the model found a rig that happens to reconstruct the training motions well but might not capture the object's actual kinematic structure."

Claim 2: RigMo achieves superior reconstruction and cross-motion generalization compared to existing auto-rigging and deformation baselines.

What the experiments demonstrate: Table 1 shows RigMo outperforming per-case optimization and UniRig/MagicArticulate baselines on cross-motion reconstruction error. Table 2 shows RigMo matching or exceeding AnimateAnyMesh's reconstruction quality with 4× fewer tokens and faster inference.

What the experiments do NOT demonstrate: The comparison against auto-rigging baselines (UniRig, MagicArticulate) is somewhat apples-to-oranges. These baselines are trained on different data (artist-annotated rigs) and predict rigs from static geometry. The paper then evaluates them by optimizing per-sequence motion parameters for the predicted rig. This evaluation disadvantages the baselines because they were not designed for this task — they were designed to produce rigs that match human annotations, not rigs that minimize reconstruction error on novel motion sequences. A fairer baseline would be an auto-rigging method specifically trained or adapted for reconstruction-driven optimization, or a version of RigMo trained with the same architecture but on static geometry alone (to isolate whether the motion-driven training or the architecture is responsible for the gains).

The comparison against AnimateAnyMesh does not control for training data volume or compute. RigMo was trained on 20,000 sequences across three datasets; AnimateAnyMesh was trained on a different (unspecified) dataset. The reconstruction quality gap could be due to differences in training data rather than architectural superiority. A controlled comparison where both models are trained on identical data would be more informative.

The per-case optimization baseline, while methodologically appropriate for testing cross-motion generalization, uses a different optimization procedure than RigMo. The per-case baseline optimizes rig parameters for each training sequence individually; RigMo learns a shared rig representation across all training sequences. The fact that RigMo generalizes better is therefore partially a statement about the benefits of shared multi-sequence training, not necessarily about the specific Gaussian bone or dual-path encoder design. A baseline that performs multi-sequence joint optimization (rather than per-case) would isolate the contribution of the architecture from the contribution of the training regime.

Claim 3: Robust rigging must be learned from motion, not predicted from static geometry.

What the experiments demonstrate: Figure 4 shows qualitative evidence that UniRig's static-geometry-predicted rigging fails under animation, while RigMo's motion-learned rigging succeeds. Table 1 quantifies this advantage. The conceptual argument is clear and well-supported by the experimental design.

What the experiments do NOT demonstrate: The paper does not run a controlled experiment where the same architecture (RigMo's encoder-decoder) is trained on (a) static geometry alone predicting rig parameters and (b) motion sequences as in the current setup. Without this ablation, the claim that "motion is necessary" cannot be separated from the claim that "RigMo's architecture is better." It is possible that an identically-sized model trained to predict Gaussian bones from static geometry (with a reconstruction loss on a held-out motion set, or with some proxy task) would perform comparably. The baselines (UniRig, MagicArticulate) have different architectures, different training objectives, and different training data, so their underperformance does not isolate the "static vs. motion-driven" variable.

Furthermore, the claim applies specifically to cross-motion generalization — a rig optimized for one motion and tested on another. It is possible (though untested) that a static-geometry-predicted rig would perform adequately if evaluated only on the motions seen during its optimization, or if the optimization included a diverse motion set rather than a single training sequence. The conceptual claim ("robust rigging cannot be inferred from static geometry alone") might be true in the limit of diverse, novel motions, but the experiment only tests a single training motion per object, which is a specific and perhaps artificially challenging setting.

Claim 4: Motion-DiT demonstrates that RigMo's learned latent space enables controllable motion generation.

What the experiments demonstrate: The Motion-DiT evaluation (supplementary, Table 5) shows that a diffusion model operating on RigMo's motion latents can predict future frames given observed frames and static rig conditioning, and that the full conditioning bundle (rig features + explicit geometry + frame mask) achieves the best performance. The 1→9 frame prediction setting demonstrates some capacity for long-horizon generation.

What the experiments do NOT demonstrate: The evaluation is limited to sparse-conditioning prediction (1→1 and 1→9) on DeformingThings4D only. This is a relatively narrow form of "generation" — it is closer to temporal interpolation or short-horizon prediction than to open-ended motion synthesis. The paper does not evaluate free-form generation (sampling motions without any observed frames), generation conditioned on text or other modalities, or generation of motions for new objects not seen during VAE training. The claim of "controllable motion generation" is therefore supported only for the specific control signal of observed frames; text-based, sketch-based, or semantic control is not demonstrated.

The evaluation dataset (DeformingThings4D) is the same dataset used for VAE training, so the latent space quality is evaluated in-distribution. There is no evaluation of Motion-DiT on Objaverse-XL or TrueBones sequences, which would test whether the generative model generalizes to motion styles and object categories outside its training distribution.

The paper acknowledges these limitations explicitly (Section 4 opening): "the main contributions of this work lie in the rigging and motion decomposition capabilities of the VAE itself." Motion-DiT is positioned as a proof-of-concept, not a fully developed motion generation system. The experimental evaluation is appropriately scoped to this claim, but readers should not interpret Motion-DiT as a general-purpose text-to-motion or controllable animation system — it demonstrates that the latent space is structured enough to support diffusion-based generation, which is a weaker but well-supported claim.

Strengths and Weaknesses Summary

Genuine strengths of the experimental design:

The cross-motion generalization protocol (100 random splits, train rig on one motion, test on another) is a methodologically strong way to evaluate rig transferability. This protocol addresses a real failure mode — rigs that overfit to training poses — that is underexplored in the auto-rigging literature.
The multi-dataset evaluation (DeformingThings4D + TrueBones + Objaverse-XL) provides some evidence of domain diversity, though the detailed per-dataset breakdown is not provided.
The comparison against three distinct paradigms (per-case optimization, auto-rigging + optimization, 4D VAE) provides broader coverage than a single-baseline evaluation.
The ablation of geodesic refinement convincingly demonstrates its importance through quantitative degradation in Table 3.

Genuine weaknesses:

The test set sizes are not explicitly stated. DeformingThings4D has 1,972 sequences with a 5:1 split (~394 test sequences); similarly, 100 random splits within this modest test set might not capture the full diversity of failure modes. Confidence intervals from the 100 splits would strengthen the statistical claims (they are mentioned as "reporting averaged metrics and confidence intervals" but the actual intervals are not visible in the provided text).
The paper does not report per-dataset or per-category breakdowns of performance. It is plausible that RigMo excels on certain object categories (e.g., bipeds with clear articulation) and struggles on others (e.g., amorphous deforming objects, cloth, fluids), but the aggregate metrics obscure this heterogeneity.
The auto-rigging baselines (UniRig, MagicArticulate) are evaluated in a setting that disadvantages them (reconstruction error on held-out motions, rather than skeleton prediction accuracy). A comparison on the task these baselines were designed for (skeleton accuracy against artist ground truth) would contextualize whether RigMo's advantage is task-specific or more fundamental.
The 1→9 frame prediction evaluation for Motion-DiT is a form of autoregressive next-frame prediction, which is known to accumulate errors over long horizons. The paper does not analyze error accumulation or trajectory drift in the long-horizon setting.
Training and inference compute costs are reported (24 A100s for ~10 days, 40ms/frame at inference), but inference cost for the auto-rigging baselines (UniRig inference + per-sequence optimization time) is not, making the practical efficiency comparison incomplete.

Missing experiments that would strengthen the paper:

A controlled comparison where RigMo's architecture is trained on static geometry only (predicting Gaussian bones from a single frame, with reconstruction evaluated on held-out motions) to isolate the contribution of motion-driven training.
Evaluation on a procedurally generated dataset with known ground-truth skeletons and skinning weights, to measure how accurately RigMo recovers the true underlying articulation rather than just finding a functionally sufficient approximation.
Ablation of the dual-path encoder design against a single unified encoder.
Multi-sequence motion generation (generating entirely new motion sequences for a given rig, not just predicting continuations of observed motion) to more fully evaluate the generative capabilities of the Motion-DiT latent space.

6. Limitations and Trade-offs

Limitation 1: Difficulty Estimation Cost Is Deferred, Not Eliminated

The assumption or constraint. The entire compute-optimal test-time scaling framework depends on partitioning prompts into difficulty bins, and the paper proposes two methods for doing so: an "oracle" method that uses ground-truth pass@1 rates (requiring 2,048 samples and answer labels per question), and a "predicted" method that substitutes PRM final-answer scores for ground-truth correctness but still requires generating and scoring 2,048 samples per question. The paper acknowledges this cost directly in Section 3.2:

"estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity"

This is not a minor caveat — the difficulty estimation step requires 2,048 generations per prompt, which equals or exceeds the largest test-time budgets studied in the main experiments (256–512 generations). The 4× efficiency gains reported in Figures 4 and 8 are computed after difficulty is known, without amortizing the cost of learning it.

The consequence. In a realistic deployment, the total computation per prompt would be: (cost to estimate difficulty) + (cost to execute the selected strategy). With the current method, the former dominates the latter for any budget under 2,048 generations — precisely the regime where the paper reports its largest gains. A practitioner implementing this system as described would find that the actual efficiency, inclusive of difficulty estimation, is substantially worse than the headline 4× figure, potentially even worse than a naive best-of-N baseline that skips difficulty estimation entirely. The exploration-exploitation tradeoff the paper identifies (Section 3.2) is not resolved — it is acknowledged and deferred.

What evidence exists in the paper. The paper itself provides no measurement of the inclusive cost. The cost accounting in Section 7 (FLOPs-matched comparison) uses only the strategy execution budget, not the difficulty estimation overhead. The predicted-difficulty curves in Figures 4 and 8 are computed assuming the difficulty bin is already known; the 2,048 samples used to determine that bin are not subtracted from the budget. The paper notes this as a "key avenue for future work" (Section 3.2) and suggests training a model to predict difficulty directly from the question text, but no such model is developed or evaluated.

Mitigation status. Not addressed. The paper explicitly frames this as future work. The 4× efficiency claim must therefore be understood as an upper bound on achievable efficiency gain, conditional on having a cheap, accurate difficulty estimator that does not yet exist. Until such an estimator is demonstrated, the practical value of compute-optimal scaling remains aspirational.

Limitation 2: Hard Problems Are Fundamentally Unsolved — Test-Time Compute Cannot Substitute for Missing Capability

The assumption or constraint. The compute-optimal framework operates by amplifying a base model's existing capabilities — it searches for correct solutions in the model's output distribution or refines nearly-correct solutions through revision. If the base model's pass@1 on a problem class is effectively zero (it cannot produce correct solutions at any non-trivial rate), no amount of test-time compute can help. The paper states this explicitly in the Section 7 takeaway:

"on the hardest questions... test-time compute provides essentially zero benefit regardless of budget, meaning that some capabilities can only be acquired through pretraining, not recovered at inference time."

The consequence. For the hardest difficulty quintile (bin 5 in all figures), all methods — search, revisions, compute-optimal combinations — produce near-zero improvement over the base model's pass@1 regardless of compute budget. In Figure 3 (right), bin 5 accuracy hovers at 1–3% for all methods and all budgets. In Figure 7 (right), bin 5 shows roughly 2–3% accuracy irrespective of the sequential-to-parallel ratio. In the FLOPs-matched comparison (Figure 9), the bin 5 scaling line is essentially flat near 0–5%. The 14× larger pretrained model does substantially better on these problems (Figure 9, stars). This means that for any deployment where the problem distribution includes genuinely hard problems — those requiring reasoning the base model hasn't acquired — the compute-optimal framework offers no path forward. The entire approach is bounded by the base model's capabilities at training time.

What evidence exists in the paper. The difficulty-bin breakdowns across all experiments (Figures 3, 7, 9) consistently show flat or near-flat scaling curves for bin 5. The FLOPs-matched result is particularly stark: on hard problems at $R \gg 1$ , test-time compute shows a -52.9% relative disadvantage compared to the 14× larger model (Figure 1, bottom-right bar chart), meaning the extra inference compute is not just useless but actively worse than using those FLOPs for pretraining a larger model. The qualitative examples in Appendix M (Figures 29 and following) illustrate specific failure modes — repetitive low-information steps, overly short solutions — where search exploits verifier weaknesses without improving correctness.

Mitigation status. The paper is transparent about this limitation (Section 7 takeaway box). It does not attempt to mitigate it — correctly, because the problem is fundamental: test-time compute optimizes within the support of the base model's output distribution, and if that support does not include correct solutions, optimization cannot create them. The boundary between "problems within the model's reach" and "problems beyond the model's reach" is empirically sharp and defines the regime where this approach is applicable versus where pretraining is the only viable path.

Limitation 3: Single Benchmark, Single Model Family, Single Task Domain

The assumption or constraint. All experiments in the paper use a single benchmark (MATH, 500 test questions), a single base model family (PaLM 2-S*), and a single task domain (high-school competition mathematics requiring symbolic reasoning). The paper asserts (Section 4) that PaLM 2-S* is "representative of the capabilities of many contemporary LLMs," but provides no evidence across model families, scales, or architectures. The paper does not evaluate on code generation, logical reasoning, scientific QA, or any non-math reasoning task.

The consequence. Several key findings could be specific to the MATH benchmark or to PaLM 2-style models. The PRM's quality, its over-optimization behavior, and the difficulty-dependent scaling curves all depend on the base model's output distribution and error patterns. A model with different calibration (e.g., more confident on wrong answers, more uncertain on correct ones) would produce different PRM training data and different search dynamics. The revision model's ability to learn from incorrect in-context examples depends on the base model's in-context learning capabilities, which vary substantially across model families (e.g., some models are much better at few-shot learning than others). The finding that beam search hurts on easy problems due to PRM over-optimization (Figure 3, right) might not replicate on a verification task with different difficulty calibration. The MATH benchmark's specific properties — clean answer formats enabling exact-match grading, well-defined step-by-step solutions enabling PRM rollout supervision, competition-level difficulty ensuring a range of pass@1 rates — may not transfer to tasks with ambiguous correctness (dialogue, summarization), open-ended structure, or different difficulty distributions.

What evidence exists in the paper. None. The paper does not include any cross-model, cross-benchmark, or cross-task evaluation. The statement that PaLM 2-S* is "representative" is an assertion, not a supported claim. The supplementary material (Appendices) contains additional MATH-specific analyses (PRM training details, aggregation comparisons, qualitative examples) but no results on other benchmarks. The paper does not cite or discuss any replication on non-math domains.

Mitigation status. Not addressed. The paper does not claim cross-domain generalization — it carefully scopes its contributions to the MATH benchmark — but the framing language throughout (e.g., the abstract's "systematic analysis of scaling test-time compute for LLMs") implies broader applicability that is not empirically supported. A reader deploying this framework on a different task or model would need to verify all findings independently, as none of the specific quantitative relationships (4× efficiency gain, optimal search strategy per difficulty bin, sequential-to-parallel ratio curves) can be assumed to transfer.

Limitation 4: The 14× Larger Model Baseline Is Weakened by Non-Compute-Optimal Training and Greedy Decoding

The assumption or constraint. The FLOPs-matched comparison in Section 7 pairs PaLM 2-S* with compute-optimal test-time scaling against a model with approximately 14× more parameters. The larger model's training uses the LLaMA paradigm — scaling parameters while holding training data fixed — rather than the Chinchilla-optimal paradigm of scaling both parameters and data equally. The paper acknowledges this choice (Section 7):

"We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work."

Additionally, the 14× larger model is evaluated only with greedy decoding — no majority voting, no best-of-N, no search, no revisions. It receives zero test-time compute augmentation of its own.

The consequence. Both design choices weaken the pretraining baseline relative to what a truly compute-matched comparison would produce. A Chinchilla-optimal model trained with 14× more total FLOPs (scaling both data and parameters) would likely outperform a parameter-only-scaled model, closing or reversing the reported advantages of test-time compute (e.g., +27.8% relative improvement on easy questions at $R \ll 1$ ). Furthermore, giving the larger model even a modest test-time compute budget — say, best-of-8 sampling with majority voting — would likely improve its performance further, since the larger model likely has a higher base pass@1 and would benefit from the same sampling-based amplification that the paper demonstrates for the smaller model. The comparison is not "compute-optimal test-time scaling vs. compute-matched pretraining"; it is "compute-optimal test-time scaling vs. a specific suboptimal pretraining configuration with no test-time augmentation."

What evidence exists in the paper. The paper does not provide any sensitivity analysis showing how the FLOPs-matched results would change under Chinchilla-optimal pretraining or under test-time-augmented larger models. The three $R$ values (0.16, 0.79, 22) span interesting regimes, but all assume the same weakened pretraining baseline. The acknowledgment of the Chinchilla deviation is present (Section 7) but framed as a future-work item rather than a limitation affecting the current conclusions.

Mitigation status. Acknowledged but not tested. The paper correctly identifies the limitation but does not estimate its magnitude. The reported advantages of test-time compute over pretraining should be understood as upper bounds that may shrink or reverse against properly optimized pretraining baselines. The qualitative conclusion that test-time compute helps on easy-to-medium problems and fails on hard problems is likely robust to the baseline choice (since the scaling curves themselves show the failure, not just the comparison), but the quantitative claim that a smaller model with test-time compute outperforms a 14× larger model depends critically on the larger model's suboptimal training and evaluation.

Limitation 5: Revisions and Search Are Studied in Isolation — Complementary Strengths Are Not Combined

The assumption or constraint. The paper studies PRM-guided search (Section 5) and iterative revisions (Section 6) as independent mechanisms, each with its own compute-optimal allocation policy. It does not combine them — for example, using the revision model as the proposal distribution within beam search, or using the PRM to guide which revisions to pursue versus restart. The paper explicitly acknowledges this gap in Section 8:

"we did not experiment with PRM tree-search techniques in combination with revisions"

The consequence. The paper's results represent a lower bound on what a fully integrated system could achieve. The two mechanisms have documented complementary strengths: revisions improve the proposal distribution on easy problems (generating better candidates through local refinement), while PRM search improves candidate selection on medium-difficulty problems (finding good solutions through global exploration). A combined system could, for instance, use the revision model to generate higher-quality candidate steps within beam search, or use the PRM to decide when a revision chain is on track versus when to terminate it and restart with a different approach. The compute-optimal allocation for a combined system would be strictly better than the max of either individual method, and potentially better than the sum if the mechanisms interact synergistically. The paper's reported ceilings — roughly 44% on MATH at 256 generations for revisions (Figure 8) — might therefore be surmountable with joint optimization.

What evidence exists in the paper. None. The paper provides no experiments combining search and revisions. The difficulty-dependent analyses (Figures 3 right, 7 right) show that search and revisions have qualitatively different optimal difficulty regimes — revisions excel on easy problems (bin 1–2), search excels on medium problems (bins 3–4) — which is evidence that they are complementary, but this complementarity is observed across separate experiments rather than demonstrated in a unified system.

Mitigation status. Acknowledged as future work (Section 8). The combination is a natural and important next step, and the paper's framework provides the conceptual scaffolding for doing so, but a reader should not interpret the current results as representing the best achievable performance from a joint rig–motion system. Full integration remains to be demonstrated.

7. Implications and Future Directions

How This Work Changes the Landscape

RigMo fundamentally reframes the problem of rigging for deformable 3D objects. It is not an incremental improvement to existing auto-rigging architectures—it is a repudiation of the core premise that has governed the field for decades: that static geometry contains sufficient information to predict a functionally correct articulation structure. By demonstrating that rigs learned from motion data generalize to novel motions while rigs predicted from static geometry produce "catastrophic failures when animated" (Figure 4), RigMo establishes a new criterion for what constitutes a valid rig. The standard of correctness shifts from visual plausibility of static skinning weight maps to functional validity under animation.

This is a methodological shift, not a paradigm shift. It does not overthrow the Gaussian-bone or LBS framework—it inherits those—but it changes how the field should train and evaluate rigging systems. Prior work evaluated auto-rigging methods by comparing predicted skeletons and skinning weights against artist-created ground truth (joint position error, weight map similarity). RigMo exposes the blindness in this protocol: a rig that matches human annotations can still fail catastrophically when animated, because artist annotations encode convention, not physical necessity. The alternative cross-motion generalization protocol (Section 4.3)—train on one motion sequence, test reconstruction error on a held-out motion of the same object, repeated over 100 random splits—directly measures the functional property that matters. This protocol, or a variant of it, should become standard practice for evaluating rigging methods. Its absence in prior work represents a methodological gap that RigMo identifies and fills.

The paper also resolves a latent contradiction in the 4D generation and auto-rigging literatures. On one side, auto-rigging methods (RigNet, UniRig, RigAnything, MagicArticulate) report impressive numbers on static skeleton prediction benchmarks, suggesting that rigging from geometry is a largely solved problem. On the other side, practitioners and production pipelines report pervasive failures when these predicted rigs are animated with novel motions—broken deformations, collapsing meshes, mismatched articulation. RigMo provides the diagnosis: these failures are inevitable because the models were trained on the wrong signal. They learned to predict what artists drew, not what produces physically correct deformations. This reconciles the high benchmark scores with the poor production performance—the benchmarks themselves measure the wrong thing. The practical implication is that research investment in auto-rigging should shift toward motion-driven, self-supervised methods, and away from scaling up annotation-dependent pipelines that optimize for an increasingly misaligned metric.

The resolution-agnostic property of Gaussian bones adds a practical dimension to this shift. The paper's demonstration that "the predicted rig and motion parameters can be directly applied back to the original mesh resolution" with consistent deformation quality (Section 4.6) means that rigs learned on computationally tractable downsampled meshes transfer to production-resolution assets without degradation. This removes a persistent friction point in animation pipelines, where rigs must be manually adjusted or recomputed when assets are promoted from proxy to high-resolution meshes. Combined with feed-forward inference (~40 ms per frame on an A100), RigMo makes the prospect of automatically rigging massive asset libraries—tens of thousands of characters from 4D generation pipelines, motion capture archives, or simulation outputs—genuinely feasible for the first time.

Finally, the Motion-DiT component, while positioned as a secondary proof-of-concept, establishes a template for structure-aware generative modeling that extends beyond the specific Gaussian-bone instantiation. The architectural pattern—a VAE that factorizes dynamic content into static structure and temporal variation, followed by a diffusion model that generates the dynamic component conditioned on the static component—is a general design principle. It ensures that generated content is automatically compatible with the learned structure, avoiding the physically impossible or topologically inconsistent outputs that plague unconstrained vertex-space generation methods. This template could be applied to facial animation (structure = blendshape basis; variation = expression coefficients), cloth simulation (structure = material parameters; variation = external forces and collision responses), or any domain where a static template and dynamic variation naturally factorize.

Follow-Up Research This Work Enables

Controlled experiment: train RigMo's architecture on static geometry only, then evaluate on held-out motions. The paper's central claim is that motion-driven training is necessary for functional rigging. But the evidence compares RigMo (different architecture, different training data, different objective) against UniRig and MagicArticulate (different everything). A clean ablation would take RigMo's exact architecture—the same dual-path encoder, Gaussian bone decoder, and LBS module—and train one variant on static geometry only (predicting Gaussian bone parameters from the canonical frame, with some proxy loss or KL prior) and another variant on motion sequences (the current setup). Evaluate both on the cross-motion generalization protocol from Section 4.3. If the static-only variant performs comparably to the motion-driven variant, the value proposition of motion-driven training weakens considerably—the architectural innovations (Gaussian bones, geodesic refinement) would carry the improvement. If a large gap remains, the motion-driven claim is strongly validated. Either outcome is informative and refines our understanding of what the architecture versus the training signal contributes.

Procedurally generated dataset with known ground-truth skeletons. The current evaluation cannot distinguish between "RigMo discovered the true articulation" and "RigMo found a rig that happens to reconstruct the training motions." To test whether RigMo recovers causal structure (the actual kinematic decomposition) rather than just a functionally sufficient approximation, a synthetic dataset is essential. Generate articulating objects—humanoids, quadrupeds, mechanical assemblies—with known ground-truth skeletons (bone positions, joint hierarchies, skinning weights). Apply motion sequences to produce mesh animations. Train RigMo on these animations (without providing the ground-truth rig) and measure: (a) how many of the learned Gaussian bones correspond to true bones (matching by spatial overlap or influence region IoU), (b) whether the learned skinning weights approximate the ground truth, and (c) whether RigMo's failure modes (extra bones, missing bones, misaligned influences) correlate with specific object categories or motion complexity. The TrueBones dataset (included in RigMo's training) contains articulated animations with skeletal annotations, and could serve as a partial ground-truth test set if the annotations are reliable. This experiment would determine whether RigMo's representation is identifiable—do the discovered bones have a causal interpretation, or are they merely a convenient basis for reconstruction?

Replace Farthest Point Sampling with learned bone token selection. The current architecture uses deterministic, geometry-based FPS to select initial bone token positions, then lets cross-attention and displacement prediction fine-tune them. This is architecturally simple but potentially suboptimal—FPS guarantees spatial coverage but has no notion of kinematic function. A learned selection mechanism (e.g., attention-based pooling with a learned query vector, or a separate small network that predicts bone locations given the mesh) could place bones at joint centers, mechanical pivot points, or regions of high deformation variability, rather than uniformly across the surface. Compare against the FPS baseline on the cross-motion generalization protocol: does learned selection improve reconstruction accuracy or reduce the number of bones needed? Does it produce more interpretable rigs (bones that correspond more closely to anatomical joints)? The hypothesis is that FPS is a reasonable initialization but learned selection could produce a sparser, more semantically meaningful bone set, particularly for objects where uniform surface coverage over-represents geometrically complex but kinematically simple regions (e.g., a textured but rigid chest plate receiving many bones while a simple but highly mobile joint receives few).

Combined PRM-like refinement for bone placement with iterative motion fitting. The paper treats rig and motion as outputs of a single feed-forward pass. An alternative formulation, inspired by the analysis-by-synthesis approach in the compute-optimal test-time scaling literature, would alternate between (a) predicting bone parameters given current motion estimates and (b) fitting motion parameters given the current rig, iteratively refining both. At test time, this could be run as an inner loop optimization that starts from the feed-forward prediction but adapts the rig to the specific motion sequence being processed—combining the speed of feed-forward inference with the adaptability of per-case optimization. Measure whether iterative refinement improves cross-motion reconstruction beyond the feed-forward baseline, and at what computational cost. The risk is overfitting to the test motion, defeating cross-motion generalization; evaluate whether the refined rig still transfers to held-out motions of the same object. This connects RigMo to a broader class of methods that use test-time optimization to adapt feed-forward predictions to specific instances.

Text-conditioned motion generation in RigMo's latent space. The Motion-DiT experiments (supplementary, Table 5) evaluate sparse-conditioning prediction (1→1 and 1→9 frames) but not open-ended generation from high-level specifications. A natural extension is to condition Motion-DiT on natural language descriptions of motion ("the character walks forward, then jumps") by encoding text into the cross-attention conditioning pathway alongside the static rig features. Train on a dataset of paired text descriptions and mesh animations (TrueBones has some textual annotations; the Objaverse-XL subset could be captioned using a vision-language model). Evaluate generated motion quality using standard metrics (FID on rendered videos, human preference studies) and, critically, measure whether generated motions respect the learned rig constraints—do joint limits emerge naturally, or does the model produce impossible contortions? This would transform RigMo from a representation-learning system into a practical text-to-animation pipeline, and would test whether the structure-aware latent space genuinely constrains generation to physically plausible outputs.

RigMo as a pre-processing step for 4D reconstruction pipelines. Current 4D reconstruction methods (AnimateAnyMesh, GVFDiffusion, Sync4D) output vertex-space animations without explicit structural abstraction. RigMo could be applied as a post-processing step: take the output vertex sequence, infer a rig and motion representation, and output a structured, animatable, reusable asset. The key question is whether RigMo's rigs, trained primarily on synthetic and captured animation data, transfer to the output distribution of 4D generative models, which may contain artifacts, temporal inconsistencies, or deformation patterns not seen during training. Evaluate on a held-out set of 4D reconstructions: does RigMo's rigging generalize, or does it break on out-of-distribution deformations? If it generalizes, this would make RigMo a near drop-in upgrade for making 4D generation outputs controllable and reusable—turning raw vertex streams into production-ready assets.

Practical Applications and Downstream Use Cases

Automated rigging for large-scale 3D asset libraries. Game studios, VFX houses, and simulation companies maintain libraries of tens of thousands of 3D models that need to be rigged for animation. Current practice is either manual (expensive: hours per character by skilled artists) or auto-rigging (fast but unreliable: Figure 4 shows visually plausible rigs that collapse under animation). RigMo's feed-forward inference (40 ms per frame on A100, Section 4.2) can process a 5,000-vertex mesh sequence in under a second, producing both a rig and motion parameters. The resolution-agnostic property means the rig transfers to production-resolution meshes without recomputation. For a studio with 10,000 assets, RigMo reduces rigging time from months of artist labor to hours of GPU time, with the critical advantage that the rigs are validated by motion—they produce correct deformations because they were optimized to do so on training data—rather than merely looking reasonable in static visualization. The cross-motion generalization result (Table 1) provides evidence that these rigs will function correctly across a range of poses, not just the specific poses used during training.

Converting 4D generative model outputs into animatable assets. The emerging ecosystem of 4D diffusion models (AnimateAnyMesh, Sync4D, 4Diffusion, L4GM) can generate impressive mesh animations from text or video, but their outputs are vertex streams—not rigged, not reusable, not editable. A downstream user who generates an animation of a dancing character cannot easily modify the dance, retarget it to a different character, or export it to a game engine. RigMo bridges this gap: take the generated vertex sequence → feed it through RigMo-VAE → obtain a static rig (Gaussian bones + skinning weights) and motion parameters (SE(3) transforms) → export to standard animation formats (FBX, glTF with skeletal animation). The 128-token configuration achieves the best reconstruction (Table 3), preserving the visual quality of the generation while adding editability. This makes 4D generative models practically useful beyond one-shot visualization—they become production tools that output assets, not just animations. The 4× token compression relative to AnimateAnyMesh (48 tokens vs. 512) means this post-processing is computationally lightweight enough to run on every generated output at scale.

Motion retargeting and editing without manual skeleton mapping. Transferring a motion from one character to another (e.g., making a dragon walk like a dog) typically requires manual correspondence specification between source and target skeletons—bone name mapping, coordinate frame alignment, joint limit calibration. RigMo's self-supervised approach learns a shared latent space for rig and motion, which opens an alternative: given two mesh sequences (source motion + target character in canonical pose), encode both through RigMo-VAE, use the source's motion latents to drive the target's Gaussian bones, and decode through LBS. This is not demonstrated in the current paper, but the architectural separation (rig = static, motion = dynamic, shared bone token indexing) makes it structurally natural: the rig defines what can move; the motion defines how it moves. Transfer succeeds to the extent that the source and target have compatible articulation topologies discovered by the model. The practical value is eliminating the manual mapping step that currently bottlenecks motion transfer pipelines—a particularly acute problem for non-humanoid characters where no standard skeleton template exists.

Data-efficient self-supervised pre-training for downstream articulation tasks. RigMo's training requires only mesh sequences and a reconstruction loss—no manual labels, no skeleton templates. This makes it suitable as a pre-training stage for tasks where labeled rigging data is scarce but unlabeled motion data is abundant. For example, animal motion capture (from multi-view video or markerless systems) produces large volumes of mesh sequences without skeletal annotations. Pre-train RigMo on these sequences to learn category-general articulation priors, then fine-tune (with the same reconstruction objective) on smaller, task-specific datasets of particular animal species or motion styles. The learned Gaussian bone representation serves as a strong initialization—the model already knows that deformable objects tend to have coherent, spatially localized articulation regions—reducing the data required for task-specific adaptation. The multi-dataset training setup (DeformingThings4D + TrueBones + Objaverse-XL, ~20K sequences) demonstrates that RigMo learns transferable priors across categories; applying this to domain-specific fine-tuning is a natural extension that the current results support but do not directly evaluate.