ArtLLM: Generating Articulated Assets via 3D LLM

ArXiv: 2603.01142

🎯 Pitch

A 3D large language model can look at a point cloud of any object—door, drawer, scissors—and directly predict every moving part and its kinematic joints in one shot, without ever seeing that specific object before. It cuts joint-fitting time from minutes to seconds and beats retrieval-based methods that were stuck recycling parts from a fixed library.

1. Executive Summary

This paper introduces ArtLLM, a novel framework for generating high-quality articulated 3D assets directly from complete meshes—or from images and text via an upstream 3D generation model—by autoregressively predicting a tokenized blueprint of part layouts and kinematic joints from point clouds using a 3D multimodal large language model. Trained on a curated dataset of 20,673 objects spanning PartNet-Mobility, PhysX3D, and procedurally generated assets, ArtLLM unifies part detection and articulation inference into a single language-modeling pipeline with quantized continuous parameters—discretizing bounding box coordinates into 128 bins, joint origins into 128 bins, and rotation limits into 48 bins—enabling the model to handle variable numbers of parts and joints without the fixed-database retrieval bottlenecks of prior methods. On the PartNet-Mobility test set of 77 objects across 7 categories, ArtLLM achieves a part layout mIoU of 83.0% and a joint type accuracy of 69.3%, substantially outperforming retrieval-based baselines like SINGAPO and Articulate-Anything, while reducing inference time by roughly 2–3 orders of magnitude—completing in seconds rather than minutes—and generalizing robustly to real-world objects for digital twin construction, establishing that language-model-based articulation prediction can replace retrieval pipelines only when the underlying 3D generative model can faithfully reconstruct the object geometry from the input modality.

2. Context and Motivation

The Core Problem: Geometry and Motion Are Treated as Independent Problems

The fundamental gap this paper addresses is the disconnect between 3D geometry generation and kinematic structure prediction. When you look at a door, you don't just see a rectangular panel — you understand that it swings on hinges along a specific axis, with a specific range of motion. This understanding isn't decorative; it's what makes the door functionally useful in a simulation, a game, or a robot's mental model of the world. Yet nearly all work in 3D content creation treats these as separate, sequential problems: first generate the shape, then (optionally, as an afterthought) figure out how it moves.

This gap matters because articulated objects are the building blocks of interactive environments. The paper motivates this across three domains (Section 1):

• Gaming and virtual environments: Players expect objects to behave realistically — drawers that slide open, laptops that close, refrigerator doors that swing. Manually authoring these kinematic properties for every asset is labor-intensive and doesn't scale to the demands of large, procedurally generated worlds.

• Robotics and simulation: Training robots in simulation requires digital twins of real-world articulated objects. If the simulated drawer doesn't open the same way the real one does, the robot's learned manipulation policy fails when deployed. The paper explicitly connects this to the real-to-simulation (real2sim) gap, which is one of the central bottlenecks in scalable robot learning.

• Automated content creation pipelines: As foundation models for 3D generation mature — the paper cites Hunyuan3D, TripoSG, and Trellis as examples of systems that can produce high-fidelity meshes from images or text — the next frontier is making those generated assets functional, not just visually plausible. A generated laptop that can't close isn't really a laptop.

The theoretical significance goes deeper than any single application. Articulation prediction forces a model to reason about object affordances — not just what an object looks like, but what it does. A box with a lid and a box without one may look similar in certain views, but their kinematic structures are fundamentally different. Bridging geometry and motion requires a model to develop representations that encode functional semantics, not just surface-level appearance.

Conflicting Design Philosophies in Prior Work

The paper positions itself against two dominant paradigms in articulated object generation, neither of which fully solves the problem (Section 2):

Optimization-based reconstruction methods — exemplified by PARIS (Liu et al., 2023), VideoArtGS (Liu et al., 2025), ArtGS (Liu et al., 2025), and work by Peng et al. (2025) — take multi-view images or video as input and perform per-object optimization to estimate joint parameters alongside neural scene representations (NeRF or 3D Gaussian Splatting). These methods have three critical weaknesses that the paper catalogs:

1. They are slow. Each object requires its own optimization loop, making them impractical for batch processing or real-time applications. The paper notes "slow, per-object optimization" as a first-order limitation.

2. They produce low-fidelity geometry. Neural representations optimized from sparse views tend to produce blurry or incomplete surfaces, particularly in occluded regions where joints are often located (hinge mechanisms, internal sliding tracks).

3. They are constrained to simple objects. Most optimization-based methods assume a single joint structure — a door that swings, a drawer that slides. Real articulated objects routinely have 4, 6, or 10+ joints (think of an oven with a door, multiple racks, and a control panel), and these methods don't scale to that complexity.

Feedforward retrieval-based methods — including SINGAPO (Liu et al., 2024), CAGE (Liu et al., 2024), URDFormer (Chen et al., 2024), and Articulate-Anything (Le et al., 2024) — train neural networks to predict part layouts and joint parameters, then retrieve part geometries from a fixed, pre-built database. These methods solve the speed problem (inference is feedforward, not iterative) but introduce a different set of limitations:

1. Geometric repetitiveness. Because parts come from a fixed library, the generated assets inherit the geometric biases of that library. Two different ovens will tend to have identical or near-identical door geometries, even if the input images suggest different styles. This fundamentally limits diversity and novelty — exactly the properties that generative models are supposed to provide.

2. Poor generalization to novel shapes. If an input object contains a part geometry that doesn't exist in the retrieval database, the method has no mechanism to synthesize it. It can only approximate with the closest match, which leads to structural inaccuracies (as the paper shows in Figure 4, where baselines fail to recover accurate geometry on real-world examples).

3. Database dependency. The quality of the output is bottlenecked by the coverage and quality of the retrieval library. Building and maintaining such a library for every possible object category is itself a massive curation task.

The paper highlights a deeper conceptual issue with both paradigms: they treat the problem as a detection task (find the parts, find the joints) rather than a joint understanding task where geometry and articulation inform each other. A method that knows a joint is a revolute hinge should have a prior that the connected parts rotate around a shared axis — and, critically, that the part geometries should be consistent with that motion (no intersecting surfaces, appropriate clearance). Neither optimization-based nor retrieval-based methods exploit this mutual information.

Where Emerging 3D Foundation Models Fall Short

A third thread of prior work — and one that makes this paper particularly timely — is the recent explosion of general-purpose 3D generation models (Section 2, "3D Generation" paragraph). The paper cites a progression: early work using Score Distillation Sampling (DreamFusion, Poole et al., 2022) and multi-view synthesis (One-2-3-45, MVDream, Wonder3D) gave way to feedforward large reconstruction models (LRM, LGM, GRM) that could generate 3D assets in seconds rather than hours. More recently, native 3D representations (3DShape2VecSet, Trellis) and diffusion transformer architectures (Clay, Craftsman3D, Hunyuan3D, TripoSG) have pushed generation fidelity to impressive levels.

Critically, several of these foundation models now support part-level generation: OmniPart (Yang et al., 2025), XPart (Yan et al., 2025), PartCrafter (Lin et al., 2025), and others can generate coherent multi-part objects conditioned on bounding box layouts. However, the paper identifies a crucial missing piece:

"a fundamental limitation persists: a disconnect between geometry and motion. These models are unaware of the underlying kinematic structures that dictate how parts should move, leading to a potential mismatch between a part's visual semantics and its intended mechanical role."

In other words, a part generation model can produce a beautiful oven door, but it doesn't know that the door should have hinge geometry on one side and handle geometry on the other, or that the internal face should be inset to accommodate a sealing gasket. These are mechanical constraints that arise from the function of the part, not just its appearance. The paper's insight is that articulation prediction should precede part generation — first decide what moves and how, then synthesize geometry that supports that motion.

Where LLM-Based Approaches to Articulation Fall Short

A small but growing body of work uses language models for articulation modeling (Section 2, final paragraph). Real2Code (Mandi et al., 2024) and URDF prediction work by Li et al. (2025) leverage LLMs' reasoning capabilities to predict joint parameters. However, the paper identifies two critical shortcomings in these approaches:

1. Direct float prediction. These methods have the LLM directly output continuous floating-point values for joint origins, axes, and limits. LLMs are notoriously bad at this — they're token predictors, not function approximators, and small numerical errors compound geometrically (a 1° axis error translates to centimeters of positional error at the far end of a part). The paper's quantization strategy (converting continuous values to discrete bins) is a direct response to this failure mode.

2. Limited training data and poor generalization. These LLM-based methods are trained on relatively small datasets (the paper says "limited data, leading to poor generalization"), which restricts their ability to handle the diversity of real-world articulated objects. This is exacerbated by the fact that their outputs feed into point-cloud-based mesh reconstruction, which "further constrains output quality" — the geometry is only as good as the reconstruction pipeline, and sparse point clouds from LLM predictions don't provide enough geometric detail.

How ArtLLM Positions Itself

The paper's positioning emerges from synthesizing the strengths of these disparate threads while systematically addressing their weaknesses:

From retrieval-based methods, it adopts the speed but rejects the database dependency. ArtLLM predicts part layouts and joints in a single feedforward pass, like SINGAPO or URDFormer, achieving inference times of seconds rather than the minutes-to-hours of optimization methods. But instead of retrieving parts, it conditions a generative model (XPart) on the predicted layout, enabling novel geometry synthesis. This is the key architectural innovation: separate the structural reasoning (what parts exist, how they connect) from the geometric realization (what those parts look like).

From optimization-based methods, it adopts the goal of accurate articulation but rejects the per-object cost. ArtLLM produces joint parameters (axis, origin, limits) that guide subsequent generation, but does so amortized across a dataset rather than optimized per-instance. The physics-based joint-limit correction (Section 3.4) is effectively a lightweight, targeted optimization that addresses the most common failure mode (self-collision) without the full cost of joint-and-geometry co-optimization.

From 3D foundation models, it adopts the generative capability but adds articulation awareness. The paper explicitly builds on XPart for part geometry synthesis, treating it as a modular component. This is a deliberate design choice: as part generation models improve, ArtLLM's output quality improves with them, without requiring changes to the articulation prediction pipeline. The articulation blueprint (part bounding boxes + joint definitions) serves as a clean interface between the reasoning module and the generation module.

From LLM-based articulation work, it adopts the language-modeling paradigm but fixes the numerical instability. The quantization strategy — coordinates into 128 bins, rotation limits into 48 bins, joint axes into a 128-entry codebook hierarchically sampled from coordinate planes and a Fibonacci sphere — converts a regression problem into a classification-over-discrete-tokens problem that LLMs handle naturally. The codebook design is particularly thoughtful: it densely covers axis-aligned directions (the vast majority of real joints) while retaining flexibility for off-axis orientations.

From all prior work, it inherits the data scarcity problem and addresses it through curation and augmentation. The training dataset (Table 1) combines PartNet-Mobility (the standard benchmark for articulated objects), PhysX3D (which adds physical property annotations), and 12,000 procedurally generated assets from Infinite-Mobility (which provides structural diversity beyond what manually modeled datasets contain). After filtering (removing objects with >20 joints, tiny parts like keyboard buttons) and normalizing (watertight meshes, global coordinate frames, [−0.9, 0.9] range), the dataset spans 20,673 objects across 43 categories — substantially larger and more diverse than what prior LLM-based methods trained on.

The Unstated Assumption That Makes This Work

Underlying the entire framework is an assumption that deserves explicit attention: the input point cloud contains sufficient geometric information to infer kinematic structure. This is not obviously true. A point cloud of a closed laptop looks very similar to a point cloud of a solid rectangular block — the articulation is latent in the geometry, not directly visible from a single view of the closed state. The paper's method works because:

1. It trains on objects in their canonical (often partially open) states, where articulations are observable from geometry alone.
2. The upstream 3D generation model (Hunyuan3D 3.0) produces full 3D meshes from images, so even if the input image shows a closed object, the generated mesh can incorporate learned priors about expected 3D structure.
3. The encoder (Point Transformer v3) operates on the full point cloud with normals, capturing fine surface discontinuities at part boundaries that signal articulation seams.

This assumption has boundary conditions that the paper's failure cases (Figure 9) begin to explore: when the articulation is internal and fully occluded (e.g., the bread-rack structure in Case 103555), neither the input geometry nor the articulation model can recover it. This is a fundamental limitation of single-view or single-state input that the paper acknowledges but doesn't solve — it's inherent to the problem formulation, not a weakness of the specific method.

3. Technical Approach

3.1 Reader Orientation

This paper presents a pipeline system that takes a 3D point cloud as input and produces a fully articulated 3D asset — complete with part geometries, joint types, axis directions, joint origins, and collision-free motion limits — suitable for direct use in physics simulators. The system solves the problem of disconnected geometry and articulation prediction by decomposing it into three sequential stages: first, a language model reasons about what parts exist and how they connect (the kinematic blueprint); second, a generative model synthesizes what those parts look like (the geometry) conditioned on that blueprint; third, a physics-based post-processing step ensures the resulting motion is collision-free (the physical refinement). This is fundamentally a language-modeling approach to structured prediction, where continuous 3D parameters (bounding box coordinates, joint axes, motion limits) are converted into discrete tokens so that a pretrained LLM can autoregressively generate them — the same way it would generate text — while a separate 3D encoder handles the geometric perception.

3.2 Big-Picture Architecture (Diagram in Words)

The system has four major components connected in sequence:

1. Input processing pipeline: Raw input (image, text, or existing mesh) is converted into a canonical 3D mesh via an off-the-shelf generation model (Hunyuan3D 2.5 or 3.0 for image/text inputs), then uniformly sampled into 32,768 surface points with normals that feed into the encoder.

2. 3D Articulation Language Model (ArtLLM): A Point Transformer v3 encoder projects the point cloud into token embeddings that a Qwen3 0.6B LLM consumes, autoregressively generating a tokenized blueprint — first all part bounding boxes, then all joint definitions — using quantized discrete tokens for all continuous parameters.

3. Part-Aware Geometry Synthesizer: The predicted bounding boxes, after an expansion step that ensures complete point cloud coverage, condition a generative model (XPart) that synthesizes high-fidelity part geometries, one per predicted part.

4. Physics-Based Joint Limit Corrector: Each predicted joint undergoes collision detection along its motion range, with a hierarchical search identifying the precise angle (or distance) of first contact, which becomes the refined collision-free limit.

Information flows strictly forward: raw input → mesh → point cloud → ArtLLM → tokenized blueprint → expanded bounding boxes → part geometries + joints → collision-corrected joints → simulation-ready URDF asset.

3.3 Roadmap for the Deep Dive

• First, the quantization strategy and token vocabulary — because this is the foundational design decision that makes LLM-based articulation prediction numerically stable, converting continuous regression into discrete classification. Without understanding the quantization, nothing else in ArtLLM makes sense.
• Second, the 3D encoder and modality bridging — how raw point clouds become a representation the LLM can consume, including the architectural choices (Point Transformer v3, position embeddings, MLP projector) and why alternatives were rejected.
• Third, the autoregressive generation template and output structure — the exact sequence the LLM produces, the ordering of part and joint tokens, and the rationale for predicting parts before joints.
• Fourth, the multi-task, multi-stage training strategy — the three SFT tasks, the two-stage curriculum (geometry first, then kinematics), and why this decoupling was necessary for stable training.
• Fifth, the training corpus construction — the dataset curation pipeline (filtering, simplification, normalization, normal correction), the three data sources, and the final statistics that enable the model to handle variable numbers of parts and joints.
• Sixth, the part-aware geometry synthesis with bounding box expansion — how predicted layouts connect to generative models, and the point-assignment-and-expansion mechanism that prevents truncated parts.
• Seventh, the physics-constrained joint limit correction — the collision-detection algorithm, the derivative-based spike detection, the hierarchical search, and why post-hoc correction is necessary given that ArtLLM sees only a single static pose.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems paper whose core idea is that articulation prediction can be reformulated as a language modeling problem by quantizing all continuous parameters into discrete token spaces, enabling a 3D-aware LLM to autoregressively generate a complete kinematic blueprint that subsequently conditions a part-level generative model — avoiding both the slow per-object optimization of reconstruction methods and the geometric repetitiveness of retrieval-based methods.

The Quantization Strategy: Converting Continuous Articulation Parameters into Discrete Tokens

The central technical challenge in having an LLM predict articulation parameters is that LLMs are token predictors, not continuous function approximators. Directly regressing floating-point values (as prior LLM-based methods like Real2Code and Li et al. 2025 do) produces numerically unstable outputs because the model has no inductive bias toward smoothness or physical consistency — small token prediction errors compound into large geometric errors. A 1° error in predicting a joint axis direction translates to centimeters of positional error at the far end of a long part (e.g., the tip of a refrigerator door).

ArtLLM's solution is comprehensive quantization: every continuous parameter that the model must predict is discretized into a fixed vocabulary of special tokens, converting a regression problem into a classification-over-discrete-tokens problem. The vocabulary is organized into four distinct token families, each with its own quantization granularity chosen based on the required precision for that parameter type.

Bounding box coordinate quantization. Each part is parameterized by its 3D axis-aligned bounding box (AABB), defined by six values: $x_{\text{min}}, y_{\text{min}}, z_{\text{min}}, x_{\text{max}}, y_{\text{max}}, z_{\text{max}}$. All coordinates are first normalized to the range $[-1, 1]$, then quantized into 128 discrete bins per axis:

$\hat{c}_{\text{min}} = \left\lfloor \frac{(c_{\text{min}} + 1)}{2} \times 128 \right\rfloor, \quad \hat{c}_{\text{max}} = \left\lceil \frac{(c_{\text{max}} + 1)}{2} \times 128 \right\rceil$

where $c \in [-1, 1]$ is a normalized continuous coordinate value and $\hat{c} \in \{0, 1, \ldots, 127\}$ is its corresponding discrete bin index. The floor operation for minimum coordinates and ceiling for maximum coordinates are deliberate: they ensure that the quantized bounding box always contains the original continuous bounding box, never shrinking it and risking part truncation.

What this computes: given a part's continuous bounding box expressed in the normalized coordinate frame, this formula maps each of the six coordinate values to an integer between 0 and 127. For example, a part spanning from $x = -0.3$ to $x = 0.5$ would have its $x_{\text{min}}$ mapped to bin $\lfloor (-0.3 + 1) / 2 \times 128 \rfloor = \lfloor 44.8 \rfloor = 44$ and $x_{\text{max}}$ mapped to bin $\lceil (0.5 + 1) / 2 \times 128 \rceil = \lceil 96 \rceil = 96$. The resulting token sequence for one part is: BBox(<P_44>, <P_y_min>, <P_z_min>, <P_96>, <P_y_max>, <P_z_max>), where each <P_N> is a special token from a vocabulary of 128 coordinate tokens.

Why 128 bins: the choice of 128 bins provides approximately 1.56 cm resolution for an object normalized to the $[-1, 1]$ range — sufficient to capture part boundaries for household objects (a 1-meter object would have ~0.78 cm resolution). Using fewer bins would lose geometric precision; using more would increase vocabulary size and potentially reduce training stability by making the classification space sparser. The paper does not ablate this specific number, but it appears motivated by the balance between precision and vocabulary compactness.

Joint origin quantization. Joint origins (the 3D point in space where the joint is located) use the same 128-bin quantization per axis as bounding box coordinates, since they operate in the same normalized coordinate space and require similar precision. A joint origin is represented as three tokens: [<P_i>, <P_j>, <P_k>] where each token indexes a coordinate bin.

Joint limit quantization. Motion limits require different precision depending on the joint type. For rotational limits (revolute, continuous, and screw joints), the paper discretizes angles into 48 bins over the range $[-2\pi, 2\pi]$:

$\hat{\theta} = \text{round}\left(\frac{\theta + 2\pi}{4\pi} \times 48\right)$

where $\theta \in [-2\pi, 2\pi]$ is the continuous angle and $\hat{\theta} \in \{0, \ldots, 47\}$ is the discrete bin. The range $[-2\pi, 2\pi]$ covers a full rotation in both directions, handling joints that can rotate beyond a single full circle. With 48 bins, each bin spans $4\pi / 48 = \pi/12$ radians, or 15 degrees — a reasonable resolution for practical joint limits (a drawer that opens 45° vs. 60° is a meaningful distinction; 45° vs. 46° usually isn't).

For translational limits (prismatic and screw joints), the paper discretizes distances into 64 bins over the range $[-2, 2]$ in normalized units:

$\hat{d} = \text{round}\left(\frac{d + 2}{4} \times 64\right)$

where $d \in [-2, 2]$ is the continuous displacement and $\hat{d} \in \{0, \ldots, 63\}$ is the discrete bin. The range $[-2, 2]$ in normalized coordinates corresponds to the full extent of the normalized bounding volume, appropriate for translational joints (sliding drawers, telescoping mechanisms) that can traverse most of the object's length. With 64 bins, each bin spans 0.0625 normalized units, or roughly 3 cm for a 1-meter object.

Joint axis quantization with a hierarchical codebook. The joint axis direction — the 3D unit vector around which a revolute joint rotates or along which a prismatic joint translates — is the most geometrically sensitive parameter. Unlike origins or limits, which are scalar values along independent axes, an axis is a point on the unit sphere that must be predicted as a coherent direction. Furthermore, the paper makes a key empirical observation:

"most joint axes align with the coordinate axes"

This is a structural prior about the world: doors swing on vertical axes, drawers slide horizontally, laptop lids rotate around horizontal axes. Encoding this prior explicitly improves prediction accuracy and reduces the effective complexity of the problem.

The solution is a hierarchical 128-entry codebook. The construction proceeds in two phases:

1. Dense coverage of coordinate-axis-aligned directions. Points are uniformly sampled from the unit circles on the XY, YZ, and XZ planes. This provides high-density coverage of exactly axis-aligned and near-axis-aligned directions — the vast majority of real-world joint axes. For example, the XY-plane circle covers all directions in the horizontal plane (typical for revolute joints on furniture), while the YZ and XZ circles cover vertical-longitudinal and vertical-lateral planes.

2. Sparse coverage of off-axis directions. Additional points are obtained via Farthest Point Sampling (FPS) on a Fibonacci sphere, which provides approximately uniform coverage of the remaining surface of the unit sphere with maximum dispersion (minimizing the maximum distance from any direction to its nearest codebook entry).

Each of the 128 points on the unit sphere receives a unique token <D_i>. During inference, the LLM predicts one of these 128 tokens to specify the joint axis.

What this codebook achieves: it concentrates representational capacity where it matters most (axis-aligned directions) while still providing a fallback for arbitrary orientations. A uniform discretization of the sphere would waste most of its 128 entries on directions that almost never occur in articulated objects. By allocating entries proportional to the expected distribution of joint axes, the codebook achieves higher effective precision for the common case without sacrificing the uncommon case.

Why not predict continuous axis values: directly predicting three floating-point numbers (the axis vector components) and then normalizing would require the LLM to learn the unit-norm constraint implicitly. Small prediction errors could produce non-unit vectors, and the normalization operation creates non-linear error propagation that LLMs handle poorly. The codebook approach guarantees that every predicted token maps to a valid unit vector.

The 3D Encoder: Bridging Point Clouds and Language

The input to ArtLLM is a point cloud, but the reasoning engine is a text-pretrained language model. Bridging this modality gap requires an encoder that converts unstructured 3D geometry into a sequence of embeddings that the LLM can attend to alongside text tokens.

Input representation. Regardless of the original input modality (image, text, or mesh), ArtLLM always operates on a point cloud with normals. For image or text inputs, the paper leverages off-the-shelf 3D generation models (Hunyuan3D 2.5 or Hunyuan3D 3.0, cited as [lai2025hunyuan3d]) to first produce a complete 3D mesh, then uniformly samples 32,768 surface points. This fixed sampling count ensures consistent encoder input size across all objects. Surface normals are included because they encode fine geometric detail — the orientation of a surface at a point is a strong cue for part boundaries and articulation seams (e.g., the gap between a door and its frame manifests as opposing normal directions on adjacent surfaces).

The paper specifies a preprocessing step: meshes are made watertight before sampling, which "ensure[s] the consistency of normals." Non-watertight meshes (common in CAD models) can have flipped or missing normals at boundaries, which would inject noise into the encoder input.

Encoder architecture: Point Transformer v3. The paper selects Point Transformer v3 [wu2024point] as the point cloud encoder, citing its "powerful yet efficient design." Point Transformer v3 is a transformer architecture adapted for 3D point clouds that applies self-attention within local neighborhoods (using vector self-attention with positional encoding based on 3D coordinates), enabling it to capture both local geometric structure and long-range dependencies. Unlike voxel-based or multi-view encoders, it operates directly on unstructured point sets without quantization or projection losses.

The choice of Point Transformer v3 (rather than, say, a PointNet++ or a sparse convolutional network) reflects two requirements: (1) it must produce a sequence of token embeddings compatible with the LLM's transformer architecture, and (2) it must handle variable-density point clouds robustly, since different objects have different surface areas and the uniform sampling produces varying local point densities.

Position embedding augmentation. Following SpatialLM [mao2025spatiallm], the paper augments the final layer's features with explicit position embeddings. The motivation is that the transformer's self-attention, while powerful, can lose fine-grained spatial information through the layers — the features at a point may encode what the local geometry looks like (a corner, a flat surface, an edge) but not precisely where in 3D space it is. Position embeddings inject this absolute location information directly, which is crucial for predicting bounding box coordinates and joint origins (both absolute spatial quantities).

Modality alignment via MLP projection. The Point Transformer v3 output is a set of feature vectors (one per input point or per downsampled point, depending on the architecture's pooling strategy). These features live in the encoder's representational space, not the LLM's embedding space. A two-layer MLP with a simple architecture projects each point feature to match the LLM's token embedding dimension. This is the standard connector design from vision-language models (e.g., LLaVA's linear projector or Qwen-VL's MLP), adapted for 3D.

LLM backbone: Qwen3 0.6B. The language model is Qwen3 [yang2025qwen3] with 0.6 billion parameters — a relatively small LLM by contemporary standards (for context, LLaMA-7B is over 10× larger). This choice is pragmatic: the articulation prediction task has a constrained output vocabulary (128 coordinate tokens + 128 axis tokens + 48 rotation limit tokens + 64 translation limit tokens + structural tokens like BBox, RevoluteJoint, etc.) and requires reasoning about spatial relationships, not broad world knowledge or complex multi-turn dialogue. A 0.6B model is sufficient for the task while keeping training and inference computationally tractable (the full training pipeline runs on 8 H20 GPUs, which is feasible for a research lab but would be challenging with a 7B+ model).

The autoregressive generation process. The point cloud tokens and a short text instruction (e.g., "Detect part boxes and joints.") are concatenated into the LLM's input sequence. The LLM then autoregressively generates output tokens until an end-of-sequence marker. These output tokens are parsed according to the structured template (Section 3.1 of the paper) to extract the part bounding boxes and joint definitions.

Design choice: why an LLM rather than a dedicated architecture? The paper could have used a graph neural network to predict joints from part features, or a specialized transformer for set prediction. The LLM approach offers several advantages that the paper leverages implicitly: (1) LLMs are pretrained on vast corpora and have learned general reasoning patterns that transfer to spatial tasks; (2) the autoregressive formulation naturally handles variable numbers of parts and joints — the model can generate 3 parts and 2 joints for a simple object or 15 parts and 20 joints for a complex one, with the same architecture; (3) the text template provides a structured output format that is human-readable and debuggable, unlike raw neural network outputs; (4) the multi-task training (predicting parts only, joints only, or both) maps naturally to different text prompts, a capability that LLMs handle natively through instruction following.

The Output Template: How Articulation Becomes Language

The LLM does not produce free-form text describing the articulated object. It produces a rigorously structured sequence that can be deterministically parsed back into geometric and kinematic parameters. The template design has three key properties: it is complete (all parameters needed for URDF export are specified), unambiguous (each token has exactly one interpretation), and ordered (parts before joints, enabling joint prediction to condition on the full part layout).

Part bounding box template. Each part is defined by an identifier and its axis-aligned bounding box:

bbox_{id} = BBox(
    <P_x_min>, <P_y_min>, <P_z_min>,
    <P_x_max>, <P_y_max>, <P_z_max>
)

The {id} is an integer assigned sequentially (0, 1, 2, ...) based on sorting order (z–y–x minimum coordinate, following OmniPart's convention). The <P_N> tokens are the quantized coordinate values from the 128-bin vocabulary. The BBox( and ) are literal text tokens that provide structural separation.

Joint definition templates. Four joint types are supported, each with its own template reflecting its parameterization:

Revolute joint (rotational hinge, most common):

joint_{id} = RevoluteJoint(
    parent, child,
    <D_axis>,
    [<P_ox>, <P_oy>, <P_oz>],
    [<LR_min>, <LR_max>]
)

where parent and child are integer IDs of the connected part bounding boxes; <D_axis> is a token from the 128-entry axis codebook; [<P_ox>, <P_oy>, <P_oz>] are three tokens specifying the joint origin (rotation pivot point); and [<LR_min>, <LR_max>] are two tokens from the 48-bin rotation limit vocabulary.

Continuous joint (unbounded rotation, e.g., wheels):

joint_{id} = ContinuousJoint(
    parent, child,
    <D_axis>,
    [<P_ox>, <P_oy>, <P_oz>]
)

Identical to the revolute joint but without rotation limits — the joint can rotate freely without angular constraints.

Prismatic joint (linear sliding, e.g., drawers):

joint_{id} = PrismaticJoint(
    parent, child,
    <D_axis>,
    [<LT_min>, <LT_max>]
)

Here <D_axis> is the sliding direction (from the same 128-entry codebook used for rotation axes — the codebook represents unit vectors regardless of joint type). [<LT_min>, <LT_max>] are two tokens from the 64-bin translational limit vocabulary. Note the absence of an origin: prismatic joints don't have a pivot point in the same sense as revolute joints; their position is defined by the translation range along the axis.

Screw joint (helical motion combining rotation and translation — less common but present in mechanisms like vise clamps):

joint_{id} = ScrewJoint(
    parent, child,
    <D_axis>,
    [<P_ox>, <P_oy>, <P_oz>],
    [<LT_min>, <LT_max>]
)

Combines a rotation axis and origin (like a revolute joint) with translational limits (like a prismatic joint), reflecting the coupled rotational-translational motion.

Generation ordering and separator tokens. The full output sequence follows a fixed structure:

<|layout_start|>
    bbox_0 = BBox(...)
    bbox_1 = BBox(...)
    ...
    bbox_N = BBox(...)
<|layout_end|>
<|art_start|>
    joint_0 = RevoluteJoint(...)
    joint_1 = PrismaticJoint(...)
    ...
    joint_M = ...
<|art_end|>

The <|layout_start|>, <|layout_end|>, <|art_start|>, and <|art_end|> are special tokens that demarcate the two sections. The paper states explicitly that this ordering ensures "joint prediction is conditioned on the complete part layout, improving structural coherence." This is a critical design decision: predicting joints requires knowing which parts exist (you can't connect part 3 to part 7 if part 7 hasn't been predicted yet), and having all parts available before any joint prediction allows the model's attention mechanism to consider the full spatial arrangement when deciding joint connectivity.

Part and joint sorting. Because autoregressive models are sensitive to the order of generated elements (the model learns a left-to-right generation distribution), the paper applies deterministic sorting to convert the inherently unordered sets of parts and joints into ordered sequences. Parts are sorted by their minimum coordinates in z–y–x order (following OmniPart's convention): parts are first ordered by $z_{\text{min}}$ (lowest first), then by $y_{\text{min}}$ for ties, then by $x_{\text{min}}$ for remaining ties. This provides a unique, deterministic ordering for any set of non-overlapping bounding boxes. Joints are sorted by their child link ID in ascending order — the child part determines the joint's position in the sequence. This sorting is applied to both training data and model outputs, ensuring consistency.

Multi-Task, Multi-Stage Training Strategy

Training ArtLLM to accurately predict both part layouts and articulations from point clouds is a challenging optimization problem. The paper decomposes it into three supervised fine-tuning (SFT) tasks and two training stages, with a curriculum that first grounds the model in geometric understanding before introducing kinematic reasoning.

The three SFT tasks:

1. Part Layout Prediction (Task 1): Given a point cloud, predict only the part bounding boxes. The input prompt is the minimal "Detect part boxes." The model outputs the <|layout_start|> ... <|layout_end|> section only. This task trains the encoder and LLM to identify parts and localize them in 3D space — a geometric perception problem.

2. Kinematic Prediction (Task 2): Given a point cloud AND the ground-truth part layout (as text tokens), predict the joints. The input prompt is "Given part boxes, detect joints." The model receives the point cloud tokens, the ground-truth bounding box tokens, and outputs the <|art_start|> ... <|art_end|> section. This task isolates kinematic reasoning: the model knows exactly where the parts are (from ground truth) and must only figure out how they connect and move.

3. End-to-End Articulation Prediction (Task 3): Given only a point cloud, predict both parts and joints. The input prompt is "Detect part boxes and joints." The model outputs the full <|layout_start|> ... <|layout_end|> <|art_start|> ... <|art_end|> sequence. This is the deployment task — the model must perform both geometric perception and kinematic reasoning jointly.

The multi-task data mixing ratio is set to 3:2:5 (Tasks 1, 2, and 3 respectively). Task 3 gets the highest weight because it's the target deployment task, but Tasks 1 and 2 provide auxiliary supervision that improves shared representations.

Two-stage training curriculum:

Stage 1: Geometric grounding — The model is trained exclusively on Task 1 (part layout prediction) using mixed data from the full training corpus (all three data sources). The point encoder is initialized from P3SAM [ma2025p3], a model pretrained on large-scale part segmentation. P3SAM provides a strong prior for identifying part-level geometry — it has already learned to distinguish different object parts from point cloud features. The encoder and projector are trained for 50 epochs on 8 H20 GPUs, requiring approximately 8 hours. The output of this stage is a point encoder and projector that produce features rich in part-level geometric information.

Why P3SAM initialization: starting from a part-segmentation pretrained model gives the encoder a head start in recognizing part boundaries, junctions, and surface discontinuities — all of which are cues for where one functional part ends and another begins. Random initialization would require the model to learn part-level features from scratch, which is substantially harder given the limited dataset size (20,673 objects).

Stage 2: Multi-task articulation learning — The encoder and projector weights from Stage 1 are loaded, and the model is trained on all three tasks simultaneously with the 3:2:5 ratio. Training proceeds for 30 epochs on 8 H20 GPUs, requiring approximately 15 hours. The LLM backbone (Qwen3 0.6B) is trained from its standard pretrained weights — it receives no special initialization beyond its language pretraining.

Why this curriculum helps: the paper's ablation (Experiment D, Table 3) shows that removing multi-stage training and simply using P3SAM-initialized encoders for end-to-end training lowers both part and joint prediction accuracy. The mechanism is that Stage 1 decouples geometric understanding from kinematic reasoning during initial training. The encoder first learns a robust part-level feature representation without the confounding objective of also predicting joints. When joint prediction is introduced in Stage 2, the encoder already produces clean part features, allowing the LLM to focus on the harder problem of inferring kinematic relationships from those features. If both objectives are trained from the start, the gradients from joint prediction early in training (when part features are still poor) can destabilize the encoder's learning.

Training hyperparameters (Section 8): The paper uses the LLaMA-Factory and SpatialLM codebases as training infrastructure. Optimization uses a cosine learning rate scheduler with a maximum learning rate of $1 \times 10^{-5}$ and a warmup ratio of 0.03 (meaning the first 3% of training steps linearly increase the learning rate from 0 to $1 \times 10^{-5}$, after which cosine decay applies). The cross-entropy loss is used as the optimization objective for all SFT tasks — the model is trained to predict the correct next token at each position in the output sequence, with loss computed only on the output tokens (not the input point cloud tokens or prompt tokens).

Data augmentation: During training, 3D point clouds are augmented with random scaling and rotation, applied independently to each sample with probability 0.75. Random scaling samples a scale factor $s \in [0.8, 1.05]$ and applies it uniformly to the object geometry and all parameters (bounding boxes, joint origins, limits). Random rotation selects an angle $\theta \in \{90°, 180°, 270°\}$ and rotates the object around the y-axis. Critically, when augmentation is applied, the part layouts and articulation parameters are transformed accordingly — bounding boxes are rotated and scaled, joint origins are transformed, and joint axes are rotated. This ensures the model sees consistent input-output pairs rather than learning to ignore orientation.

Why these specific augmentations: the 90°-increment rotations are chosen because most real-world articulated objects have a consistent upright orientation (gravity-aligned), and arbitrary rotations could create unrealistic configurations that confuse the model rather than helping. The scale range $[0.8, 1.05]$ is relatively narrow, preserving the object's approximate size while providing some size invariance. The paper's ablation (Experiment C, Table 3) confirms that removing these augmentations degrades part mIoU performance — the model becomes overfit to the canonical orientations and scales of the training data.

Training Corpus Construction and Curation

The performance of any learned articulation prediction system is fundamentally bounded by the quality, diversity, and scale of its training data. The paper constructs a new dataset by aggregating and refining three sources, applying a series of preprocessing steps to ensure consistency and remove problematic samples.

Data sources (Table 1):

• PartNet-Mobility [xiang2020sapien]: The standard benchmark for articulated object research, containing 2,346 objects across 46 categories with manually annotated part segmentations and kinematic structures. PartNet-Mobility provides the highest-quality annotations but is relatively small.

• PhysX3D [cao2025physx]: A dataset of 3D objects with physical property annotations. The paper uses 6,327 objects from PhysX3D, which adds material properties (mass, friction) beyond what PartNet-Mobility provides, though the paper primarily uses it for the geometric and kinematic annotations.

• Infinite-Mobility [lian2025infinite]: A procedural generation system that creates articulated objects with diverse kinematic topologies. The paper generates 12,000 synthetic assets using Infinite-Mobility's procedural method. These objects have known ground-truth structures by construction but exhibit greater structural diversity than manually modeled datasets because the procedural rules can generate combinations (numbers of parts, joint types, topological arrangements) that don't exist in the manually curated datasets.

The total dataset after preprocessing contains 20,673 objects across 43 categories. Figure 7 (Appendix) shows the distribution of part counts: most objects have 2–6 parts, but the dataset includes objects with up to 21 parts (the paper filters out objects with more than 20 joints, so links can number up to 21). The paper notes that "we have collected a sufficient number of samples with larger part counts," ensuring the model can handle objects with many articulated components.

Preprocessing pipeline (Section 3.2):

1. Filtering: Objects with more than 20 joints are removed — these are typically overly complex mechanisms that would dominate the loss and aren't representative of common household objects. Categories containing excessively small parts are excluded; the paper names keyboard and remote as examples. Within remaining categories, parts smaller than a volume threshold are filtered out. This threshold is not specified numerically, but the criterion is practical: tiny parts like individual keyboard keys are geometrically unresolvable from point clouds and their articulations (key presses) aren't the type of kinematic structure the paper targets.

2. Structure simplification: Fixed joints (joints with zero degrees of freedom, used in URDF to rigidly connect components like handles to doors) are removed, and their connected links are merged. This reduces the part count to only the functionally independent moving components — a door with a fixed handle becomes a single part rather than two. Screw joints, which URDF typically represents as a combination of a revolute and prismatic joint on the same axis, are merged into a single screw joint. This simplification reduces the number of joints the model must predict and removes a common source of kinematic redundancy.

3. Normalization: All joint parameters are transformed into a global coordinate frame (centered on the object, with consistent axis orientation). Geometry and parameters are normalized to the range $[-0.9, 0.9]$. The slightly smaller than $[-1, 1]$ range (used in quantization) likely provides a small margin to prevent numerical issues at the boundaries.

4. Normal correction: Some models in PartNet-Mobility have incorrect or inconsistent surface normals (a known issue with CAD-derived models where face orientations can be flipped). The paper applies watertight reconstruction — converting the mesh to a watertight representation (likely using a method like TSDF fusion or Poisson surface reconstruction) and then recomputing normals from the watertight surface. This ensures the point cloud normals that the encoder receives are geometrically consistent.

Dataset statistics by the numbers (Table 1):

• Total objects: 20,673
• Total categories: 43
• Objects from PartNet-Mobility: 2,346
• Objects from PhysX3D: 6,327
• Objects from Infinite-Mobility (procedural): 12,000
• Objects with known URDF annotations (all sources): 20,673
• Total parts across all objects (implied from part count distribution): in the range of ~100,000–150,000 (estimated from an average of ~5–7 parts per object)
• Total joints: similar order of magnitude to total parts minus number of objects (each object has $N_{\text{parts}} - 1$ joints if it forms a tree, though some objects have kinematic loops)

Test set partitioning: The evaluation uses a specific split of PartNet-Mobility from SINGAPO [liu2024singapo]: 77 objects across 7 categories (Storage, Table, Refrigerator, Dishwasher, Oven, Washer, Microwave). These objects are held out from all training — they never appear in the data from any source during training. This is a stricter evaluation than typical random splits because it tests generalization to specific object instances in specific categories that may have different structural characteristics than the training categories.

Part-Aware Geometry Synthesis with Bounding Box Expansion

Once ArtLLM predicts the kinematic blueprint (part bounding boxes and joints), the next stage is synthesizing the actual 3D geometry for each part. The paper adopts XPart [yan2025x] as its part generation backbone, but the bounding box predictions from ArtLLM require a preprocessing step before they can condition XPart effectively.

Why XPart: XPart is a recent part-level generative model that can synthesize individual part geometries conditioned on bounding box inputs. Unlike retrieval-based methods (which select parts from a database) or SDF-based reconstruction (which often produces low-quality surfaces), XPart generates novel, high-fidelity meshes. The paper treats XPart as a modular component — improvements in part generation models can be swapped in without changing the articulation prediction pipeline.

The bounding box coverage problem. ArtLLM predicts bounding boxes that approximate the spatial extent of each part. However, predicted bounding boxes may not perfectly encompass the ground-truth part geometry — they can be slightly too small, shifted, or rotated (though the AABB parameterization prevents rotation misalignment, the bounding box coordinates can still mis-estimate the true extent). If these imperfect bounding boxes are fed directly to XPart, the generated parts may be truncated or miss regions near the part boundaries. This is particularly problematic for parts with complex shapes, where small bounding box errors lead to missing geometric features.

The bounding box expansion mechanism (Section 3.3). The paper introduces a simple but effective correction:

1. Point assignment: For every point in the input object's point cloud, check whether it lies inside any predicted bounding box. Points that fall inside at least one box are assigned to that box (or to the box whose center is closest, for points in overlapping regions). Points that fall outside all predicted boxes — the "uncovered" points — are assigned to the nearest box based on Euclidean distance from the point to the box center.

2. Box expansion: Each bounding box is expanded just enough to tightly enclose all points newly assigned to it. The expansion modifies the $x_{\text{min}}, y_{\text{min}}, z_{\text{min}}$ and $x_{\text{max}}, y_{\text{max}}, z_{\text{max}}$ values of each box to be the minimum and maximum coordinates of its assigned point set.

What this achieves: the expansion ensures that every point in the input point cloud is covered by at least one bounding box, and that each bounding box is large enough to contain all points that logically belong to its part. This prevents geometric truncation when XPart generates parts — the generative model sees a bounding box that fully encompasses the part's spatial extent, even if ArtLLM's raw prediction was slightly too small.

Why distance-to-center assignment: assigning uncovered points to the nearest box center is a heuristic that assumes parts are roughly convex and centered within their bounding boxes. For articulated objects with non-overlapping parts (most household objects), this heuristic works well because uncovered points are typically near the boundaries of one specific part. An alternative — assigning to the nearest box surface — would be more geometrically precise but requires computing point-to-box distances, which is more expensive for 32,768 points.

Connection to XPart. The expanded bounding boxes, along with the full object point cloud and the part assignment (which points belong to which part), are provided to XPart. XPart then generates a complete 3D mesh for each part individually. These part meshes, combined with the joint definitions from ArtLLM, produce a complete articulated asset in URDF format.

Design choice: geometry conditioned on layout, not vice versa. The paper's architecture deliberately places articulation prediction before geometry synthesis. An alternative would be to generate part geometries first and then predict joints from the generated geometry — this is essentially what the LLM-based methods (Real2Code) attempt, using point-cloud-based mesh reconstruction. The paper's ordering is superior because articulation constraints can inform geometry generation (a hinge joint implies the connected parts should have complementary cylindrical geometry at the pivot), while the reverse direction (inferring joints from generated geometry) is harder because generative models may produce parts with geometrically ambiguous interfaces.

Physics-Constrained Joint Limit Correction

The final stage of the pipeline addresses a fundamental limitation of single-state prediction: ArtLLM sees the object in only one static pose (the canonical state used during training), but joint limits are inherently dynamic properties — they define the range of motion, which cannot be fully observed from a single configuration. For example, a partially open drawer could have a maximum opening of either 30 cm or 60 cm — both are consistent with the observed state if the drawer is currently opened 20 cm.

The self-collision problem. The model predicts joint limits based on geometric features it can observe in the static pose. However, these limits may be too large, causing the moving part to collide with other parts when articulated through the predicted range. This is particularly common for parts that are near other structures — a refrigerator door predicted to open 180° might collide with an adjacent cabinet, or an oven rack predicted to slide out fully might intersect with the oven frame.

Collision detection algorithm (Section 3.4, Figure 3). The correction method operates per-joint and uses the generated part geometries (from XPart) to simulate motion and detect contacts:

1. Articulate through the predicted range: For a revolute joint with predicted limits $[\theta_{\text{min}}, \theta_{\text{max}}]$, the child part is rotated through this angular range at discrete steps. At each step, the collision volume between the child part and all other static parts is computed — this is the volume of 3D space where the meshes intersect. For prismatic joints, the child part is translated along the joint axis, and the same collision volume computation is performed.

2. Derivative-based spike detection: The collision volume as a function of joint angle $V(\theta)$ is computed. Significant collisions manifest as sharp increases in this volume — when two parts first make contact, the intersection volume jumps from near-zero to a positive value. The paper identifies these contact events by computing the derivative of $V(\theta)$ with respect to $\theta$ (numerically, using finite differences between adjacent steps). Sharp spikes in this derivative indicate the onset of collision.

3. Coarse window identification: A coarse angular window containing the derivative spike is identified. The granularity at this stage is not specified, but it's designed to bracket the collision onset region.

4. Hierarchical search for precise contact angle: Within the identified window, a hierarchical (binary-search-like) search is performed to pinpoint the precise angle at which collision first occurs. The search progressively refines the angle estimate until the contact point is located to within a tolerance small enough for practical simulation (likely sub-degree for rotational joints).

5. Limit adjustment: The identified contact angle becomes the new, refined joint limit. If the collision occurs near the minimum angle, $\theta_{\text{min}}$ is adjusted; if near the maximum, $\theta_{\text{max}}$ is adjusted. If collisions occur at both ends, both limits are adjusted. The same procedure is applied to prismatic joints based on translational distance along the sliding axis.

What this correction does NOT do: it only tightens limits — it never expands them. If ArtLLM predicts a limit of 90° but the part can actually open to 120° without collision, the correction won't discover this because there's no collision signal to trigger expansion. The correction is therefore a conservative refinement that ensures safety (no collisions) at the potential cost of slightly restricting the motion range. This is appropriate for simulation and robotics applications where unexpected collisions are worse than slightly conservative limits.

Why derivative-based detection rather than threshold-based: a simple threshold on collision volume (e.g., "contact occurs when $V > \epsilon$") is sensitive to the choice of $\epsilon$ and the mesh resolution. Small $\epsilon$ values may detect spurious contacts from mesh discretization artifacts; large values may miss genuine contacts. The derivative-based approach is more robust because it looks for changes in collision volume — a genuine contact onset produces a large, sustained increase in the derivative, while mesh artifacts produce small, transient fluctuations.

Physical plausibility guarantee: after this correction, the articulated asset is guaranteed to be collision-free throughout its entire range of motion (for the corrected limits). This makes the output directly usable in physics simulators like SAPIEN, MuJoCo, or Isaac Sim, where interpenetrating meshes cause simulation instabilities or unrealistic behavior.

Computational cost: the paper does not specify the computational overhead of this correction step. However, since it operates on the final generated meshes (not point clouds) and performs collision detection along a single degree of freedom per joint, it should be fast — seconds per joint at most — and is applied only at inference time, not during training. This is substantially cheaper than the per-object joint-and-geometry co-optimization used in reconstruction-based methods, which must iteratively update both joint parameters and neural scene representations.

Why a post-hoc correction rather than an end-to-end trainable component: incorporating collision awareness directly into the LLM's training would require differentiable collision detection and physics simulation, which is extremely challenging for complex meshes and not feasible with current LLM training pipelines. The post-hoc correction separates the learning problem (predicting reasonable initial limits from geometry) from the physics problem (ensuring those limits are collision-free), allowing each to be solved with the most appropriate tools (language models for the former, computational geometry for the latter).

4. Key Insights and Innovations

Innovation 1: Articulation Prediction as Language Modeling via Comprehensive Quantization

The dominant assumption in prior LLM-based articulation work—exemplified by Real2Code (Mandi et al., 2024) and Li et al. (2025)—was that LLMs could directly regress continuous floating-point values for joint parameters. This assumption was fundamentally flawed: LLMs are token predictors optimized for discrete vocabularies, not function approximators with smoothness inductive biases. Small token prediction errors in axis directions compound geometrically—a 1° error at the joint origin translates to centimeters of displacement at a part's far end.

ArtLLM's central conceptual move is not merely "let's use an LLM for articulation" but rather "articulation prediction is a classification problem over a discretized geometric vocabulary, not a regression problem over continuous coordinates." This reframing converts every continuous parameter—bounding box coordinates, joint origins, axis directions, rotation limits, translation limits—into discrete tokens drawn from purpose-designed vocabularies: 128 bins for spatial coordinates, 48 bins for rotation angles, 64 bins for translation distances, and a 128-entry hierarchical axis codebook.

The hierarchical axis codebook deserves particular attention as a structural prior encoded directly into the token vocabulary. The paper observes that most real-world joint axes align with coordinate axes—doors swing on vertical axes, drawers slide horizontally. Rather than treating all directions uniformly (which would waste representational capacity on orientations that almost never occur), the codebook densely samples from the XY, YZ, and XZ planes first, then fills remaining coverage via Farthest Point Sampling on a Fibonacci sphere. This design says: "allocate vocabulary capacity proportional to the expected distribution of joint axes in the world." It's a form of architectural inductive bias implemented at the tokenization level rather than the model architecture level—a design pattern with broader implications for any structured prediction task where LLMs interface with continuous geometric quantities.

The significance extends beyond articulation. This quantization strategy is a general recipe for having language models predict 3D structures: identify the continuous parameters, determine the required precision for each, design discrete vocabularies that concentrate capacity where the data distribution is dense, and let the LLM's autoregressive classification machinery handle the rest. The ablation (Table 3, Experiment A) provides the critical evidence: removing quantization and predicting continuous values directly "significantly weakens the model's ability to infer coordinate and direction-related attributes." This is not an incremental improvement over prior LLM-based methods—it's a fundamental repair of a broken modeling assumption that explains why those methods exhibited poor generalization with limited training data. The quantization strategy makes LLM-based articulation prediction work at all; without it, the entire framework collapses.

Innovation 2: Decoupling Structural Reasoning from Geometric Realization via a Tokenized Blueprint Interface

Prior work in articulated object generation conflated two distinct problems: figuring out what structure exists (parts and joints) and realizing what that structure looks like (geometry). Retrieval-based methods (SINGAPO, URDFormer, Articulate-Anything) tied these together by design—the structural prediction selected which parts to retrieve, and the geometry came along with them. Optimization-based methods (PARIS, ArtGS) co-optimized geometry and articulation jointly, making it impossible to improve one without the other. Even the part-level 3D foundation models (OmniPart, XPart) that could generate novel geometry had no mechanism for incorporating articulation constraints.

ArtLLM introduces a clean architectural interface between structural reasoning and geometric synthesis: the tokenized blueprint. The LLM outputs a structured language sequence specifying part bounding boxes and joint parameters; this sequence is then parsed and fed as conditioning to an off-the-shelf part generative model (XPart). The two components share no weights, no joint training, and no mutual gradients.

This is a systems-level architectural innovation rather than an algorithmic one. The insight is that articulation prediction and part geometry generation are causally asymmetric: knowing the articulation structure strongly constrains what plausible part geometries look like (a hinge joint implies complementary cylindrical geometry at the pivot), but knowing the geometry only weakly constrains the articulation (a cylindrical interface could be a hinge, a sliding joint, or a fixed connection). By placing articulation prediction before geometry synthesis, ArtLLM exploits the stronger causal direction. The alternative ordering—generate geometry first, then predict joints from it—would face the harder inference problem with no compensating advantage.

The practical consequence is modular upgradeability. As part generation models improve—and this field is moving fast, with Trellis, Hunyuan3D, Clay, and TripoSG all advancing rapidly—ArtLLM's output quality improves without retraining the articulation model. The blueprint serves as a stable contract between components. This is the same design philosophy that made CLIP a foundational component in text-to-image systems: separate the semantic understanding from the pixel generation, and let each advance independently.

The bounding box expansion mechanism (Section 3.3) supports this decoupling by ensuring that even imperfect layout predictions don't propagate errors that would truncate generated geometry. The expansion assigns uncovered points to their nearest box and grows each box to enclose its assigned points—a lightweight geometric fix that compensates for the inevitable imprecision of predicted bounding boxes without requiring the LLM to achieve pixel-perfect localization. This is a robustness mechanism that acknowledges the inherent uncertainty in the blueprint while preventing that uncertainty from degrading the final output.

Evidence for the effectiveness of this decoupling appears not in an ablation (since the architecture is monolithic—there's no "coupled" variant to compare against) but in the qualitative results (Figure 4) and the real-world generalization (Figure 6). The generated assets show part geometries that match the input appearance while respecting articulation constraints—a combination that neither purely retrieval-based nor purely generation-based prior methods achieved.

Innovation 3: Difficulty-Aware Curriculum Training for Multimodal Structured Prediction

Training a model to jointly predict part layouts and kinematic structures from point clouds is a hard optimization problem: the model must simultaneously learn to perceive part boundaries (a geometric task) and reason about mechanical connectivity (a relational task), all while the encoder and language model are co-adapting from different initializations. The paper's multi-stage, multi-task training strategy embodies an insight that goes beyond standard multi-task learning: geometric perception and kinematic reasoning should be learned sequentially, with the simpler task providing a stable foundation for the harder one.

Prior work treated articulation prediction as a monolithic end-to-end prediction problem—the model received a point cloud or image and was trained to output parts and joints jointly from the start. This approach conflates the learning dynamics of two tasks with fundamentally different difficulty profiles. Part layout prediction (identifying bounding boxes from point clouds) is a well-studied problem with strong pretrained models available (the paper uses P3SAM, pretrained on large-scale part segmentation). Joint prediction (inferring mechanical connectivity from geometric features) is substantially harder and has no comparable pretraining foundation.

The two-stage curriculum implements a progressive disclosure of task complexity:

1. Stage 1 trains only on part layout prediction, initializing the point encoder from P3SAM weights. This grounds the encoder in part-level feature learning without the confounding gradients from joint prediction. The encoder learns to produce features that cleanly separate different functional parts—exactly the representation needed as input to kinematic reasoning.

2. Stage 2 adds joint prediction and end-to-end prediction as auxiliary tasks, now starting from an encoder that already produces high-quality part features. The LLM can focus its learning capacity on the harder relational reasoning because the geometry perception is already handled.

The ablation in Table 3 (Experiment D) provides the confirmatory evidence: removing multi-stage training "lowers both part and joint prediction accuracy." The interpretation is that when both objectives are trained from scratch, the gradients from joint prediction early in training—when part features are still poor—destabilize the encoder's learning. The curriculum effectively orthogonalizes the learning dynamics so that each stage can converge without interference.

This is a training methodology innovation rather than an architectural one, but it has broader implications for multimodal structured prediction. The pattern—pretrain on the simpler subtask with strong initialization, then fine-tune on the joint task with the simpler task as auxiliary supervision—is applicable to any domain where a model must learn both perception and relational reasoning. The 3:2:5 mixing ratio in Stage 2 (30% part layout, 20% kinematics given ground-truth parts, 50% end-to-end) weights the target deployment task most heavily while using the auxiliary tasks to maintain and refine the representations learned in Stage 1.

The connection to the paper's overall contribution is that this curriculum is what makes the joint prediction work at all given the modest dataset size (20,673 objects). Without Stage 1, the optimization likely either converges to a poor local minimum (good at part detection, bad at joints) or fails to converge reliably. The curriculum reduces the effective sample complexity of the harder task by providing a strong initialization.

Innovation 4: Physics-Based Joint Limit Correction as a Necessary Post-Hoc Complement to Learned Prediction

The paper identifies a fundamental limitation that is inherent to the problem formulation, not a weakness of any specific method: a model that sees an articulated object in a single static pose cannot uniquely determine its joint limits. Two different objects—one with a drawer that opens 30 cm and another with the same drawer opening 60 cm—can be indistinguishable in any single configuration if both are observed at 20 cm of extension. This is not a training data issue or an architecture issue; it's an information-theoretic limitation.

Prior methods handled this limitation poorly. Optimization-based reconstruction methods could potentially recover limits by observing motion across multiple video frames, but this requires dynamic input that isn't always available and adds computational cost. Retrieval-based methods sidestepped the issue by using fixed limit ranges from the database, which are often wrong for novel objects. LLM-based methods attempted to predict limits directly from static geometry, with predictably poor results—the model had no signal about where motion would stop beyond what it could memorize from training data correlations between geometry and limits.

ArtLLM's insight is that joint limits should not be treated as a pure prediction problem—they should be treated as a constrained optimization problem where the constraint (no self-collision) provides the missing information. The post-processing correction step (Section 3.4, Figure 3) uses collision detection as an oracle: articulate the part through the predicted range, detect where it first contacts other parts, and set the refined limit to that contact point.

This is a diagnostic innovation: it identifies the precise sub-problem where learned prediction is insufficient and replaces it with algorithmic reasoning. The learned model provides an initial estimate—good enough to get the joint type, axis, and origin approximately correct—and the physics module refines the limits to be collision-free. Neither component alone would suffice: pure prediction can't guarantee collision freedom, and pure collision detection can't infer which parts are connected by which joints or what the joint axes are.

The collision detection algorithm itself is straightforward (derivative-based spike detection followed by hierarchical search), but the conceptual contribution is the boundary between learned and algorithmic components. The paper draws this boundary at the point where additional learning would require observing the object in motion—data that isn't available in the single-image or single-mesh input setting. By instead using physics simulation (collision detection) as a form of test-time optimization, the system recovers limits that are guaranteed to produce physically plausible motion without requiring dynamic training data.

Qualitative evidence appears in Figure 5: cases where predicted limits cause visible self-collisions are corrected to produce smooth, collision-free motion. The real2sim experiments in Figure 6 provide the ultimate validation—replaying real robot trajectories on simulated assets succeeds, which would fail if joint limits were incorrect (the simulated object wouldn't open far enough, or would interpenetrate itself during motion). This is a pragmatic solution to a fundamental information-theoretic bottleneck, and it's likely the right level at which to solve the problem given current technology—attempting to learn limits from static geometry alone is doomed to high error rates regardless of model capacity or training data scale.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The evaluation uses the PartNet-Mobility dataset (Xiang et al., 2020), specifically the data split proposed by SINGAPO (Liu et al., 2024): 77 objects across 7 categories (Storage, Table, Refrigerator, Dishwasher, Oven, Washer, and Microwave). These 77 objects are held out from all training data across all sources (PartNet-Mobility, PhysX3D, and Infinite-Mobility procedurally generated assets), constituting a strict generalization test. Additionally, the paper tests on real-world images including objects outside these seven categories to assess out-of-distribution generalization.

• Base model. The core articulation prediction model is ArtLLM, built on Point Transformer v3 (Wu et al., 2024) as the 3D encoder and Qwen3 0.6B (Yang et al., 2025) as the language model backbone. The encoder is initialized from P3SAM (Ma et al., 2025), a model pretrained on large-scale part segmentation. For geometry generation, the paper adopts XPart (Yan et al., 2025) as the part-aware generative model. For upstream 3D mesh generation from images, Hunyuan3D 3.0 (Lai et al., 2025) is used.

• Metrics. The paper evaluates along two axes: part layout quality and articulation accuracy. For part layout, the primary metric is mean Intersection over Union (mIoU) between predicted and ground-truth part bounding boxes, computed after Hungarian matching aligns predicted and ground-truth parts based on center distances. For articulation, four metrics are used: (1) joint type accuracy — the fraction of matched joint pairs with correct type classification (revolute, continuous, prismatic, or screw); (2) joint axis error — the minimum angle (in degrees) between predicted and ground-truth axis directions, accounting for sign equivalence by testing both the original and reversed directions; (3) joint pivot error (referred to as "minimal distance between axis" in the paper) — computed as $|\mathbf{p} \cdot (\mathbf{a}_p \times \mathbf{a}_g)| / |\mathbf{a}_p \times \mathbf{a}_g|$, where $\mathbf{p}$ is the vector difference between predicted and ground-truth joint origins and $\mathbf{a}_p, \mathbf{a}_g$ are the axis directions, measuring the positional discrepancy orthogonal to the axis; (4) joint range IoU — the intersection-over-union of predicted and ground-truth motion ranges, with the maximum taken over both original and reversed range directions to account for sign ambiguity. Additionally, graph accuracy evaluates the kinematic hierarchy by checking whether the directed graph formed by parts and joints is isomorphic to the ground-truth graph. For all metrics, per-object scores are averaged within each category, then averaged across all seven categories.

• Baselines. The paper compares against three state-of-the-art methods: (1) URDFormer (Chen et al., 2024), a retrieval-based method that predicts part layouts and joint parameters but assumes a fixed external-frame-plus-internal-components structure — evaluation is restricted to the five categories URDFormer was trained on, as it cannot handle the remaining two; (2) SINGAPO (Liu et al., 2024), a feedforward network that predicts part layouts and retrieves geometries from a fixed database; (3) Articulate-Anything (Le et al., 2024), evaluated using the GPT-4o API, which retrieves parts from a pre-built library. For fair comparison, the ground-truth object parts are removed from the retrieval library for both SINGAPO and Articulate-Anything, ensuring they cannot simply retrieve the correct parts. For all retrieval-based methods, fixed joints connecting handles to parent parts are removed and the corresponding links merged to match ArtLLM's simplification procedure.

• Generation budget / compute accounting. The paper measures inference time as a practical compute metric, reporting that ArtLLM completes in "seconds" compared to minutes for the baseline methods (Table 2, "Time" column). The specific timing numbers in Table 2 show ArtLLM at the fastest inference time across all categories, though exact absolute values depend on hardware (all methods evaluated on identical hardware: an Ubuntu server with 2 Intel Xeon Silver 4210 CPUs and a single NVIDIA RTX 3090 GPU with 24 GB VRAM). For training compute, the paper reports 8 H20 GPUs for 50 epochs (Stage 1, ~8 hours) plus 30 epochs (Stage 2, ~15 hours).

• Cross-validation / statistical protocol. No cross-validation is reported. The evaluation uses a fixed test split (SINGAPO's 77-object split) evaluated once. There are no confidence intervals, standard deviations, or statistical significance tests reported for any metric. Results in Table 2 are reported as single-point estimates per category, then averaged.

Main Quantitative Results

Aggregate Comparison Against Baselines

The headline result from Table 2 is that ArtLLM achieves a part layout mIoU of 83.0% and a joint type accuracy of 69.3%, substantially outperforming all three baselines on the majority of metrics. Breaking this down:

• Part layout (mIoU): ArtLLM at 83.0% vs. SINGAPO at 65.5%, URDFormer at 54.8% (restricted to 5 categories), and Articulate-Anything at 81.8%. ArtLLM leads, though Articulate-Anything is competitive on part localization. The gap between ArtLLM and SINGAPO (17.5 percentage points) reflects the fundamental limitation of retrieval-based part prediction: when the predicted bounding box doesn't match any database part's natural extent, the retrieved geometry is forced into a mismatched spatial configuration.

• Joint type accuracy: ArtLLM at 69.3% vs. Articulate-Anything at 67.1%, SINGAPO at 40.0%, and URDFormer at 20.0% (5 categories). SINGAPO and URDFormer show dramatically lower joint type accuracy — they frequently misclassify revolute joints as prismatic or vice versa. Articulate-Anything is competitive, which the paper attributes to GPT-4o's strong reasoning capabilities, though this comes at the cost of API dependency and slower inference.

• Joint axis error (degrees): ArtLLM at 12.64° vs. SINGAPO at 14.06°, URDFormer at 37.48° (5 categories), and Articulate-Anything at 31.86°. ArtLLM achieves the lowest axis error, with a particularly large margin over Articulate-Anything (19.22°). The paper notes that Articulate-Anything "often misidentifies the axis direction, producing incorrect motion types" (Section 4.3), and the nearly 32° average axis error — almost a third of a right angle — confirms this qualitatively.

• Joint pivot error (axis distance): ArtLLM at 0.0248 vs. SINGAPO at 0.0727, URDFormer at 0.0396 (5 categories), and Articulate-Anything at 0.0700. ArtLLM achieves roughly 3× lower pivot error than the next-best method. This is a critical metric for functional correctness: a pivot error of 0.07 in normalized coordinates on a 1-meter object corresponds to a 7 cm offset — enough to make a door visibly misaligned or cause simulation failures.

• Joint range IoU: ArtLLM at 31.8% vs. Articulate-Anything at 73.4%, SINGAPO at 39.4%, and URDFormer at 34.1% (5 categories). This is the one metric where ArtLLM is not best — Articulate-Anything substantially outperforms, which the paper attributes to its "rule-based limit prediction" (Section 4.3). Articulate-Anything uses heuristic rules to set joint limits based on part geometry, which proves more accurate than ArtLLM's learned limit prediction from static geometry. SINGAPO also slightly outperforms ArtLLM on this metric (39.4% vs. 31.8%).

• Graph accuracy: ArtLLM at 39.7% vs. SINGAPO at 23.0%, URDFormer at 3.1% (5 categories), and Articulate-Anything at 34.5%. The kinematic hierarchy graph (which part connects to which, forming the tree structure of the articulated object) is recovered correctly roughly 40% of the time by ArtLLM — far from perfect but substantially better than baselines. URDFormer's 3.1% reflects its fixed structural assumption collapsing on diverse topologies.

• Inference time: ArtLLM reports the fastest time (exact seconds not specified in the main text, but Table 2's Time column shows ArtLLM uniformly lowest across all categories). The paper claims "roughly 2–3 orders of magnitude" faster than baselines, though from Table 2 the margin appears smaller (e.g., SINGAPO is roughly 2–5× slower across categories, not 2–3 orders of magnitude). The key comparison is against optimization-based methods (not in Table 2) that the paper cites as requiring "minutes to hours."

Figure 4 (qualitative comparison) visually reinforces these results: URDFormer produces geometries that "differ significantly from the ground truth appearance" and fails to predict correct part counts and rotation directions; SINGAPO retrieves approximately correct geometries but predicts inaccurate part scales and axis positions, causing misalignment; Articulate-Anything recovers near-identical geometry from its database but shows substantial axis localization errors. On real-world examples (bottom rows of Figure 4), URDFormer remains limited, Articulate-Anything continues to show axis errors, and SINGAPO "fails to retrieve accurate geometry and often misses essential parts." ArtLLM's outputs "closely match the input images" with correct articulation.

Per-Category Breakdown

The more detailed per-category results in Table 4 (Appendix, Section 11.3) reveal substantial category-dependent variation:

• Storage (10 objects): ArtLLM achieves 82.0% mIoU, 62.0% joint type accuracy, 9.57° axis error — strong across the board compared to baselines.

• Table (8 objects): ArtLLM achieves 87.3% mIoU, 75.0% joint type accuracy, 6.64° axis error — among the best category-level results, likely because tables have simple, predictable kinematic structures (typically a revolute joint for a folding mechanism or prismatic joints for drawers).

• Refrigerator (11 objects): ArtLLM achieves 79.1% mIoU, 81.8% joint type accuracy, 11.48° axis error — high joint type accuracy suggesting refrigerator articulations (door hinges) are relatively stereotyped.

• Dishwasher (8 objects): ArtLLM achieves 86.1% mIoU, 62.5% joint type accuracy — strong part layout but joint type classification is notably lower, possibly because dishwasher racks are sometimes prismatic (sliding) and sometimes not, creating ambiguity.

• Oven (9 objects): ArtLLM achieves 83.2% mIoU, 62.5% joint type accuracy — similar pattern to dishwasher.

• Washer (11 objects): ArtLLM achieves 81.2% mIoU, 72.7% joint type accuracy, 18.59° axis error — notably higher axis error than other categories.

• Microwave (20 objects): ArtLLM achieves 79.1% mIoU, 80.0% joint type accuracy, 20.15° axis error — the highest axis error of all categories. Microwaves often have doors with buttons on the front panel, creating part ambiguity: should the front panel be one part (door + button panel) or two? This ambiguity propagates to joint axis prediction.

For comparison, SINGAPO shows particularly poor performance on Refrigerator (31.3% mIoU), Dishwasher (49.8% mIoU), and Microwave (64.9% mIoU) — categories where the retrieval database likely has poor coverage. URDFormer, restricted to 5 categories, shows 0% graph accuracy on Oven and Microwave (the two categories it was evaluated on outside its original training categories), confirming its structural assumption fails on diverse topologies. Articulate-Anything shows competitive mIoU across categories (72–87%) but large axis errors, particularly on Oven (48.68°) and Washer (49.22°), where axis prediction essentially fails.

Real-World Generalization and Robotics Application

Beyond the PartNet-Mobility test set, the paper demonstrates generalization through two qualitative evaluations:

Real-world image reconstruction (Figure 4, bottom rows): ArtLLM processes real-world images (via Hunyuan3D 3.0 for mesh generation, then ArtLLM for articulation) and produces articulated assets with "geometry that closely matches the input." In contrast, URDFormer "still produces highly limited results" on real examples; Articulate-Anything can reconstruct almost identical geometry (GPT-4o identifies visual matches in its database) but "continues to suffer from substantial errors in axis localization"; and SINGAPO "fails to retrieve accurate geometry and often misses essential parts." No quantitative metrics are reported for real-world images — this is purely qualitative.

Real2sim robotics evaluation (Figure 6): The paper teleoperates a Franka Panda robot to perform three tasks — closing a laptop, closing a box, and moving a bucket handle — records the full pose sequence, then uses ArtLLM to reconstruct the articulated objects from video frames (using Hunyuan3D 3.0 for mesh generation). The reconstructed assets are placed in the SAPIEN simulator, and the recorded pose trajectories are replayed. The paper reports that "all three tasks were successfully reproduced in the simulator," with the simulated objects exhibiting the same articulation behavior as the real ones. This is an existence proof that ArtLLM's outputs are functional enough for robot learning, but no quantitative metrics (reproduction error, trajectory deviation, success rate) are reported — the evaluation is binary (works / doesn't work) with N=3.

Ablation Studies and Robustness Checks

Table 3 reports four ablation experiments, each training on PartNet-Mobility only (not the full curated dataset) for 30 epochs and evaluating on 144 objects (2 per category across all PartNet-Mobility categories, not the 7-category subset). The reduced training data and evaluation set mean absolute numbers aren't comparable to Table 2 — the focus is on relative differences between configurations.

A. Removing quantization and predicting continuous values directly: This "significantly weakens the model's ability to infer coordinate and direction-related attributes." Specifically, part mIoU drops (exact numbers: ArtLLM full model achieves some baseline mIoU on this reduced split; the continuous-prediction variant achieves lower — the paper states the finding qualitatively but Table 3 provides the quantitative comparison, though exact values are not quoted in the main text). The joint axis error increases when predicting continuous values directly. This ablation validates the central design decision: quantization is not an optimization but a necessity for stable LLM-based continuous parameter prediction. The likely failure mode is that the LLM's token-level cross-entropy loss provides no gradient signal about geometric consistency — predicting "0.42" vs "0.43" for a joint origin produces virtually identical loss if both are equally close to the ground-truth in token space, but in geometric space these represent different positions that compound across predictions.

B. Removing the multi-task setup (no auxiliary Tasks 1 and 2): Training purely on end-to-end prediction (Task 3 only) "slightly improves axis-direction prediction but degrades all other metrics." Table 3 shows that joint axis error slightly decreases (improves) without multi-task learning, but part mIoU and joint type accuracy degrade. This is a non-obvious tradeoff: the auxiliary tasks — particularly Task 1 (part layout only) and Task 2 (joints given ground-truth parts) — improve overall performance but create a slight interference with axis prediction specifically. The paper's interpretation is that "multi-task learning with varied difficulty reinforces part and articulation understanding," but the axis-prediction degradation suggests the shared encoder representation optimizes for a compromise across tasks that slightly penalizes pure direction estimation.

C. Removing random scaling and rotation augmentation: This leads to "lower part-IoU performance," confirming that "3D augmentation enhances spatial perception of part position and scale." The effect is most pronounced on part mIoU rather than articulation metrics, which makes sense: the augmentation directly impacts the encoder's invariance to orientation and scale, which feeds into part localization. Joint prediction, which operates on relative spatial relationships between parts, is less affected by global transformations since those relationships are preserved under rotation and scaling (assuming the model learns relative rather than absolute coordinate reasoning).

D. Removing multi-stage training (no Stage 1 pretraining on part layout): Using P3SAM-initialized point encoders directly for end-to-end training (all three tasks from the start, without Stage 1) "lowers both part and joint prediction accuracy." This ablation confirms that the curriculum matters: Stage 1's exclusive focus on part layout prediction provides a superior encoder initialization compared to starting with the full multi-task objective. The proposed mechanism is that joint prediction gradients early in training (when part features are poor) destabilize encoder learning. The ablation provides evidence that decoupling geometry and kinematics during early training is beneficial, but it doesn't distinguish between two possible explanations: (a) the encoder benefits from task-specific pretraining regardless of what that task is, or (b) part layout is specifically the right pretraining task because it provides the geometric foundation kinematics requires. An additional ablation — pretraining on a different task (e.g., joint prediction from ground-truth parts) and then fine-tuning on end-to-end — would disambiguate these, but is not reported.

Physics-based limit correction (qualitative ablation, Figure 5): The paper shows qualitative examples where predicted joint limits cause self-collisions, and the correction step adjusts the limits to produce collision-free motion. This is not a controlled ablation (no quantitative metric like "fraction of objects with post-correction collisions" or "average limit adjustment magnitude"), but the visual evidence in Figure 5 is clear: cases where the initial prediction produces interpenetrating meshes are visibly corrected. The real2sim experiments (Figure 6) provide indirect validation — if limits were incorrect, the replayed trajectories would fail — though N=3 is insufficient to quantify reliability.

Missing ablation — number of quantization bins: The paper does not ablate the specific bin counts (128 for coordinates, 48 for rotation limits, 64 for translation limits, 128 for axis codebook). These choices are motivated geometrically (128 bins gives ~1.56 cm resolution for a 1 m object; 48 rotation bins gives 15° resolution; 64 translation bins gives ~3 cm resolution for a 1 m object), but the sensitivity to these choices is untested. It's possible that coarser quantization (e.g., 64 bins for coordinates) would be sufficient and would reduce vocabulary size, or that finer quantization would improve precision without hurting training stability.

Missing ablation — axis codebook design: The hierarchical codebook (XY/YZ/XZ plane sampling + Fibonacci sphere FPS) is compared only implicitly to the alternative of direct continuous prediction (Ablation A), not to alternative codebook designs (e.g., purely uniform sphere sampling with 128 entries, or a learned codebook). The claim that "most joint axes align with coordinate axes" motivates the design, but the ablation doesn't quantify how much the hierarchical sampling improves axis error over a uniform baseline.

Missing ablation — LLM backbone scale: The paper uses Qwen3 0.6B. No comparison with larger (e.g., 1.8B, 4B) or smaller LLM backbones is reported, leaving open the question of whether the 0.6B scale is optimal or merely sufficient. Given the constrained vocabulary and task complexity, it's plausible that a smaller model would perform similarly, or that a larger one would improve joint prediction accuracy.

Missing ablation — point cloud sampling density: The paper uses 32,768 points uniformly. No ablation varying the point count is reported, so the sensitivity of articulation prediction to input resolution is unknown.

Critical Assessment

Does ArtLLM actually "significantly outperform" state-of-the-art methods?

The Table 2 results support this claim for part layout mIoU (83.0% vs. next-best 81.8%), joint type accuracy (69.3% vs. 67.1%), joint axis error (12.64° vs. 14.06°), and joint pivot error (0.0248 vs. 0.0396). However, the comparison has important qualifications:

Articulate-Anything outperforms ArtLLM on joint range IoU by a massive margin (73.4% vs. 31.8%). The paper acknowledges this (Section 4.3) but the framing as "significant outperformance" is somewhat misleading — ArtLLM wins on 5 of 7 metrics, loses badly on one, and ties roughly on one. Joint range IoU is arguably the most practically important metric for simulation: an incorrect range means the object doesn't open far enough (or opens too far, causing collisions before the correction step). The 31.8% range IoU means ArtLLM's predicted limits have, on average, less than one-third overlap with the true range — a substantial error that the post-hoc correction only partially addresses (the correction tightens limits to avoid collision but doesn't expand them if they're too conservative).

The "2–3 orders of magnitude faster" claim is not supported by Table 2 data. The Time column in Table 2 shows ArtLLM faster than baselines, but the margin appears to be roughly 2–5×, not 100–1000×. The "orders of magnitude" claim may refer to comparison with optimization-based methods (PARIS, ArtGS) that are cited as requiring "minutes to hours" but aren't included in the quantitative comparison. This is a conflation: ArtLLM is compared against retrieval-based methods in Table 2 (which are also feedforward and fast), while the speed advantage is claimed relative to optimization-based methods that aren't in the table.

The test set is small (77 objects, 7 categories) and inherited from SINGAPO. The per-category breakdown in Table 4 shows as few as 8 objects per category (Table, Dishwasher). With N=8, a single failure can shift metrics by 12.5 percentage points. The paper reports no variance estimates, so we cannot assess whether the 83.0% vs. 81.8% mIoU difference between ArtLLM and Articulate-Anything is statistically significant or within sampling noise.

The baseline comparison has an asymmetry in input processing. ArtLLM uses Hunyuan3D 3.0 to generate meshes from images, then samples point clouds — a pipeline that has access to strong 3D generation priors. The baseline methods (URDFormer, SINGAPO, Articulate-Anything) operate on different input representations (SINGAPO on point clouds of ground-truth meshes; Articulate-Anything on images via GPT-4o; URDFormer on its specific input format). The paper states that "for a consistent evaluation protocol" they apply coordinate alignment and matching, but the input quality differences are not controlled. If Hunyuan3D 3.0 produces higher-quality geometry for certain categories than others, ArtLLM inherits that bias — a limitation acknowledged only implicitly.

Does the real-world generalization claim hold?

The real-world evaluation (Figure 4 bottom rows, Figure 6) is entirely qualitative with N=3 for the robotics experiment and unstated N for the real-image examples. "All three tasks were successfully reproduced" is an existence proof, not a systematic evaluation. The paper does not report: how many real-world objects were tested, how many succeeded vs. failed, what failure modes occurred, or whether the articulated assets required manual adjustment before simulation. The laptop-closing, box-closing, and bucket-handle tasks are all single-joint motions on relatively simple objects — they don't test the method's ability to handle multi-joint articulated objects in real-world settings.

The claim that ArtLLM "generalizes robustly to real-world objects" (Section 1) is therefore supported only anecdotally, not experimentally. A robust evaluation would require a real-world articulated object dataset with ground-truth URDF annotations, or at minimum a systematic assessment across dozens of objects with success/failure quantification and analysis of common failure modes.

Does the ablation convincingly establish the importance of each component?

The four ablations in Table 3 each test one component, but they have several weaknesses:

The ablations are trained on PartNet-Mobility only (not the full curated dataset) and evaluated on a different set (144 objects across all categories vs. the 77-object, 7-category test set). This makes the ablation results not directly comparable to the main Table 2 results, and it's unclear whether the ablation findings generalize to the full training setup. The reduced training data (PartNet-Mobility only, ~2,300 objects vs. 20,673) may exaggerate or suppress certain effects — for example, the benefit of data augmentation (Ablation C) may be larger with less training data.

The ablation metrics in Table 3 aren't discussed with specific numbers in the main text (Section 4.4 describes findings qualitatively — "significantly weakens," "degrades all other metrics," "slightly improves" — but doesn't quote the table values). This makes independent assessment difficult. The omission of specific numerical comparisons from the ablation discussion is a weakness in the paper's reporting.

Key ablations are missing (as noted above): quantization bin counts, axis codebook design alternatives, LLM backbone scale, point cloud sampling density. These are not peripheral — they're central to understanding whether the specific design choices are optimal or merely sufficient. The 128-bin coordinate quantization, in particular, represents a precision-vocabulary tradeoff that should be empirically justified.

Does the physics-based limit correction solve the joint limit problem?

The correction guarantees collision-free motion (by construction), but does not guarantee correct limits. If ArtLLM predicts a limit of 45° but the true limit is 90° (with no collision at any point in between), the correction leaves the limit at 45° — the object is collision-free but functionally restricted. The range IoU of 31.8% (uncorrected) suggests this is a common failure mode. The correction addresses the false positive problem (limits too large, causing collisions) but not the false negative problem (limits too small, restricting legitimate motion). For robotics applications, overly conservative limits are less problematic than collisions, but they still reduce the fidelity of the digital twin.

What experiments would have strengthened the paper?

1. A real-world articulated object benchmark with quantitative metrics. Even 20–30 real objects with manually annotated URDFs would substantially strengthen the generalization claims.

2. Ablation of quantization granularity. Testing 64, 128, and 256 bins for coordinate quantization would establish whether 128 is a sweet spot or merely adequate.

3. Comparison with optimization-based methods on the same test set. Since the paper claims speed advantages over PARIS/ArtGS, including them in Table 2 (with appropriate compute budget normalization) would make the comparison complete.

4. Sensitivity analysis to upstream mesh generation quality. Testing ArtLLM with different mesh generation backends (Hunyuan3D 2.5 vs. 3.0, TripoSG, etc.) would reveal how dependent articulation prediction is on input geometry quality — a practical concern since real-world deployment will use whatever generation model is available.

5. Failure mode quantification on the full pipeline. The paper shows failure cases in Figure 9 (4 examples), but doesn't report what fraction of the 77 test objects exhibit each failure mode (XPart failing to generate internal structures, part intersections, missing occluded geometry). A systematic failure taxonomy would be more informative than four cherry-picked examples.

Summary of what the experiments do and don't establish

What is convincingly established: On the PartNet-Mobility 7-category test split, ArtLLM's quantized language-model approach to articulation prediction produces more accurate part layouts, joint types, joint axes, and joint pivot positions than retrieval-based baselines, with substantially lower inference time. The ablation experiments confirm that quantization, multi-task training, data augmentation, and the two-stage curriculum all contribute positively to performance on this dataset.

What is not convincingly established: The real-world generalization claim rests on anecdotal qualitative evidence. The physics-based correction addresses collision avoidance but not limit accuracy. The specific design choices (bin counts, codebook design, model scale) are not empirically justified through ablation. The major remaining failure mode — joint range prediction — remains poorly solved with 31.8% range IoU, and the post-hoc correction only addresses one aspect of this (collision) while leaving conservative underestimation unaddressed.

What the experiments reveal about the method's practical readiness: ArtLLM is ready for applications where approximate articulation is acceptable and physical plausibility (collision-free motion) is more important than exact limit accuracy. This fits the paper's motivating use case — generating diverse articulated assets for robot training simulations, where a drawer that opens 40 cm instead of 50 cm is usable while a drawer whose parts interpenetrate is not. For applications requiring high-fidelity digital twins with precisely correct kinematics (e.g., replicating a specific real-world appliance for manipulation policy transfer), the 31.8% range IoU and unquantified real-world failure rate suggest ArtLLM is not yet reliable enough without manual verification and correction.

6. Limitations and Trade-offs

6.1 Joint Range Prediction Remains Fundamentally Poor — and the Post-Hoc Correction Only Mitigates One Half of the Problem

The assumption or constraint. The paper's post-hoc physics-based joint limit correction (Section 3.4) operates on a single principle: detect where self-collision first occurs and tighten the limit to that point. This addresses the case where predicted limits are too large (causing interpenetration), but it cannot fix the case where predicted limits are too small — if ArtLLM predicts a drawer opens 30 cm but the true limit is 60 cm, no collision signal exists to trigger expansion. The paper implicitly acknowledges this limitation in the correction algorithm's design — it only tightens, never loosens — but does not discuss the asymmetry explicitly.

The consequence. The joint range IoU of 31.8% (Table 2) — less than one-third overlap between predicted and ground-truth motion ranges — is the quantitatively weakest result in the paper, substantially trailing Articulate-Anything at 73.4%. This means that on average, ArtLLM's predicted joint limits miss approximately two-thirds of the true motion range. For simulation-to-reality transfer in robotics, this is problematic: a robot trained in simulation on a door that opens 45° will fail when deployed to a real door that opens 90°, because the learned policy never experienced the full motion envelope. The correction step guarantees collision-free motion but does not guarantee correct motion — it trades safety (no interpenetration) for fidelity (accurate range), and the paper provides no mechanism to recover from overly conservative initial predictions.

What evidence exists in the paper. Table 2 reports the 31.8% range IoU directly. Figure 5 shows qualitative evidence of the correction resolving collisions, but does not show cases where limits were already correct or already too small — the demonstration is selective. The real2sim experiments (Figure 6, N=3) succeed, but the three tasks (closing a laptop, closing a box, moving a bucket handle) involve motion toward a closed state where the limit is naturally constrained by the geometry (the laptop lid hits the keyboard base; the box lid hits the box rim). These are the cases where the correction works best — the collision signal is strong and unambiguous. Tasks requiring opening to a specific angle (opening a door to 90°, extending a drawer to its full length) are not tested.

Mitigation status. The paper does not address the underestimation problem. The physics-based correction operates only on the collision-detection principle, which cannot expand limits. The paper flags this implicitly by noting that Articulate-Anything's rule-based limit prediction (which uses part geometry heuristics rather than learned prediction) achieves dramatically higher range IoU (73.4% vs. 31.8%), suggesting that the learned prediction approach is fundamentally limited by the single-pose input constraint discussed in Innovation 4 (Section 4). The paper frames future work around "training on large-scale datasets with annotated physical attributes" (Section 5), which could potentially provide the missing signal for limit prediction, but this is a data acquisition problem — annotating accurate joint limits requires observing objects in motion or having CAD specifications — that the paper does not claim to have solved.

6.2 Real-World Generalization Is Claimed Based on Anecdotal Qualitative Evidence with No Quantitative Rigor

The assumption or constraint. The paper asserts that ArtLLM "generalizes robustly to real-world objects" (Section 1) and demonstrates its utility for "scalable robot learning" through digital twin construction. The real-world evaluation consists of qualitative examples in Figure 4 (bottom rows, unstated number of objects) and three teleoperated robot tasks in Figure 6 (laptop closing, box closing, bucket handle moving). There is no real-world articulated object dataset with ground-truth URDF annotations, no systematic evaluation across diverse real objects, and no reported failure rate or quantitative metrics for the real-world setting. The paper states in Section 5 that "the diversity of object categories is still limited. As a result, the model generalizes well to common household items but struggles with more complex categories such as vehicles or robots."

The consequence. A practitioner evaluating whether to deploy ArtLLM for real-world digital twin construction has no way to estimate the expected success rate. The three demonstrated tasks are all single-joint motions on geometrically simple objects where the articulation is fully visible from the canonical pose (a laptop hinge, a box lid, a bucket handle). The paper's own failure case analysis (Figure 9, Appendix Section 13) reveals that internal occluded structures (Case 103555, the bread rack) and overlapping parts (Case 103303) cause geometric failures that the articulation model cannot compensate for because the upstream mesh generation (Hunyuan3D 3.0) cannot reconstruct structures invisible from a single view. Real-world objects routinely have such occlusions — the internal mechanism of a reclining chair, the multiple hinge points of a foldable table, the nested sliding tracks of a multi-stage drawer — and the paper provides no evidence that ArtLLM handles these cases.

The dependence on Hunyuan3D 3.0 for upstream mesh generation further complicates generalization: real-world deployment quality is bounded by the mesh generation model's reconstruction fidelity, which varies with object material properties (reflective, transparent, or textureless surfaces are notoriously difficult for 3D reconstruction), viewpoint coverage, and lighting conditions. A failure in mesh generation propagates directly to articulation prediction, but the paper never isolates these two failure sources — when ArtLLM produces a poor articulation on a real object, it is impossible to determine from the reported results whether the articulation model or the mesh generation model is responsible.

What evidence exists in the paper. The real-world evaluation is entirely qualitative and small-scale (Section 4.5, Figure 6). The paper reports that "all three tasks were successfully reproduced in the simulator" and that the "simulated objects reproduce the real articulation behavior." No metrics comparable to Table 2 are reported for real objects. The failure case analysis (Figure 9, Appendix Section 13) is performed on the PartNet-Mobility test set, not on real-world objects, so the distribution of failure modes in real-world deployment is uncharacterized. The paper acknowledges the category diversity limitation in Section 5 but frames it as a matter of expanding to "more complex categories such as vehicles or robots" rather than as a gap in the evaluation methodology itself.

Mitigation status. The paper does not propose a quantitative real-world evaluation protocol, a real-world articulated object benchmark, or even a systematic manual evaluation. The acknowledgment in Section 5 — "Future work could incorporate open-vocabulary approaches like Kinematify to expand category coverage" — addresses the category diversity dimension but not the evaluation methodology dimension. A practitioner would need to conduct their own evaluation on their target object distribution before trusting the system for downstream tasks, and the paper provides no guidance on expected performance characteristics.

6.3 Difficulty Estimation Cost Is Unaccounted For in the Headline Efficiency Claims

The assumption or constraint. The paper reports in Table 2 that ArtLLM achieves inference times of "seconds" — substantially faster than the baseline methods evaluated (SINGAPO, Articulate-Anything, URDFormer). However, this timing accounts only for the ArtLLM forward pass (point cloud encoding + autoregressive token generation). It does not include the cost of upstream mesh generation from images or text (via Hunyuan3D 3.0), which is a prerequisite for any input modality other than an existing 3D mesh. For the dominant use case the paper motivates — generating articulated assets from a single image or text description — the full pipeline cost is Hunyuan3D 3.0 inference + ArtLLM inference + XPart part generation + physics-based limit correction.

The consequence. The "2–3 orders of magnitude" speed claim relative to optimization-based methods (Section 4.3) conflates the articulation prediction speed with the full asset generation speed. Furthermore, ArtLLM is compared against retrieval-based methods (SINGAPO, Articulate-Anything) in Table 2, but these methods do not include a mesh generation step — they operate on point clouds of existing geometry or retrieve from databases. The comparison is not cost-normalized: if a practitioner starts with an image, ArtLLM requires running Hunyuan3D 3.0 first, which adds inference cost and latency not reflected in the Table 2 timing numbers.

Even ignoring upstream generation, the physics-based limit correction (Section 3.4) incurs additional computation — collision detection along each joint's motion range with hierarchical search — that is not included in the reported ArtLLM inference time. For objects with many joints (up to 20 in the training data), this post-processing could become non-trivial, though the paper does not characterize its scaling behavior. The bounding box expansion step (Section 3.3), which iterates through all 32,768 input points to assign uncovered points to nearest boxes, also adds computational overhead not separately reported.

What evidence exists in the paper. The Time column in Table 2 provides inference-time comparisons but does not specify what is included and what is excluded. The paper describes the full pipeline in Section 3 (Figure 2) as including mesh generation, ArtLLM, part generation, and limit correction, but the timing numbers appear to cover only ArtLLM. There is no end-to-end latency measurement for the full pipeline starting from an image. The training compute is reported (8 H20 GPUs, ~23 hours total for both stages), but inference compute per object is not broken down by component.

Mitigation status. The paper does not address this directly. The speed comparison framing emphasizes ArtLLM's feedforward efficiency relative to per-object optimization, which is valid for the articulation prediction component in isolation, but the practical speed advantage for the full task (image → articulated URDF asset) depends on the cumulative cost of all pipeline stages. As 3D mesh generation models improve in speed — and this field is progressing rapidly — the upstream cost will decrease, but the paper's current headline numbers overstate the practical throughput advantage for the most common use case.

6.4 Training Data Scale and Diversity Are Moderate for a Generative Model, and Out-of-Distribution Behavior Is Uncharacterized

The assumption or constraint. ArtLLM is trained on a curated dataset of 20,673 articulated objects across 43 categories (Table 1), assembled from three sources: PartNet-Mobility (2,346 objects), PhysX3D (6,327 objects), and 12,000 procedurally generated assets from Infinite-Mobility. While this is larger than prior LLM-based articulation methods trained on, the paper acknowledges in Section 5 that "the diversity of object categories is still limited" and the model "struggles with more complex categories such as vehicles or robots." The test set of 77 objects across 7 categories (Section 4.1) is drawn exclusively from PartNet-Mobility — a dataset of household furniture and appliances — and does not test generalization to the procedural assets or to categories outside the training distribution.

The consequence. The model's behavior on objects with structural characteristics not represented in the training data is unknown. Vehicles (cars with doors, hoods, trunks; bicycles with chains and pedals; construction equipment with hydraulic arms), robots (multi-link manipulators with spherical joints; humanoid robots with dozens of degrees of freedom), and complex mechanisms (clocks, foldable furniture, multi-stage telescoping systems) all present articulation topologies that differ substantially from the primarily tree-structured, single-degree-of-freedom-per-joint objects in the training data. The paper's filtering step removes objects with more than 20 joints, deliberately excluding the most complex articulations, but doesn't characterize what performance would look like near that boundary (e.g., 15–20 joint objects).

More subtly, the training data is biased toward objects in canonical poses (partially open states). The paper's method operates on a single static point cloud, and the training data provides only one pose per object — typically the default pose from the dataset, which for PartNet-Mobility objects is often partially articulated (doors half-open, drawers partially extended). This means the model has never seen a fully closed or fully open version of most objects during training, potentially making it brittle to input poses that differ significantly from the canonical state. A closed laptop and an open laptop generate very different point clouds — the hinge geometry is occluded when closed — yet the articulation (the hinge axis and limits) is identical. The paper does not evaluate robustness to pose variation.

What evidence exists in the paper. The per-category breakdown in Table 4 shows substantial variation across the 7 test categories — microwave axis error is 20.15° while table axis error is 6.64° — suggesting category-dependent performance that likely extends to categories outside the test set. The failure cases in Figure 9 reveal structural failures (internal geometry not captured, part intersections) even within the test distribution, implying that more structurally divergent categories would exhibit higher failure rates. The paper does not report performance on the procedurally generated Infinite-Mobility assets (which have diverse, novel topologies not present in PartNet-Mobility) despite using 12,000 such assets in training — this would be a natural out-of-distribution generalization test within the curated dataset but is not performed.

Mitigation status. The paper proposes "open-vocabulary approaches like Kinematify to expand category coverage" as future work (Section 5), which would address the category diversity limitation if successful. However, this is a research direction, not an existing solution, and adapting a model trained on 43 categories to open-vocabulary articulation prediction would require substantially different training methodology (likely involving text-conditioned prediction rather than purely geometry-conditioned). The paper does not propose evaluating robustness to input pose variation, which is a more immediately addressable limitation — augmenting training data with multiple poses per object would be relatively straightforward but is not explored.

6.5 The Method Is Inherently Limited to Single-State, Single-Modal Geometric Input and Cannot Exploit Multi-View or Temporal Information

The assumption or constraint. ArtLLM operates on a single point cloud sampled from a watertight mesh, which represents the object in exactly one static pose. This is a fundamental architectural assumption — the transformer encoder processes a fixed set of 32,768 points with normals, and there is no mechanism for incorporating multiple views, video frames, or temporal observations of the object in motion. The paper's Section 3.1 specifies that for image or text inputs, an off-the-shelf 3D generation model (Hunyuan3D) first produces a complete mesh, which is then sampled into a point cloud. The model never sees the original multi-view images, the video, or any motion sequences.

The consequence. This single-state limitation creates an information-theoretic ceiling on what ArtLLM can learn about joint limits (as discussed in Limitation 6.1) and joint connectivity. Two parts that appear separate in the canonical pose could be rigidly connected through an internal mechanism invisible from the surface geometry — a single point cloud provides no signal to distinguish this case from a true kinematic joint. Conversely, two parts that visually merge in a particular pose (e.g., a sliding drawer that is fully closed, with no visible gap between the drawer front and the cabinet frame) may be incorrectly merged during part segmentation because the geometric boundary is absent.

More critically for real-world deployment, a user with a video of an articulated object — perhaps the most natural way to capture articulation information (opening a door, pulling a drawer, rotating a laptop screen) — cannot directly feed that temporal signal into ArtLLM. The rich information about joint axes (visible from the motion trajectory), joint limits (observable from the range of motion in the video), and part connectivity (apparent from which components move together) that video provides is discarded by the single-pose pipeline. The paper's optimization-based competitors (PARIS, VideoArtGS) explicitly exploit multi-view and temporal information, and while they are slower, they can recover articulation parameters that are fundamentally invisible in a single frame.

What evidence exists in the paper. The physics-based limit correction (Section 3.4) exists specifically because single-pose prediction cannot determine dynamic limits — this is the paper's own acknowledgment of the limitation. The failure case of the bread-rack structure (Case 103555, Figure 9) illustrates the single-view geometry problem: internal structures occluded from any single viewpoint cannot be reconstructed by Hunyuan3D, and ArtLLM consequently cannot articulate them. The paper does not systematically evaluate how often real articulated objects have articulation-relevant geometry occluded in a single canonical view.

Mitigation status. The paper partially mitigates the single-pose joint limit problem through physics-based correction (Section 3.4), but this addresses only collision avoidance — not limit accuracy, not joint axis refinement, and not part connectivity disambiguation. The paper does not propose a mechanism for incorporating multi-view or temporal inputs into the ArtLLM framework, which would require architectural changes (e.g., multi-frame point cloud encoding, temporal attention mechanisms) beyond the scope of the current work. For practitioners who have access to video or multi-view data, optimization-based methods may remain preferable despite their computational cost, because they can exploit information that ArtLLM's single-pose pipeline fundamentally cannot access.

6.6 The Evaluation Is Performed on a Small, Fixed Test Split Without Statistical Rigor, Making the Claimed Performance Advantages Difficult to Assess

The assumption or constraint. All quantitative results in Table 2 and Table 4 are computed on a single fixed test split of 77 objects across 7 categories, inherited from SINGAPO (Liu et al., 2024). The per-category sample sizes are as small as 8 objects (Table, Dishwasher). The paper reports no confidence intervals, no standard deviations, no statistical significance tests, and no cross-validation. The evaluation is performed once — there is no assessment of how sensitive the results are to the specific 77 objects chosen for the test set, or how the performance ranking between ArtLLM and the baselines might change with a different random split.

The consequence. The headline comparisons — ArtLLM at 83.0% mIoU vs. Articulate-Anything at 81.8%, ArtLLM at 69.3% joint type accuracy vs. Articulate-Anything at 67.1% — could fall within the sampling noise of a 77-object test set. With 8–20 objects per category, a single failure can shift metrics by 5–12.5 percentage points within a category, and those shifts propagate to the cross-category average. A practitioner cannot determine whether ArtLLM's advantage over Articulate-Anything on part mIoU and joint type accuracy is robust or would disappear with a different test split. The joint axis error (12.64° vs. 14.06° vs. 31.86° vs. 37.48°) has a larger relative margin, making it more likely to be robust, but the absence of variance estimates means even this is an assumption.

The small test set also means the difficulty distribution of the test objects is effectively uncontrolled. If the 77 SINGAPO-split objects happen to be easier or harder than the average PartNet-Mobility object on certain metrics — which is plausible given that the split was designed for SINGAPO's specific evaluation needs, not for ArtLLM — the reported performance could systematically over- or under-estimate true generalization performance. The paper does not characterize the test set's difficulty relative to the training distribution.

What evidence exists in the paper. Table 2 reports single-point estimates with no error bars or confidence intervals. The per-category breakdown in Table 4 (Appendix) reveals the small sample sizes and high per-category variance — for example, ArtLLM's joint range IoU varies from 17.0% (Oven) to 52.1% (Table) across categories, suggesting that the 31.8% average is heavily influenced by the specific categories and object counts in the test split. The ablation experiments (Table 3) use a different evaluation set (144 objects across all PartNet-Mobility categories, 2 per category), which makes them not directly comparable to the main results and does nothing to address the statistical rigor issue in the primary evaluation.

Mitigation status. The paper does not address this limitation. No statistical analysis is provided, no alternative test splits are evaluated, and no justification is given for the 77-object, 7-category split beyond that it follows SINGAPO's convention. The field of 3D articulated object generation lacks a standardized large-scale benchmark — PartNet-Mobility's test splits vary across papers — which makes cross-paper comparisons fragile in general, but the paper does not acknowledge this as a limitation of its evaluation methodology. A straightforward mitigation would be to report bootstrap confidence intervals on the metrics or to evaluate on multiple random splits and report mean and standard deviation. Neither is done.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper does not introduce a new algorithm for articulated object generation so much as it reframes the entire problem from a regression task to a structured language modeling task—and in doing so, it exposes a flawed assumption that had quietly constrained prior LLM-based approaches to articulation prediction. The flawed assumption was that LLMs could directly regress continuous floating-point values for joint axes, origins, and limits. Real2Code (Mandi et al., 2024) and Li et al. (2025) both attempted this, and both produced numerically unstable outputs with poor generalization. The field had attributed this to limited training data or insufficient model scale, but ArtLLM's ablation (Table 3, Experiment A) demonstrates a more fundamental issue: removing quantization causes the model's ability to infer coordinate and direction-related attributes to collapse, even when training data, model architecture, and compute are held constant. The problem was never the model's capacity—it was the mismatch between the continuous nature of geometric parameters and the discrete-token autoregressive objective used to train LLMs.

This reframing has two consequences that extend beyond articulation prediction specifically. First, it establishes comprehensive quantization as a general design pattern for any task where LLMs must interface with continuous 3D quantities. The paper's four-tier vocabulary—128 bins for spatial coordinates, 48 bins for rotation angles, 64 bins for translational distances, a 128-entry hierarchical axis codebook—is not a universal recipe (other domains will require different granularities and codebook designs), but the principle of converting continuous regression into discrete classification over purpose-built token vocabularies generalizes cleanly. Any problem where an LLM needs to predict camera poses, object trajectories, deformation fields, or material parameters from geometric input can adopt this pattern. The key design decisions—how many bins, what range, whether to use uniform or hierarchical discretization—become explicit hyperparameters to tune rather than implicit failure modes to discover empirically.

Second, the paper demonstrates that the order of operations in a generative pipeline matters causally, not just architecturally. By placing articulation prediction (the structural blueprint) before geometry synthesis (the part generation), ArtLLM exploits an information asymmetry: knowing the articulation strongly constrains what plausible part geometries look like, but knowing the geometry only weakly constrains the articulation. This is a systems-level insight rather than a novel algorithm—the blueprint serves as a clean interface between a reasoning module and a generation module, with neither sharing weights nor gradients—but it has immediate implications for how other multi-stage 3D generation pipelines should be designed. Scene generation, for example, could similarly benefit from predicting object layouts and spatial relationships before synthesizing individual object geometries, rather than generating objects independently and attempting to arrange them post-hoc.

The paper also provides the first empirical evidence that a language-model-based approach to articulation can match or exceed retrieval-based methods on geometric novelty while maintaining feedforward inference speed. Prior to ArtLLM, the dominant tension in articulated object generation was between quality and speed: optimization-based methods (PARIS, ArtGS) could produce accurate articulations but required minutes to hours per object; retrieval-based methods (SINGAPO, Articulate-Anything) were fast but could only produce geometries present in a fixed database. ArtLLM breaks this tradeoff by generating part layouts and synthesizing novel part geometries, achieving inference times of seconds while producing geometrically diverse outputs—the 83.0% part mIoU and 69.3% joint type accuracy on the PartNet-Mobility test set (Table 2) are achieved without any retrieval database, and the qualitative results in Figure 4 show geometries that match the input appearance rather than being database approximations.

This is not a paradigm shift in the sense that transformer-based 3D generation or LLM-based structured prediction replace all prior approaches—optimization-based methods will remain preferable when multi-view or video input is available and joint accuracy is paramount, and retrieval-based methods will remain useful when the target object is known to exist in a database. But it is a methodological reframing with practical consequences: the quantization strategy makes LLM-based articulation prediction work reliably for the first time, and the modular pipeline design (separating structural reasoning from geometric synthesis) creates an architecture where improvements to either component—better part generation models, better point cloud encoders, larger LLM backbones—compound without requiring retraining of the full system.

The paper also implicitly reconciles a tension in the literature about whether language models are appropriate for 3D structured prediction at all. The negative results from Real2Code and contemporaneous LLM-based URDF predictors had suggested that LLMs fundamentally lacked the inductive biases needed for geometric reasoning. ArtLLM shows that the problem was not the LLM but the representation of the prediction target: when continuous values are properly quantized into discrete tokens drawn from purpose-designed vocabularies, the LLM's autoregressive classification machinery handles the task naturally. This shifts the research question from "can LLMs do geometric reasoning?" (yes, with appropriate tokenization) to "what vocabulary designs enable LLMs to reason about which geometric structures?"—a more productive framing that focuses attention on the tokenization interface rather than the model architecture.

Follow-Up Research This Work Enables

1. Quantifying the precision-vocabulary tradeoff for geometric quantization. The paper uses 128 bins for coordinates, 48 for rotation limits, and 64 for translation limits, but never ablates these specific choices. A systematic study varying the bin counts—64, 128, 256, 512 for spatial coordinates; 24, 48, 96 for rotation limits—would characterize the relationship between quantization granularity and prediction accuracy. The hypothesis is that there exists a sweet spot: too few bins lose geometric precision and degrade joint axis and pivot accuracy; too many bins create a sparse classification problem where most bin indices appear rarely in training, destabilizing the LLM's token prediction. The experiment would train ArtLLM variants at each granularity on the same data and measure the resulting part mIoU, joint axis error, and joint pivot error, producing curves that reveal whether 128 bins is near-optimal or merely adequate. A particularly informative variant would test non-uniform quantization—denser bins near common joint positions (object centers, edges, corners) and sparser bins elsewhere—to determine whether the uniform binning used in the paper leaves precision on the table.

2. Ablating the hierarchical axis codebook against uniform and learned alternatives to determine whether the structural prior actually matters. The paper's axis codebook is built by sampling from the XY, YZ, and XZ planes first, then filling remaining coverage via Farthest Point Sampling on a Fibonacci sphere. The motivation—"most joint axes align with the coordinate axes"—is intuitively plausible but never empirically validated against alternatives. A direct comparison would train ArtLLM with three codebook designs: (a) the hierarchical codebook from the paper, (b) a purely uniform 128-point Fibonacci sphere sampling with no coordinate-plane bias, and (c) a learned codebook where the 128 axis directions are treated as trainable embeddings optimized jointly with the model. The comparison metric would be joint axis error (currently 12.64°), with the prediction that (a) outperforms (b) on common axis-aligned joints but (b) could outperform on off-axis joints, and that (c) might learn an intermediate distribution. A null result—uniform sampling matching hierarchical sampling—would indicate that the structural prior provides no benefit and that the model can learn the axis distribution from data alone.

3. Training an end-to-end version where the articulation blueprint and part geometries are generated jointly rather than sequentially, to test whether the causal asymmetry exploited by the paper's pipeline is necessary or merely convenient. The current system is strictly pipelined: ArtLLM predicts the blueprint, then XPart generates geometries conditioned on that blueprint. A joint model would train ArtLLM and a part generator end-to-end, with the part generation loss backpropagating through the blueprint tokens (using straight-through gradient estimation or a differentiable relaxation of the discrete quantization). The hypothesis is that end-to-end training would improve part geometry quality—the part generator could signal to the articulation model when predicted bounding boxes are slightly suboptimal for geometric synthesis—but might also degrade articulation accuracy if the part generation loss interferes with the blueprint prediction objective. The experiment would measure both articulation accuracy (mIoU, joint type accuracy, axis error) and part geometry quality (Chamfer distance to ground truth, visual fidelity) for the pipelined vs. jointly trained variants, revealing whether the modular decomposition is a necessary architectural choice or merely a training convenience that could be relaxed.

4. A systematic real-world articulated object benchmark with ground-truth URDF annotations to replace the current qualitative anecdotal evaluation. The paper's claim that ArtLLM "generalizes robustly to real-world objects" rests on N=3 robot tasks and an unstated number of qualitative examples in Figure 4. A proper evaluation would collect 50–100 real-world articulated objects spanning the categories ArtLLM was trained on (furniture, appliances, storage) plus out-of-distribution categories (vehicles, tools, toys), obtain ground-truth URDF annotations through manual measurement or CAD model matching, and evaluate ArtLLM end-to-end—from single image input through Hunyuan3D mesh generation, ArtLLM articulation prediction, XPart geometry synthesis, and physics-based limit correction—against retrieval baselines and optimization-based methods on the same objects. The metrics would be the same as Table 2 (mIoU, joint type accuracy, axis error, pivot error, range IoU, graph accuracy), with per-category breakdowns and failure mode classification (mesh generation failure vs. articulation prediction failure vs. part generation failure). This benchmark would simultaneously characterize the real-world generalization gap and identify which pipeline stage is the primary failure bottleneck.

5. Extending ArtLLM to consume multi-frame point clouds or video as input, to determine whether temporal information can close the joint limit prediction gap. The single largest quantitative weakness in ArtLLM is joint range IoU (31.8% vs. Articulate-Anything's 73.4%), and the paper attributes this to the information-theoretic limitation of single-pose input—you cannot uniquely determine joint limits from one static configuration. A natural extension would modify the input pipeline to accept multiple point clouds from different articulation states (e.g., a closed and partially open view of the same door), encoded either as separate point cloud tokens concatenated in the LLM input or via temporal cross-attention in the encoder. The experiment would test whether adding even 2–3 frames of articulation motion substantially improves range IoU, and whether the improvement saturates quickly (suggesting most of the missing information is in the first few frames) or continues to scale with frame count. This would also reveal whether the physics-based correction (Section 3.4) becomes unnecessary when temporal information is available—if multi-frame input enables accurate limit prediction, the collision-detection post-processing might be redundant.

6. Stress-testing ArtLLM on procedurally generated objects with systematically varied structural complexity to map the failure boundary as a function of part count, joint count, and topological depth. The training data includes 12,000 procedurally generated assets from Infinite-Mobility, but the evaluation uses only PartNet-Mobility objects with relatively simple kinematics (most have 2–6 parts and 1–5 joints). A diagnostic experiment would generate synthetic articulated objects with controlled complexity—varying part count from 2 to 20, joint count from 1 to 19, maximum tree depth from 1 to 10, and mixture of joint types—and evaluate ArtLLM's part mIoU, joint type accuracy, and graph accuracy as functions of each complexity dimension. The result would be a "capability envelope" showing, for example, that graph accuracy remains above 50% for objects with up to 8 joints but collapses to near-zero beyond 12, or that prismatic joint prediction degrades faster with part count than revolute joint prediction. This would provide practitioners with actionable guidance on which articulated objects ArtLLM can reliably handle and which require alternative approaches.

Practical Applications and Downstream Use Cases

1. Batch generation of diverse articulated assets for robotic manipulation training in simulation. The dominant bottleneck in scaling robot learning through simulation is the availability of diverse, functionally accurate training environments. A research group training a general-purpose kitchen manipulation policy needs hundreds of distinct articulated objects—cabinets with different door configurations, drawers with varying slide lengths, appliances with multiple interacting joints—to prevent the policy from overfitting to specific geometric instances. ArtLLM's pipeline takes a single image or text description as input and produces a simulation-ready URDF asset in seconds (Table 2, Time column), enabling batch generation of varied articulated objects from a catalog of product images or procedurally generated descriptions. The 69.3% joint type accuracy and 12.64° axis error mean that most generated assets will have functionally correct kinematic structures (doors that swing rather than slide, axes oriented approximately correctly), while the physics-based limit correction ensures collision-free motion—sufficient for training robust manipulation policies, which benefit from geometric and kinematic diversity more than from exact fidelity to any specific real-world instance. The real2sim demonstration in Figure 6 shows that assets generated this way can faithfully reproduce real articulation behavior when replaying recorded trajectories, suggesting the simulation-to-reality transfer gap is manageable for policies trained on these generated assets.

2. Rapid digital twin construction from single images for augmented reality and virtual commissioning. Industrial AR applications—overlaying maintenance instructions on equipment, visualizing retrofit options, simulating workflow changes—require digital twins of existing physical assets, which are typically created through labor-intensive manual modeling or expensive 3D scanning pipelines. ArtLLM could reduce this to a single-image capture workflow: a facility manager photographs a piece of equipment (a control cabinet, an access panel, a machinery enclosure), the image is processed through Hunyuan3D for mesh generation and ArtLLM for articulation prediction, and the resulting URDF asset is immediately usable in an AR environment for interactive visualization. The 83.0% part mIoU means the generated part layouts align well with the visible object structure, and the sub-second-to-seconds inference time enables real-time or near-real-time asset creation in the field. The primary limitation for this use case is the 31.8% joint range IoU—the digital twin's motion range may not exactly match the physical asset—but for visualization and planning applications (rather than precision manipulation), approximate limits are often acceptable, and the physics-based correction guarantees collision-free motion, which is the minimum bar for interactive AR.

3. Data augmentation for training part segmentation and articulation inference models. The current bottleneck in training models like ArtLLM itself is the availability of annotated articulated object data—PartNet-Mobility contains only 2,346 objects, and manual URDF annotation is slow and expensive. ArtLLM, combined with a 3D generation model like Hunyuan3D that can produce varied geometries from text prompts, could generate synthetic training data with known ground-truth articulations: prompt the 3D generator for "a modern kitchen cabinet with two drawers and a door," run ArtLLM to predict the articulation, apply the physics-based correction to ensure collision-free limits, and use the resulting asset as a training example for the next generation of articulation prediction models. This is an amplification loop: the current model (trained on 20,673 objects) generates training data for a successor model, which can then be trained on a much larger dataset (potentially millions of procedurally varied objects), improving its accuracy and closing the gap on metrics like joint range IoU that are currently limited by training data scale. The key risk—and the experiment that would validate this approach—is whether errors in the current ArtLLM's predictions propagate and amplify through successive training generations (a "model collapse" scenario) or whether the physics-based correction and diversity of generated geometries provide sufficient signal for the successor model to improve.

4. Accelerating content creation pipelines for game development and interactive simulation. Game studios and simulation companies spend substantial artist time manually authoring articulation parameters for interactive objects—setting hinge axes, configuring motion limits, defining collision boundaries. ArtLLM can automate the kinematic authoring step from a static 3D mesh (which artists have already created or which was generated by a 3D asset pipeline), producing a URDF file that can be imported directly into game engines (Unity, Unreal) or physics simulators (MuJoCo, Isaac Sim). The seconds-scale inference time (Table 2) means an artist could process hundreds of assets in a batch workflow, manually verifying and adjusting only the cases where ArtLLM's predictions are visibly incorrect. Given the 69.3% joint type accuracy, roughly 7 in 10 joints are correctly classified on the first attempt; for the remaining 3 in 10, the artist adjusts the joint type and regenerates, still saving substantial time compared to manual authoring from scratch. The modular architecture—ArtLLM for articulation, XPart for geometry—means that if a studio already has high-quality part geometries (e.g., from their existing asset library), they can skip the part generation step and use ArtLLM purely for kinematic structure prediction, further streamlining the pipeline.