P3-SAM: Native 3D Part Segmentation

🎯 Pitch

For the first time, a native 3D model can automatically segment any object’s parts without any 2D supervision, achieving state-of-the-art precision on complex, non-watertight meshes. Trained on 3.7M artist-created assets with automatically derived labels, P³-SAM predicts multi-scale part masks from a single point click. It even handles new shapes in real time, finally closing the gap between interactive 3D segmentation and practical, automated pipelines.

1. Executive Summary

This paper introduces P³-SAM, a native 3D point-promptable part segmentation model that fully automates the segmentation of arbitrary 3D objects into constituent parts without relying on 2D foundation models. Trained on a newly curated dataset of nearly 3.7 million artist-created meshes with automatically derived part labels, the architecture combines a Sonata point cloud encoder, a two-stage multi-scale segmentor (predicting three masks at different granularities from a single positive point prompt), and an IoU predictor (evaluating mask quality to select the best one automatically). The method achieves state-of-the-art performance across three evaluation benchmarks — PartObj-Tiny, PartObj-Tiny-WT, and PartNetE — while operating with real-time interactive speed (feature extraction computed once, then mask prediction at inference latency measured in milliseconds), and demonstrates robustness across diverse data types including non-watertight meshes, watertight meshes, and raw point clouds. A key boundary finding is that fully automatic segmentation is achieved through Farthest Point Sampling of prompt points followed by Non-Maximum Suppression for mask deduplication, establishing that geometric learning from native 3D supervision alone suffices for precise part decomposition only when the training data covers both watertight and non-watertight geometry distributions.

2. Context and Motivation

The Core Problem: 3D Part Segmentation Without Human Intervention

The fundamental challenge this paper addresses is deceptively simple: given an arbitrary 3D object, automatically decompose it into its constituent semantic parts without knowing anything about the object's category, part vocabulary, or expected number of parts. This is the problem of class-agnostic part instance segmentation — each part must be identified as a distinct instance (e.g., "the left arm" vs. "the right arm"), but no categorical label (e.g., "arm") is required or provided.

This problem stands in sharp contrast to the traditional semantic segmentation setting, where you know in advance that you're looking at, say, a chair, and you have a predefined ontology of parts (leg, seat, backrest, armrest) with labeled training examples. In that setting, the model learns a mapping from geometry to one of $K$ known part classes, and the number of output segments is fixed by the ontology. But this traditional setting has a fundamental limitation: it cannot generalize beyond its training categories. A model trained on chairs and tables has no mechanism for segmenting a stapler, a drone, or a fantasy creature — objects whose part decompositions are semantically meaningful but not represented in any benchmark dataset.

The paper's goal is to move beyond this closed-world assumption to enable unrestricted part segmentation of any object. The practical motivation is clear: part-level understanding is a prerequisite for many downstream applications that the paper explicitly enumerates (Section 1):

Shape editing and reuse: An artist who wants to modify just the hilt of a sword model needs to isolate that component without manually selecting vertices.
Mesh simplification: Different parts of a model benefit from different levels of geometric detail; automatic part identification enables adaptive simplification strategies.
Animation design: Rigging a character requires knowing where the limbs attach — information that part segmentation provides structurally.
Part generation (Section 4.2): Models like HoloPart need segmented input to generate complete, manifold part geometries; segmentation quality directly impacts generation quality.

But beyond these application-specific motivations, there is a deeper, more structural reason this problem matters for 3D understanding: parts are the intermediate level of structure between raw geometry and full object semantics. They capture functional organization (what detaches from what, what articulates relative to what) without requiring full categorical knowledge. Building systems that can recover this structure from geometry alone is a test of how well we can learn the compositional grammar of shape — a question with both practical and cognitive science implications.

The Two Existing Paradigms and Why Both Fall Short

The paper identifies two broad approaches to class-agnostic part segmentation in prior work, each with fundamental weaknesses:

Paradigm 1: Traditional Supervised Learning on Labeled Part Datasets

These methods (PointNet, PointTransformerV3, MeshCNN, Mesh Transformer; Section 2.1) train neural networks to predict per-point or per-face part labels using datasets like PartNet (Mo et al., 2019) or Princeton Mesh Segmentation Benchmark (Chen et al., 2009). The training pipeline is straightforward: encode geometry with a point cloud or mesh encoder, then use a segmentation head to classify each element into one of $K$ predefined part categories.

Where they fall short: The core limitation is scalability of supervision. PartNet contains approximately 26,000 objects across 24 object categories, with part labels that are category-specific. This is orders of magnitude too small to learn general-purpose part decomposition heuristics — the model learns what a "chair leg" looks like, not what makes any protrusion a distinct part. The paper notes this directly:

"traditional methods suffer from limited categories and part labels and struggle to generalize to arbitrary categories and parts" (Section 2.1)

This is not a fixable limitation within the paradigm; it's inherent to the requirement of human-annotated part labels, which are time-consuming and expensive to produce at scale. Importantly, these methods also require knowing the object category at inference time (to know which vocabulary of parts to predict), meaning they fundamentally cannot handle novel object types.

Paradigm 2: 2D Lifting — Using 2D Foundation Models as a Bridge to 3D

The more recent and technically sophisticated approach leverages the remarkable capabilities of 2D foundation models — particularly SAM (Segment Anything Model; Kirillov et al., 2023), DINOv2 (Oquab et al., 2023), CLIP (Radford et al., 2021), and various vision-language models — to perform 3D segmentation by proxy. The general recipe (Sections 2.2 and A.2.2) works as follows:

Render the 3D object from multiple viewpoints (typically 12–36 views at vertices of an icosahedron or similar sampling).
Run a 2D segmentation model (SAM) or feature extractor (DINOv2, CLIP) on each rendered image to produce 2D masks or dense feature maps.
Project these 2D results back onto the 3D geometry — a mask in a single view gets associated with the visible 3D faces/points.
Aggregate across views to resolve conflicts and produce a consistent 3D segmentation. This typically involves some form of voting, graph-cut optimization, or clustering that reconciles the fact that the same 3D face may be assigned to different masks in different views.
Optionally, use 2D vision-language models or text prompts to assign semantic labels to the resulting parts.

Specific implementations vary in their details. SAMesh (Tang et al., 2024) directly applies SAM to rendered images and aggregates the resulting multi-view masks — it is one of the few methods that achieves fully automatic segmentation without human prompts, but as the paper shows in Table 2, it tends to over-segment objects (producing too many parts with fragmented boundaries). SAMPart3D (Yang et al., 2024) first distills DINOv2 features onto 3D point clouds via a neural network, then trains per-object lightweight MLPs using contrastive learning on SAM-projected masks, and finally uses a multi-modal LLM to annotate parts — a multi-stage pipeline where each stage introduces potential error accumulation. PartField (Liu et al., 2025) uses a voxel CNN and tri-plane transformer to learn a 3D feature field supervised by contrastive learning on both 2D (SAM-generated) and 3D (PartNet) masks, then performs hierarchical clustering with connectivity priors to produce segments. Point-SAM (Zhou et al., 2024) adapts the SAM architecture directly to 3D point clouds and uses a 2D-based data engine to iteratively train and refine a PointViT model, supporting both positive and negative point prompts.

Where these methods fall short — four interrelated failures:

1. The 2D–3D data gap (the fundamental problem). This is the most deep-seated issue and the one the paper returns to repeatedly. When you render a 3D object to a 2D image, you inevitably lose geometric information: occluded surfaces, depth relationships between parts that appear adjacent in projection, and fine geometric detail that doesn't correspond to strong image gradients. SAM and DINOv2 were trained on natural images with rich texture, lighting, and color cues. A rendered 3D model — especially the untextured, single-color meshes common in CAD and 3D asset repositories — provides none of these signals. The result is that 2D segmentation boundaries, optimized for photographic discontinuities, frequently fail to align with genuine 3D part boundaries. The authors state this explicitly:

"segmentation based on 2D data can still suffer from boundary ambiguities and data gaps, leading to inaccurate segmentation results, especially on complex data" (Section 2.3)

This boundary ambiguity is visible in Figure 4, where SAMPart3D and PartField produce masks with jagged, imprecise edges — exactly the pattern you would expect when 2D segmentation boundaries (computed on pixel grids) are projected onto 3D surfaces at oblique angles, introducing quantization noise and misalignment.

2. 3D consistency and view aggregation failures. Even when individual views produce reasonable 2D masks, aggregating them into a globally consistent 3D segmentation is a hard combinatorial optimization problem. A part visible across multiple views should be assigned the same label in all projections, but 2D SAM has no mechanism for cross-view correspondence — it processes each image independently. The post-hoc aggregation step (voting, graph-cut, clustering) is an engineering patch, not a principled solution, and can produce artifacts where the same 3D face receives conflicting labels from different views. This is particularly problematic for objects with self-occlusions or complex topology, where a part may be visible in only a subset of views and its boundaries in those views may differ significantly.

3. Computational cost. The paper emphasizes (Table 1) that 2D lifting methods require rendering multi-view images and running 2D foundation models on each one. SAMesh renders 12 images at icosahedron vertices and runs SAM on each; other methods may require more views for adequate coverage. For a single object, this is manageable. For processing the 3.7 million objects in the paper's training dataset — or for any production-scale pipeline — the cost becomes prohibitive. The paper's native 3D method, by contrast, processes each object directly without rendering, achieving inference speeds orders of magnitude faster (the feature extractor runs once per object, and mask prediction from a point prompt is real-time).

4. Incomplete automation. With the exception of SAMesh, the methods described in Paradigm 2 require some form of human input at inference time. Find3D needs text prompts to query parts. SAMPart3D and PartField need the user to specify the number of part categories for clustering (or to select the correct level from a hierarchical clustering dendrogram). Point-SAM requires manual selection of prompt points — it is fundamentally an interactive tool, not an automatic segmentation system. The paper positions this as a critical gap:

"these methods either require specifying the number of categories or need user-provided prompt points, which means they cannot fully automate object segmentation" (Section 2.3)

This is not merely an inconvenience. For batch processing of large 3D asset libraries, for integration into automated 3D generation pipelines, or for serving as input to downstream models like HoloPart, requiring per-object human intervention renders these methods impractical at scale.

Paradigm 2.5: 2D Data Engines (A More Sophisticated Lifting Strategy)

A subset of methods (SAMPart3D, PartField, Point-SAM, Find3D, Segment3D) attempt to mitigate the 2D–3D consistency problem by using 2D foundation models not directly at inference time, but as a data generation engine for training 3D-native networks. The idea is: use SAM to generate pseudo-ground-truth masks on many views of many objects, then train a 3D point cloud or mesh network to predict these masks from 3D geometry alone. At inference time, the 3D network runs natively — no rendering, no multi-view aggregation.

This approach partially addresses the consistency and speed problems (the 3D network learns to internalize geometric priors, and inference is faster), but it creates a new fundamental limitation: the quality ceiling is set by the 2D teacher. If SAM's masks are imprecise or inconsistent (due to the 2D–3D data gap), the 3D student network learns to reproduce those same imprecisions. You cannot train your way past the limitations of your supervision signal. The paper's central architectural choice — to train exclusively on native 3D part labels derived from artist-authored mesh structure — is a direct response to this limitation.

The Missing Data Problem and the Paper's Key Strategic Insight

A crucial sub-problem that makes the 3D-native approach challenging is the scarcity of high-quality 3D part segmentation data. PartNet, the largest existing dataset with human-annotated part labels, contains only ~26,000 objects across 24 categories. By contrast, ImageNet has 14 million images; COCO has 330,000. Training a general-purpose part segmentation model on PartNet-scale data would inevitably overfit to the limited geometry and part vocabulary represented.

The paper's most important strategic insight — the one that enables everything else — is the observation that artist-created 3D models in repositories like Objaverse already encode part-level structure in their scene graph. When an artist models a car, they typically create separate mesh components for the body, wheels, windows, and doors, then assemble them into a single scene. The mesh connectivity information — which faces belong to which sub-mesh — is a de-facto part segmentation label. This information is not explicitly annotated as "this is a part label," but it can be extracted programmatically using what the paper describes as a part annotation pipeline (Section 3.1 and detailed in Appendix A.3.1).

The pipeline works as follows:

Decompose the object into connected components. Since artists typically don't merge meshes of distinct parts, each sub-mesh forms a connected component in the face graph.
Compute surface areas and adjacency. For each connected component, calculate its surface area. Build an adjacency graph between components by voxelizing space (128³ resolution) and connecting components that share any voxel.
Iteratively merge small components. Components with surface area less than 1% of the total are merged into their adjacent larger components. This bottom-up process continues until all remaining components exceed the 1% threshold. The purpose is to clean up the long tail of tiny decorative elements (screws, buttons, rivets) that are technically separate meshes but shouldn't constitute independent parts.
Filter degenerate cases. Objects with fewer than 2 parts (no segmentation possible), more than 50 parts (excessively fragmented, as shown in Figure 8), objects with a single part occupying >85% of surface area (imbalanced, Figure 9), and objects where the collective area of sub-1% fragments exceeds 10% of total area are all removed.

This pipeline transforms the raw artist mesh structure into clean, coarsely meaningful part labels — not manually annotated by humans, but algorithmically derived from the artist's own modeling decisions. The result is nearly 3.7 million labeled objects, over 100× larger than PartNet, spanning a vastly more diverse set of object categories and geometries.

This is the key insight that breaks the data bottleneck. It's not that the paper proposes a new segmentation architecture (though the architecture is novel); it's that the paper recognizes that the data needed to train such an architecture already exists in artist-created 3D repositories — you just need to extract and clean it.

The Watertight/Non-Watertight Distribution Challenge

A subtler data problem that the paper identifies and addresses is the distribution mismatch between artist-created meshes and real-world 3D data. Artist-created meshes (the raw Objaverse data) are typically non-watertight at the object level: they contain internal structures, have clear boundaries between parts, and are composed of separate sub-meshes that may not form a closed manifold surface. In contrast, scanned 3D data (from photogrammetry or depth sensors) and AI-generated 3D models (from diffusion-based or NeRF-based methods) are typically watertight — a single closed manifold without internal structure, where part boundaries are purely geometric discontinuities rather than mesh connectivity breaks.

Training exclusively on non-watertight data produces models that exploit the connectivity signals present in training but absent at inference, leading to brittle generalization. The paper's solution is elegant and practical: take the filtered non-watertight meshes and compute their watertight hulls, using the nearest-neighbor label transfer procedure described in Appendix A.3.1 to propagate part labels from the non-watertight point cloud to the watertight surface. Of the 3.7 million objects, approximately 2.3 million could be successfully watertightened. During training, with 80% probability, the watertight version of an object is used, and with 20% probability, the non-watertight version. This mixed training strategy forces the model to rely on geometric features rather than connectivity shortcuts, resulting in robust performance on both data types (as demonstrated in Tables 2 and 3).

How the Paper Positions Itself

The paper's positioning can be understood along three axes:

Axis 1: Native 3D vs. 2D lifting. The paper draws a sharp line between its approach and all methods that rely on 2D foundation models. The title itself — "Native 3D" — signals this positioning. The claim is not just that native 3D is more convenient or faster (though it is both); the claim is that 2D-derived supervision is inherently limited by the 2D–3D data gap, and that only direct 3D geometric learning can achieve the precision, robustness, and consistency required for production-quality part segmentation. Table 1 in the paper makes this comparison explicit across multiple dimensions: training data quantity (3.7M vs. at most hundreds of thousands for 2D-engine methods), number of parameters, inference time cost (ours: feature extraction once + real-time mask prediction; theirs: multi-view rendering + 2D model inference), and ability to handle watertight data.

Axis 2: Single positive point prompt vs. complex prompting. Unlike SAM or Point-SAM, which support multiple prompt types (positive points, negative points, bounding boxes, masks), P³-SAM uses only a single positive point prompt. This deliberate simplification serves two purposes. First, it eliminates the need for a complex prompt encoder — the point coordinate is simply concatenated (repeated $N_p$ times) with the point features and input coordinates. Second, and more importantly for the paper's goal of automation, single-point prompting enables the FPS + NMS automatic segmentation pipeline: you can sample many candidate points, predict a mask from each, and use IoU-based deduplication to recover the full set of parts. Multi-point or negative-point prompting would complicate this pipeline considerably, since you would need to decide which combination of prompts to use for each candidate part.

Axis 3: Full automation as a first-class design goal. The paper treats automatic segmentation not as an afterthought (as in prior work, where automation was either absent or achieved through fragile heuristics) but as a primary system requirement that shapes every architectural decision. The multi-scale prediction (three masks at different granularities) combined with the IoU predictor enables the model to produce a ready-to-use mask from a single click without user selection. The FPS sampling strategy for prompt points is designed to be dense enough that every part receives at least one prompt. The NMS post-processing with a 0.9 IoU threshold is calibrated to merge over-segmented predictions while preserving genuine part boundaries. The flood-fill label propagation step handles the face-level projection cleanly. Together, these components form a complete, end-to-end automated pipeline that requires zero human input at inference time — a capability that none of the compared methods (except the over-segmenting SAMesh) provide.

In the broader landscape of 3D understanding, this paper represents a bet that geometry-first, data-scale-driven learning can outperform the more complex but data-limited approaches based on transferring 2D foundation model knowledge. The bet pays off in the experimental results (Tables 2 and 3, Figure 4), but the more significant contribution may be methodological: demonstrating that by rethinking what constitutes "labeled data" for 3D part segmentation, you can unlock training-scale regimes that were previously infeasible.

3. Technical Approach

3.1 Reader Orientation

P³-SAM is a neural network that, given a 3D object and a single click on its surface, immediately predicts which complete part that click belongs to—and does this quickly enough that, by clicking many points and filtering duplicates, the system can fully decompose any object into its parts without any human guidance. The system solves the class-agnostic part instance segmentation problem through a design where one heavyweight feature extraction step (run once per object) is followed by many lightweight mask predictions (run per click), making the architecture simultaneously precise enough for production use, fast enough for interactive applications, and structured so that automation emerges naturally from sampling and deduplication rather than requiring complex post-processing heuristics.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components connected in a feed-forward pipeline:

Feature Extractor (Sonata, a self-supervised pretrained PointTransformerV3): Takes a point cloud sampled from the input mesh (positions + normals) and produces multi-scale per-point feature vectors. This runs once per object, independent of any prompt.
Two-Stage Multi-Head Segmentor: Takes the frozen per-point features, the raw point coordinates, and a single prompt point coordinate (replicated $N_p$ times), and predicts three alternative segmentation masks at different granularities (small, medium, large) in two refinement passes. The first stage makes initial predictions; the second stage incorporates a global context feature and refines them.
IoU Predictor: Evaluates the three masks from the second stage by predicting their Intersection-over-Union with the true part, then selects the best mask as the network's final output for that prompt point.
Automated Segmentation Pipeline (Algorithm 1): Given the trained model, samples $N_{pp} = 400$ prompt points via Farthest Point Sampling, runs the model on each to produce 400 candidate masks with IoU scores, applies Non-Maximum Suppression (IoU threshold $T_{NMS} = 0.9$ ) to deduplicate overlapping masks, and projects the surviving point-level masks onto mesh faces with flood-fill for unlabeled faces.
Training Data Pipeline (Section 3.1): Extracts part labels from artist-created meshes by decomposing into connected components, merging tiny fragments, and filtering degenerate cases. Produces 3.7M labeled objects with both non-watertight and watertight versions.

Information flows as follows: an input mesh → point sampling → feature extraction (once) → for each of $N_{pp}$ prompt points → feature+prompt fusion → stage-1 mask prediction → global feature extraction → stage-2 mask refinement → IoU scoring → best mask selection → NMS deduplication → face projection → final per-face part labels.

3.3 Roadmap for the Deep Dive

First, the data curation pipeline, because it is the foundation that enables everything else—understanding how 3.7M labeled objects are derived from artist meshes explains why native 3D training is possible at all.
Second, the feature extractor (Sonata/PTv3), because all downstream components consume its output and the choice of encoder determines the quality ceiling.
Third, the two-stage multi-head segmentor, the core architectural novelty—how a single point prompt is converted into three masks at different scales, and why two refinement stages and global feature injection matter.
Fourth, the IoU predictor, which makes the multi-head design practical by automatically selecting the best mask without user intervention.
Fifth, the training procedure: losses, data augmentation, and the critical "backprop-only-to-best" strategy that forces multi-scale specialization.
Sixth, the automatic segmentation algorithm, which transforms the interactive model into a fully automated system through FPS sampling, NMS deduplication, and face projection.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and data paper whose core idea is that a native 3D promptable segmentation model, trained at massive scale on artist-derived part labels, can achieve precise, robust part decomposition without the 2D–3D data gap that limits prior methods.

Data Curation: Extracting Part Labels from Artist Meshes

The training dataset is constructed from 3D models aggregated from multiple repositories: Objaverse, Objaverse-XL, ShapeNet, PartNet, and other internet sources. The key insight enabling large-scale training is that artist-created meshes encode part structure in their scene graph—artists model parts as separate sub-meshes and assemble them without merging, making connected-component decomposition a proxy for part labeling.

Step 1: Connected-component decomposition. For each object mesh $\mathbf{M} = (\mathbf{V}, \mathbf{F})$ , where $\mathbf{V} \in \mathbb{R}^{N_v \times 3}$ are vertices and $\mathbf{F} \in \mathbb{N}^{N_f \times 3}$ are triangular faces, the mesh is split into sub-meshes based on face connectivity. Two faces are connected if they share an edge; a connected component is a maximal set of faces reachable through such edge adjacencies. Since artists typically don't merge the mesh boundaries between distinct parts, each component corresponds roughly to a part.

Step 2: Surface area computation and adjacency graph construction. For each connected component, the surface area is calculated. An adjacency graph between components is built by voxelizing the object's bounding volume (resolution $128^3$ ) and declaring two components adjacent if their voxelizations intersect—i.e., they occupy any shared voxel in the 3D grid.

Step 3: Iterative small-part merging. A bottom-up merging process eliminates the long tail of tiny decorative elements (screws, rivets, buttons) that are technically separate meshes but shouldn't constitute semantic parts:

Components with surface area less than 1% of the total object surface area are iteratively merged into the adjacent component with the largest surface area. This process repeats until all surviving components exceed the 1% threshold.

This is a greedy, deterministic procedure—at each iteration, the smallest sub-threshold component is found and merged with its largest neighbor—that progressively coarsens the part granularity to a reasonable level. The 1% threshold is an empirical choice that the paper doesn't ablate but which matches intuition: parts smaller than 1% are unlikely to be functionally meaningful.

Step 4: Degenerate case filtering. Four types of problematic objects are removed:

Too few parts (fewer than 2): An object with 0 or 1 parts after merging provides no segmentation signal—there's nothing to segment. These are typically single-mesh objects where the artist merged everything.
Too many parts (more than 50): Shown in Figure 8, these are objects with excessive fragmentation (hundreds of sub-meshes for decorative patterns, chainmail, scales). Even after merging, the part count exceeds 50, which would produce training examples where most prompts hit tiny, geometrically ambiguous parts. The threshold of 50 represents a practical upper bound on part complexity for the model's output.
Imbalanced parts (largest part >85% of surface area): Shown in Figure 9, these are objects where one component dominates—e.g., a car body that is 90% of the area with tiny wheels. In such cases, the "background" part is so large that segmentation adds little value, and prompts landing on the dominant part would have ambiguous boundaries with the tiny parts.
Excessive tiny-part residue (cumulative area of sub-1% fragments >10%): Some objects have many small fragments that individually are under 1% but collectively cover significant area. These resist the merging process (because no single merge pushes any component over 1%) and produce noisy, fragmented labels. Filtering these ensures label quality.

After filtering, approximately 3.7 million objects remain. Their part structure represents the artist's modeling decisions—not human annotation in the traditional sense, but a signal that captures functional decomposition because artists model distinct functional units as separate meshes.

Watertight/non-watertight dual dataset. Artist-created meshes are typically non-watertight: they contain internal structures, have clear boundaries between sub-meshes, and don't form a closed manifold. To ensure the trained model generalizes to watertight data (scanned meshes, AI-generated models), the paper creates watertight versions of the filtered meshes using the watertightening procedure from Hunyuan3D. Approximately 2.3 million meshes (62% of the filtered set) are successfully watertightened; the remaining ~1.4 million objects either fail the watertightening algorithm or produce degenerate results.

For the watertight versions, part labels must be transferred from the non-watertight source. The procedure (Appendix A.3.1) works as follows:

Sample points $\mathcal{P}_{nwt}$ from the non-watertight mesh, recording their part labels $\mathcal{L}_{nwt}$ from the connected-component decomposition.
Sample points $\mathcal{P}_{wt}$ from the watertight mesh.
For each watertight point $\mathbf{p}_{wt} \in \mathcal{P}_{wt}$ , find its nearest neighbor $\mathbf{p}_{nwt}$ in $\mathcal{P}_{nwt}$ and assign the label of $\mathbf{p}_{nwt}$ to $\mathbf{p}_{wt}$ .

This is a nearest-neighbor label transfer in 3D Euclidean space, which works because the watertight mesh approximates the outer envelope of the non-watertight mesh—points on the watertight surface should be close to points on the corresponding non-watertight part surfaces, except where internal structures have been filled in. The resulting labels $\mathcal{L}_{wt}$ for the watertight point cloud form the supervision signal.

During training, for each object that has a watertight version, the watertight data is selected with 80% probability and the non-watertight data with 20% probability. This biased mixture ensures the model primarily learns geometric features (watertight data has no internal structure to exploit) while retaining the ability to handle non-watertight meshes at inference time.

Training-time point sampling. During training, $N_p = 100{,}000$ points are sampled from the selected mesh (watertight or non-watertight). Each point inherits the part label of its source face/component. The same $N_p = 100{,}000$ count is used during evaluation and inference.

Feature Extractor: Sonata (Self-Supervised Pretrained PointTransformerV3)

The feature extractor $\mathcal{E}$ is the only component that processes the entire point cloud; all downstream mask predictions reuse its output. The paper chooses Sonata (Wu et al., 2025), a self-supervised pretrained PointTransformerV3, for its state-of-the-art performance on point cloud understanding tasks.

Input. The extractor takes a point cloud $\mathbf{P} \in \mathbb{R}^{N_p \times 3}$ and corresponding normals $\mathbf{N} \in \mathbb{R}^{N_p \times 3}$ sampled from the input mesh. Normals are included because part boundaries often correspond to surface orientation discontinuities—where a protrusion meets a base surface, the normal direction changes sharply, and this signal is geometrically fundamental to part decomposition.

Multi-scale feature extraction. The extractor processes the point cloud through a hierarchical transformer architecture that progressively downsamples the point set and computes features at multiple spatial scales:

$\mathbf{f}_p = \mathcal{F}_e(\mathcal{E}(\mathbf{P}, \mathbf{N})_1, \mathcal{E}(\mathbf{P}, \mathbf{N})_2, \ldots, \mathcal{E}(\mathbf{P}, \mathbf{N})_n)$

where $\mathcal{E}(\mathbf{P}, \mathbf{N})_k$ represents the feature map at scale $k$ (from the $k$ -th stage of the hierarchical encoder), and $\mathcal{F}_e$ is a shared-weight MLP that fuses these multi-scale features.

What it computes: For each of the $N_p$ input points, the extractor produces a $512$ -dimensional feature vector that encodes geometric context at multiple spatial scales—a point on a chair leg will have features that capture both local curvature (small scale) and its role as part of an elongated protrusion from a larger base (large scale). The shared-weight MLP $\mathcal{F}_e$ projects the concatenated multi-scale features to a fixed $512$ -dimensional space.

Why this form: A single-scale feature would be either too local (missing the global context needed to distinguish, say, a chair leg from a similarly-curved table leg in a complex scene) or too global (blurring the precise boundary information needed for accurate mask edges). Multi-scale fusion, which is standard in point cloud architectures following PointNet++ and PTv3, provides both. The shared MLP $\mathcal{F}_e$ ensures the same projection is applied pointwise, maintaining permutation invariance while reducing dimensionality.

Design choice: frozen vs. fine-tuned. The paper uses Sonata with its pretrained weights as the feature extractor. The pretraining (self-supervised on large-scale point cloud data) provides strong geometric priors that transfer to the part segmentation task. The paper does not specify whether the feature extractor is fine-tuned during segmentation training or kept frozen—the description "we employ Sonata with its pre-trained weights as our feature extractor" suggests the weights are used as initialization and may be fine-tuned, but this is ambiguous. In practice, the batch size of 2 on 64 H20 GPUs and the 9-epoch training duration suggest end-to-end training, with the feature extractor adapting to the part segmentation distribution.

Voxel size adjustment. The paper mentions reducing the voxel size of the input to Sonata "to better handle complex objects." This controls the grid resolution at which the point cloud is voxelized for the transformer's sparse convolution backbone. A smaller voxel size preserves finer geometric detail at the cost of increased computation. The exact voxel size is not specified, but the adjustment indicates that the default Sonata configuration was inadequate for the geometric precision required by part segmentation.

Critical property: run-once efficiency. The feature extractor is computed once per object, producing $\mathbf{f}_p$ (stored in memory). All subsequent mask predictions for different prompt points reuse the same $\mathbf{f}_p$ . This property is what enables both interactive speed (feature extraction is the computational bottleneck; mask prediction is lightweight) and the automatic segmentation pipeline (400 masks can be predicted without recomputing features).

Two-Stage Multi-Head Segmentor: From Point Prompt to Three Masks

This is the core architectural contribution. Given the frozen per-point features, the input points, and a single prompt point, the segmentor predicts three alternative masks representing possible part segmentations at different granularities, then refines them with global context.

Why three masks? The multi-scale ambiguity problem. The paper identifies a fundamental ambiguity in point-promptable part segmentation: a single click on a surface could indicate different "scales" of part depending on the user's intent or the object's structure. In a hierarchical decomposition, a click on a finger could mean "segment the finger" (fine-grained), "segment the hand" (coarse), or "segment the entire arm" (coarser). Different objects in the training data have different part granularities—some have detailed sub-parts (individual petals of a flower), others have coarse parts (the entire flower head). Training a single-head segmentor to handle this ambiguity would produce averaged, imprecise masks. The three-head design lets each head specialize in a different scale, with the IoU predictor selecting the appropriate one at inference time.

Stage 1: Initial mask prediction. The first stage consists of three independent MLPs $\mathcal{F}^{(1)}_1, \mathcal{F}^{(1)}_2, \mathcal{F}^{(1)}_3$ , each producing a mask:

$\mathbf{m}^{(1)}_i = \mathcal{F}^{(1)}_i(\mathbf{f}_{in}) = \mathcal{F}^{(1)}_i(\mathbf{f}_p, \mathbf{P}, \mathbf{p}), \quad i = 1, 2, 3$

where $\mathbf{f}_{in}$ is the concatenation of three components along the feature dimension: $\mathbf{f}_p \in \mathbb{R}^{N_p \times 512}$ (per-point features from the extractor), $\mathbf{P} \in \mathbb{R}^{N_p \times 3}$ (raw point coordinates), and $\mathbf{p} \in \mathbb{R}^{N_p \times 3}$ (the single prompt point coordinate, replicated $N_p$ times—one copy per input point).

Each $\mathcal{F}^{(1)}_i$ is an MLP that maps this concatenated input to a scalar per point, producing $\mathbf{m}^{(1)}_i \in [0, 1]^{N_p}$ via a sigmoid activation. The output is a soft mask where $\mathbf{m}^{(1)}_{ij}$ (the $j$ -th element) represents the predicted probability that point $j$ belongs to the prompted part.

What the stage-1 input represents: For each point $j$ , the network sees three things simultaneously: (a) the geometric feature $\mathbf{f}_{p,j}$ describing the local and regional shape around point $j$ , (b) the absolute 3D position $\mathbf{P}_j$ of point $j$ , and (c) the absolute 3D position $\mathbf{p}$ of the prompt point. By comparing (b) and (c), the network can learn spatial relationships—"am I close to the prompt?" "am I on the same side of a concavity as the prompt?" By using (a), it can learn geometric similarity—"does my local shape match the prompt's local shape in a way that suggests we're part of the same connected component?"

Why only a single positive point prompt? The paper deliberately simplifies the prompting interface relative to SAM and Point-SAM, which support positive points, negative points, bounding boxes, and mask inputs. The rationale (Section 3.2.1) is that single-point prompting allows the network to avoid adapting to diverse prompt types, "simplifying the network and improving its convergence, generalization, and accuracy." More concretely, it eliminates the need for a prompt encoder that handles variable numbers and types of prompts—the point coordinate is simply treated as additional per-point features. This also simplifies the automated segmentation pipeline: FPS sampling generates single points naturally, whereas generating sensible negative points or bounding boxes automatically is non-trivial.

Design choice: concatenating prompt as per-point features vs. using a separate prompt encoder. SAM uses a separate prompt encoder that produces a dense embedding from sparse prompts, then cross-attends this embedding with image features. P³-SAM's simpler approach—replicating the prompt coordinate $N_p$ times and concatenating it with per-point features—is possible because point clouds are inherently sparse and the prompt is always a 3D coordinate, not a box or mask. This avoids the need for cross-attention between prompt and point features, making the architecture more lightweight and faster to train.

Stage 2: Refinement with global context. The first stage is described as "a naive implementation that lacks support for global information." The problem is that each point's mask prediction in Stage 1 is based only on local and regional features—the network has no summary of the overall shape context, such as "this object is roughly symmetric" or "this part is one of four similar protrusions." Stage 2 addresses this by introducing a global feature $\mathbf{f}_g$ that summarizes the entire object and the Stage 1 predictions.

The global feature is computed as:

$\mathbf{f}_g = \text{MaxPool}(\mathcal{F}_g(\mathbf{f}_{in}, \mathbf{m}^{(1)}_1, \mathbf{m}^{(1)}_2, \mathbf{m}^{(1)}_3))$

where $\mathcal{F}_g$ is an MLP that takes the Stage 1 input $\mathbf{f}_{in}$ concatenated with the three Stage 1 masks $\mathbf{m}^{(1)}_1, \mathbf{m}^{(1)}_2, \mathbf{m}^{(1)}_3$ and produces per-point features. MaxPool then aggregates these per-point features across the $N_p$ dimension, producing a single fixed-size global vector $\mathbf{f}_g$ .

What this computes: For each point, $\mathcal{F}_g$ processes the point's geometry, location, relation to the prompt, and its predicted membership in each of the three Stage 1 masks. MaxPool then selects the strongest activation across all points for each feature dimension, producing a "signature" of the prompted part and its context. If the Stage 1 masks have correctly captured a candidate part, the global feature will encode properties like its size, shape, and relationship to the rest of the object—information that Stage 1 lacked.

The three Stage 2 masks are then predicted by new MLPs $\mathcal{F}^{(2)}_1, \mathcal{F}^{(2)}_2, \mathcal{F}^{(2)}_3$ :

$\mathbf{m}^{(2)}_i = \mathcal{F}^{(2)}_i(\mathbf{f}_{in}, \mathbf{f}_g, \mathbf{m}^{(1)}_1, \mathbf{m}^{(1)}_2, \mathbf{m}^{(1)}_3), \quad i = 1, 2, 3$

Each $\mathcal{F}^{(2)}_i$ sees the original per-point features $\mathbf{f}_{in}$ , the global feature $\mathbf{f}_g$ (replicated $N_p$ times so it's available at every point), and the Stage 1 masks. The Stage 1 masks serve as a coarse initialization that the Stage 2 MLPs can refine—they know roughly which points are candidates, and can use the global context to correct boundary errors, fill holes, or adjust scale.

Why two stages rather than one deeper stage: This is a form of iterative refinement where the first pass provides a rough estimate and the second pass uses global aggregation over that estimate to improve it. The global feature $\mathcal{F}_g$ depends on the Stage 1 masks, creating a dependency that prevents this refinement from being done in a single forward pass. An alternative would be a recurrent or iterative refinement that applies the same MLP multiple times, but the two-stage design with separate parameters for each stage gives the network more capacity for refinement.

Why the Stage 1 masks are fed into $\mathcal{F}_g$ : If the global feature were computed only from $\mathbf{f}_{in}$ (geometry + prompt), it would encode the entire object's shape but wouldn't know which part is being segmented. By concatenating the Stage 1 masks, $\mathcal{F}_g$ can compute features specifically for the region that Stage 1 thinks is the part—the global feature becomes part-specific rather than object-generic, making it more informative for refinement.

Output granularity: soft masks. All six masks ( $\mathbf{m}^{(1)}_1, \ldots, \mathbf{m}^{(1)}_3, \mathbf{m}^{(2)}_1, \ldots, \mathbf{m}^{(2)}_3$ ) are soft: each element is a value in $[0, 1]$ representing the probability that the corresponding point belongs to the prompted part. Binary masks are obtained by thresholding at 0.5 via an indicator function $\mathcal{I}$ only during evaluation (for IoU computation) and inference (for final output).

IoU Predictor: Automatic Best-Mask Selection

With three masks produced at different scales, the system needs a mechanism to choose the best one automatically—both for the interactive use case (so users don't have to manually select among three options) and for the automated pipeline (which has no human in the loop).

Architecture. The IoU predictor takes the Stage 2 inputs and predictions and computes three scalar values $\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3$ :

$\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3 = \mathcal{F}_{iou}(\text{MaxPool}(\mathcal{F}'_{iou}(\mathbf{f}_{in}, \mathbf{f}_g, \mathbf{m}^{(2)}_1, \mathbf{m}^{(2)}_2, \mathbf{m}^{(2)}_3)))$

Here, $\mathcal{F}'_{iou}$ is an MLP that processes the per-point features $\mathbf{f}_{in}$ , the global feature $\mathbf{f}_g$ , and the three Stage 2 masks, producing per-point features. MaxPool aggregates to a global vector. $\mathcal{F}_{iou}$ is a second MLP that maps this global vector to three scalars—one predicted IoU value per mask.

What it computes: For each of the three Stage 2 masks, the IoU predictor estimates its Intersection-over-Union with the (unknown) ground truth part mask. The predicted IoU values $\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3$ are used to select the mask with the highest predicted quality as the final output: $i^* = \arg\max_i \mathbf{v}_i$ , and the final mask is $\mathbf{m}^{(2)}_{i^*}$ .

Why predict IoU rather than using a heuristic: A heuristic like "select the mask with median size" or "select the mask with the most confident predictions" would be fragile—optimal scale varies by object and prompt location. Learning to predict IoU directly from the Stage 2 features and masks lets the model calibrate its confidence based on geometric cues: if a predicted mask has clean boundaries and coherent shape, the IoU predictor should assign it a high score; if it's fragmented or overlaps poorly with geometry, a low score.

Why MaxPool for global aggregation: MaxPool selects the most activated feature across all points, which is appropriate for a quality-assessment task where a few points with strong signals (e.g., points at clean boundaries) may be more informative than an average over all points (which could be dominated by interior points with ambiguous features).

Training signal. The IoU predictor is supervised with MSE loss against the true IoU between each predicted mask (thresholded at 0.5) and the ground truth mask. This is described in the training section below.

Training Procedure: Losses, Augmentation, and Multi-Scale Specialization

Data augmentation. Three types of augmentation are applied during training:

Random noise to input points $\mathbf{P}$ : A noise scale $s$ is sampled uniformly from $(0, s_{max})$ where $s_{max} = 0.01$ (in normalized coordinate space). Gaussian noise $\mathcal{N}(0, s)$ is added to point coordinates. This simulates varying levels of point cloud noise from scanning or sampling, making the model robust to imprecise geometry.
Random noise to normals $\mathbf{N}$ : A noise scale $s$ is sampled from $(0, s_{max})$ , and noise $\mathcal{N}(0, s) \times 10$ is added to the raw normals, followed by renormalization to unit length. The factor of 10 amplifies the perturbation because normals are unit vectors and small noise would have negligible effect. This simulates normal estimation errors.
Random noise to prompt $\mathbf{p}$ : Gaussian noise with fixed scale $s_{max} = 0.01$ is added to the prompt coordinate. This makes the model robust to imprecise clicks during interactive use.

Additionally, normals are randomly removed with probability 0.3—the corresponding normal vectors are set to zero, forcing the model to rely on positional information alone when normal data is unavailable. This is important because some mesh formats don't include normals, or normals may be unreliable on noisy scanned data.

Training data sampling. For each object in a training batch, $K = 8$ part masks are randomly selected. For each selected mask, one point is randomly chosen from among the points belonging to that part to serve as the prompt $\mathbf{p}_j$ . This yields $K$ prompt–mask pairs $(\mathbf{p}_j, \mathbf{m}^{(gt)}_j)$ per object, where $\mathbf{m}^{(gt)}_j \in \{0, 1\}^{N_p}$ is the binary ground truth mask for part $j$ .

Mask loss: minimum-over-heads with Dice + Focal loss. The mask loss for stage $t$ (where $t = 1, 2$ ) is:

$\mathcal{L}^{(t)}_{mask} = \frac{1}{K} \sum^{K}_{j=1} \min^{3}_{i=1} \left( \alpha_{dice} \mathcal{L}_{dice}(\mathbf{m}^{(t)}_{ij}, \mathbf{m}^{(gt)}_j) + \mathcal{L}_{focal}(\mathbf{m}^{(t)}_{ij}, \mathbf{m}^{(gt)}_j) \right)$

where $\alpha_{dice} = 0.5$ is a weighting parameter balancing Dice and Focal loss contributions.

What it computes: For each ground truth part $j$ , we have three predicted masks from stage $t$ . The combined Dice + Focal loss is computed for each of the three masks against the ground truth. The $\min$ operator selects the best of the three masks (lowest loss), and only that mask's loss contributes to the gradient for that part—backpropagation is applied to the winning head only. The losses are then averaged across the $K$ parts.

Why minimum-over-heads: This is critical for multi-scale specialization. If all three heads were supervised with the same ground truth mask, they would converge to similar predictions (the training signal would push them all toward the same target). By backpropagating only to the head that already best matches the ground truth, each head specializes in the masks it can already predict well. Heads that predict poor masks for a given part receive no gradient from that example, so they don't get pulled toward a target they're not well-suited for. Over the course of training, the heads naturally diverge to capture different scales: one head may become good at fine-grained parts, another at medium-scale parts, and a third at coarse parts.

Why Dice + Focal loss: Dice loss is defined as $1 - \frac{2|A \cap B|}{|A| + |B|}$ (where $A$ and $B$ are the sets of points in the predicted and ground truth masks) and is standard for segmentation tasks with class imbalance—parts typically occupy a small fraction of the object's surface area, so a loss like binary cross-entropy would be dominated by the easy negatives (the vast majority of points not in the part). Dice loss focuses on the overlap region, which is where prediction errors matter most. Focal loss down-weights well-classified examples, further focusing learning on boundary regions where predictions are uncertain. Their combination provides both region-based (Dice) and point-wise (Focal) supervision with robustness to class imbalance. The weighting $\alpha_{dice} = 0.5$ balances the two components—without this, one loss might dominate due to different scales.

IoU loss: MSE against true IoU:

$\mathcal{L}_{IoU} = \frac{1}{3K} \sum^{K}_{j=1} \sum^{3}_{i=1} \mathcal{L}_{MSE} \left( \mathbf{v}_{ij}, \text{IoU}(\mathcal{I}(\mathbf{m}^{(2)}_{ij}), \mathbf{m}^{(gt)}_j) \right)$

where $\mathcal{I}$ is the indicator function: $\mathcal{I}(x) = 1$ if $x > 0.5$ , else $0$ . It binarizes the soft mask to compute IoU.

What it computes: For each of the $K$ parts and each of the 3 heads, the binary mask (thresholded at 0.5) is compared with the ground truth mask to compute the true IoU. The IoU predictor's output $\mathbf{v}_{ij}$ is compared to this true IoU using MSE. The loss is averaged across all heads and parts.

Why MSE for IoU prediction: IoU is a continuous value in $[0, 1]$ . MSE is the standard regression loss for predicting a bounded scalar. An alternative would be binary cross-entropy on a different target, but IoU provides a natural quality metric that directly captures what we want to optimize (selecting the mask with highest overlap). Unlike the mask loss, the IoU loss backpropagates to all three heads—every head needs to produce calibrated masks so the IoU predictor can accurately compare them.

Total loss:

$\mathcal{L} = \mathcal{L}^{(1)}_{mask} + \mathcal{L}^{(2)}_{mask} + \mathcal{L}_{IoU}$

What it computes: The three loss terms are simply summed. Stage 1 and Stage 2 mask losses are equally weighted (coefficient 1.0 each), as is the IoU loss. The paper does not mention any loss weighting beyond $\alpha_{dice}$ . This equal weighting implies that the loss magnitudes are naturally balanced—if one term dominated, training would under-optimize the others.

Training hyperparameters. The model is trained on 64 H20 GPUs for 9 epochs with a batch size of 2 per GPU (effective batch size 128). The Adam optimizer is used with learning rate $10^{-5}$ . Training takes approximately 4 days. The paper does not specify a learning rate schedule, weight decay, or gradient clipping—these are either Adam defaults or omitted.

Automatic Segmentation Pipeline: From Interactive Model to Fully Automated System

The trained P³-SAM model is interactive by design: you click a point, you get a mask. The automated segmentation pipeline (Algorithm 1) converts this interactive capability into fully automatic part decomposition of a complete object through three phases: prompt sampling, mask prediction and deduplication, and face projection.

Phase 1: Prompt point sampling via Farthest Point Sampling (FPS). Given the input mesh $\mathcal{M}$ , $N_p = 100{,}000$ points $\mathbf{P}$ with normals $\mathbf{N}$ are sampled. From these, $N_{pp} = 400$ prompt points $\mathbf{p}_j$ are selected using FPS.

What FPS does and why: Farthest Point Sampling iteratively selects points that are maximally distant from all previously selected points. Starting from a random initial point, each subsequent point is chosen as the one with the largest minimum distance to the already-selected set. This produces a uniform coverage of the object's surface—no region is left without at least one prompt point nearby. The number 400 is deliberately an over-sample: it exceeds the maximum possible number of parts (50, from the data filtering) by a factor of 8, ensuring that even with random variation and parts of different sizes, every genuine part receives at least one prompt point that falls within it.

Why FPS over grid sampling or random sampling: Grid sampling (voxelizing and taking one point per voxel) would be sensitive to voxel size and could miss small parts that fall entirely within one voxel. Random sampling could leave gaps where some parts receive no prompts due to chance. FPS guarantees coverage while maintaining a fixed budget of 400 prompts, making the pipeline deterministic and reproducible.

Phase 2: Mask prediction and Non-Maximum Suppression (NMS). The point-wise features $\mathbf{f}_p$ are extracted once from $\mathbf{P}$ and $\mathbf{N}$ using the feature extractor. For each of the 400 prompt points $\mathbf{p}_j$ , the segmentor and IoU predictor produce a mask $\mathbf{m}_j$ (the best of the three Stage 2 masks) and an IoU score $\mathbf{v}_j$ .

At this point, there are 400 masks—far more than the actual number of parts. Most masks overlap heavily because multiple prompt points fall on the same part. NMS (Algorithm 2) prunes these duplicates:

Sort the 400 masks in descending order by their predicted IoU values $\mathbf{v}_j$ . Higher-IoU masks (likely more accurate) are considered first.
Initialize an empty set of kept masks. Iterate through the sorted list. For each mask $\mathbf{m}_i$ :
- If it has not been suppressed by a previously kept mask, add it to the kept set.
- For all remaining masks $\mathbf{m}_j$ later in the list, if $\text{IoU}(\mathbf{m}_i, \mathbf{m}_j) > T_{NMS} = 0.9$ , suppress $\mathbf{m}_j$ (remove it from consideration).
The final kept set contains $N_{part}$ masks—the automatic part count.

What NMS computes: It greedily selects a set of masks that have high individual quality (selected in order of descending IoU) and low mutual overlap (any mask that overlaps a previously selected mask by more than 90% is suppressed). This eliminates the duplicate predictions from multiple prompts on the same part while preserving masks for distinct parts.

Why $T_{NMS} = 0.9$ : The threshold is high (90% overlap required for suppression) because genuine adjacent parts can have significant overlap in their boundary regions due to soft mask predictions. A lower threshold (e.g., 0.5) would risk suppressing masks for distinct but abutting parts—think of two adjacent petals of a flower, where the boundary region might have ambiguous predictions and produce moderate overlap. The 0.9 threshold ensures that only near-identical masks (almost certainly from prompts on the same part) are merged.

Why greedy selection (not global optimization): The greedy approach (sort by IoU, then suppress iteratively) is an approximation to the NP-hard maximum-weight independent set problem on the overlap graph. It's efficient ( $O(N_{pp}^2)$ in the number of prompts) and works well when the IoU scores are reliable quality indicators. The alternative—finding the globally optimal set of non-overlapping masks—would require combinatorial optimization that is impractical for 400 candidates.

Phase 3: Face projection and flood-fill. The NMS output provides $N_{part}$ point-level masks—each mask assigns each of the $N_p = 100{,}000$ sampled points to either "belongs to part $k$ " or "does not." To produce the final per-face segmentation on the original mesh, these point labels are projected to faces:

For each point, record which face it was sampled from. If a face has multiple sampled points with different mask assignments, majority voting determines the face's part label.
For faces that received no sampled points, the flood-fill algorithm propagates labels from neighboring labeled faces: for each unlabeled face, assign it the most frequent label among its edge-adjacent neighbors. If the mesh has no connectivity (e.g., raw point clouds or disconnected components), the nearest few faces in Euclidean space are used instead. This process iterates until all faces have labels.

Why flood-fill: The point sampling process may miss very small faces or faces in regions with low point density. Flood-fill provides a geometrically coherent completion—faces adjacent to a labeled region likely belong to the same part, assuming part boundaries correspond to geometric discontinuities where face adjacency is sparse. The paper notes that connectivity-based flood-fill is used for meshes with edge connectivity, and nearest-neighbor assignment for disconnected meshes. The final output is $\mathbf{m}_{part} \in \{1, 2, \ldots, N_{part}\}^{N_f}$ .

Why this pipeline achieves full automation: Unlike Point-SAM (which requires manual point selection) or PartField/SAMPart3D (which require specifying the number of categories), this pipeline requires zero human input. The FPS step automatically generates prompts covering the surface. The IoU predictor automatically selects the best mask per prompt. NMS automatically deduplicates overlapping masks and determines the part count. Flood-fill automatically propagates labels to all faces. Each component is designed to be fully automatic, and together they form an end-to-end solution.

Multi-Prompt and Hierarchical Applications

Beyond fully automatic segmentation, the paper describes two additional operating modes that leverage the same trained model.

Multi-prompt auto-segmentation (Appendix A.5): A user selects $K$ point prompts indicating specific parts they want segmented. The model predicts 3 masks per prompt (total $3K$ masks). Instead of using the IoU predictor to select one mask per prompt, a coverage-based selection algorithm is used: starting from the smallest mask for each prompt, progressively switch to larger masks until the union of all selected masks covers the entire object surface, while minimizing overlap between masks of different prompts. This ensures the user gets exactly the parts they specified, at the granularity that best covers the object without gaps or excessive overlap. This mode is useful when the user wants to segment specific parts (e.g., "segment the wheels of this car") rather than all parts.

Hierarchical part segmentation (Section 4.2 and Appendix A.5): After the automatic pipeline produces $N_{part}$ parts, hierarchical clustering is applied to the point-wise features $\mathbf{f}_p$ to organize parts into a tree. For each part, the average of $\mathbf{f}_p$ over all points in that part is computed as the part's representation. Hierarchical clustering on these part-level features groups parts by geometric similarity, producing a dendrogram that can be thresholded at different levels to yield coarser or finer segmentations. This addresses the hierarchical nature of part decomposition—a "car door" contains a "window," which contains a "window frame"—without requiring hierarchical annotations in the training data. The feature similarity structure that emerges from the Sonata encoder naturally captures geometric relationships (protrusions cluster together, planar surfaces cluster together) that align with part hierarchies.

4. Key Insights and Innovations

Innovation 1: Artist Mesh Structure as Implicit Part Supervision — Breaking the Data Bottleneck Without Human Annotation

The paper's most consequential contribution is not architectural but methodological: the recognition that artist-created 3D meshes already contain part-level labels in their scene graph, and that these labels can be extracted, cleaned, and scaled to train a general-purpose part segmentation model without any human annotation.

What the field did before: The dominant assumption in 3D part segmentation — shared by both traditional supervised methods (PointNet on PartNet) and 2D-data-engine approaches (SAMPart3D, PartField, Point-SAM) — was that high-quality part labels require either expensive human annotation (limiting scale to ~26K objects in PartNet) or noisy distillation from 2D foundation models (inheriting the 2D–3D data gap as a quality ceiling). In either case, the quantity of supervision was a binding constraint: PartNet's 24 categories and ~26K objects are orders of magnitude too small to learn general-purpose part decomposition heuristics, while 2D-derived pseudo-labels are fundamentally imprecise at boundaries and inconsistent across views. The field had accepted a tradeoff between data quality (human labels, small scale) and data quantity (2D engine, noisy labels).

What's distinctive: The paper's insight is that this tradeoff is false — you can have both quality and quantity if you redefine what counts as a "label." Artist-created meshes in repositories like Objaverse are assembled from separate sub-meshes representing distinct functional units (a car body, four wheels, two axles). The artist's modeling process — modeling parts separately and assembling them — is a form of implicit annotation: the connected-component decomposition of the mesh is the part segmentation, created by a human for functional reasons, not by an algorithm for training purposes. The paper's automated pipeline (connected-component decomposition → small-part merging → degenerate-case filtering) transforms this implicit structure into clean, coarsely meaningful part labels without any manual review.

Why this changes how we think about the problem: This reframes the 3D part segmentation data problem from "we need to collect expensive annotations" to "we need to find and clean annotations that already exist." The scale jump — from ~26K (PartNet) to ~3.7M objects — is over 100×, moving from the small-data regime (where generalization is constrained by label diversity) to the large-data regime (where geometric priors can be learned from data scale rather than architectural inductive bias). This is the same conceptual move that drove progress in 2D vision (ImageNet's 14M images) but applied to 3D for the first time at this scale for part segmentation. The paper's strong performance on out-of-distribution data — watertight meshes (Table 3), AI-generated models (Figure 14), point clouds (Table 3, PartNetE) — is direct evidence that data scale, not just architecture, is driving generalization.

The watertight/non-watertight mixed training strategy (80% watertight probability) is a subtle but important second-order insight within this innovation: simply having more data isn't enough if the data distribution doesn't match real-world use cases. Artist meshes are non-watertight; scanned and AI-generated models are watertight. Training only on non-watertight data would produce models that exploit connectivity shortcuts unavailable at inference time. By manufacturing watertight versions and training on a mixture, the model is forced to learn geometric features rather than mesh topology features, which is precisely what enables generalization.

Significance beyond raw performance: This is a fundamental contribution, not an incremental one. It changes the feasible scale of 3D part segmentation research and enables an entire class of approaches (native 3D, large-scale, geometry-only) that were previously infeasible due to data scarcity. The specific numbers — 3.7M objects, 2.3M watertight — are implementation details; the enduring contribution is the recognition that artist modeling decisions encode semantically meaningful structure that can be reverse-engineered programmatically.

Innovation 2: The Multi-Scale Ambiguity as a First-Class Design Target — Why Three Heads and "Backprop-to-Best" Training Matter

The paper identifies and architecturally addresses a problem that prior promptable segmentation work (including SAM itself) handled only partially: the scale ambiguity inherent in point-promptable part segmentation. A single click on a surface is fundamentally ambiguous about what scale of part the user intends — clicking on a fingertip could mean "segment the finger," "segment the hand," or "segment the arm." Different objects in the training data have different intrinsic part granularities (some have detailed sub-parts, others coarse parts), and different annotation conventions produce labels at different levels of detail.

What the field did before: SAM (Kirillov et al., 2023) addressed this by predicting three masks at different scales and requiring the user to select one — the ambiguity is acknowledged but pushed to the user. Point-SAM (Zhou et al., 2024) adopted the same approach. In the 2D domain, where interactive use is the primary use case, this is acceptable: the user can click and immediately see three options, selecting the appropriate one. In the non-interactive setting — which is the paper's goal for automation — this ambiguity must be resolved algorithmically, and prior work had no mechanism for doing so.

PartField (Liu et al., 2025) and SAMPart3D (Yang et al., 2024) handled scale through hierarchical clustering of feature vectors, which requires the user to specify the number of clusters or select a threshold — again pushing the ambiguity to the user. SAMesh (Tang et al., 2024) produced a single segmentation via multi-view aggregation but had no mechanism for controlling granularity, leading to systematic over-segmentation (visible in Figure 4).

What's distinctive: The paper treats the multi-scale ambiguity not as a user-interface problem but as a first-class machine learning design target. Three architectural decisions work together to address it:

Three specialized segmentation heads. Rather than one head that produces an averaged, imprecise mask on ambiguous prompts, three independent MLPs are trained to specialize at different granularities. The specialization is not hand-designed — it emerges from the backprop-to-best training strategy, where for each training example, only the head with the lowest loss receives gradient. This forces divergence: if two heads produce similar masks, the one that's slightly better will receive all the gradient and improve further, while the worse one receives none, maintaining its distinct prediction.
Automatic head selection via IoU prediction. Rather than requiring user selection among three candidates, a learned IoU predictor estimates the quality of each mask and selects the best one algorithmically. This is what enables the fully automatic pipeline — the model can self-evaluate which of its three predictions is most likely correct for a given prompt, without human input.
Stage 2 refinement with global context. The Stage 1 masks are "naive" (local-only predictions). Stage 2 introduces a global feature that summarizes the entire object's shape and the Stage 1 predictions, enabling refinement that can correct scale errors — if Stage 1 predicted too large a mask (including adjacent parts), the global context can identify that the predicted region spans multiple geometrically distinct components and shrink it.

The ablation study in Table 4 provides direct evidence: removing the multi-head design (single head, first row) or removing Stage 2 (second row) both degrade performance. The progressive improvement from single-head → stage-1-only → full model demonstrates that each component contributes independently.

Why this changes how we think about the problem: The innovation is in treating ambiguity as a prediction-and-selection problem rather than a user-interface problem. This is the conceptual move that enables automation: if the model can both predict multiple plausible segmentations and evaluate which is best, human intervention is no longer required. This is a different philosophy from SAM's "predict multiple, let the user choose" — it's "predict multiple, learn to choose yourself." The backprop-to-best training strategy (Section 3.2.2) is the mechanism that makes this work in practice, and it's a training technique that could generalize to other domains with inherent output ambiguity (e.g., image inpainting at multiple granularities, hierarchical object detection).

Significance: This is an incremental advance in architecture design but a fundamental advance in system design philosophy. The specific mechanism (two-stage multi-head segmentor with IoU predictor) is an engineering contribution; the guiding principle (resolve ambiguity through specialized prediction heads + learned self-evaluation, rather than through user interaction) is the lasting conceptual contribution.

Innovation 3: Full Automation as a First-Class Goal — The FPS + NMS Pipeline as a Principled Approach to Prompt-Based Part Discovery

Where prior promptable 3D segmentation methods treated automation as an afterthought or implemented it through fragile heuristics, P³-SAM elevates full automation to a primary design objective that shapes every component of the system. The resulting FPS + NMS pipeline is not an ad-hoc post-processing step but a principled approach to part discovery that emerges naturally from the model's design decisions.

What the field did before: The automation landscape in prior work was fragmented and limited:

SAMesh (Tang et al., 2024) achieved full automation through multi-view SAM aggregation, but produced over-segmented results with no mechanism to control granularity — it was automatic but qualitatively poor.
PartField (Liu et al., 2025) required the user to specify the number of clusters (or select from a dendrogram) for its hierarchical clustering step — automation was partial and required parameter tuning per object.
SAMPart3D (Yang et al., 2024) similarly used clustering with user-specified part counts.
Point-SAM (Zhou et al., 2024) was explicitly interactive — it could not segment an entire object without a human clicking on each part.
Find3D (Ma et al., 2024) required text prompts per part.

In every case, some aspect of the segmentation process — part count, prompt selection, granularity — required human judgment at inference time.

What's distinctive: The paper's automation pipeline works because four design decisions were made with automation in mind from the start:

Single positive point prompt only. By restricting to this simple interface, the model avoids the complexity of handling negative points, bounding boxes, or mask prompts — all of which would be difficult to generate automatically. Single-point prompts can be generated trivially by sampling points on the surface; negative points require knowing what not to segment, which presupposes segmentation.
Farthest Point Sampling for prompt generation. FPS provides a deterministic, parameter-free method for generating prompts that uniformly cover the object surface. The oversampling factor (400 prompts for at most 50 parts) ensures that every genuine part receives at least one prompt. This is a cleaner solution than grid-based or random sampling — FPS guarantees coverage without sensitivity to voxel size.
IoU predictor for automatic mask selection. Without this component, the model would produce three masks per prompt and have no way to select among them — making automation impossible. The IoU predictor closes this loop, enabling the model to self-evaluate and select its best prediction.
NMS with high IoU threshold (0.9) for deduplication. The 0.9 threshold is calibrated to suppress near-identical masks (multiple prompts on the same part) while preserving masks for adjacent but distinct parts. This is the component that determines the automatic part count $N_{part}$ — no user parameter specifies how many parts the object should have; the count emerges from the data through NMS.

Why this changes how we think about the problem: The traditional framing of promptable segmentation treats automation as a separate problem — "first build an interactive model, then figure out how to make it automatic." This paper demonstrates that automation is not an add-on but a design constraint that should shape the interactive model itself. The choice of single-point prompting, the IoU predictor, and the backprop-to-best training all serve dual purposes: they improve interactive segmentation quality and they enable the automated pipeline. This is a systems-level insight: designing for automation produces a better interactive model and vice versa, because the same capability — predicting a high-quality mask from minimal input and knowing when it's correct — is what both use cases require.

Significance: This is an incremental engineering contribution in terms of technical novelty (FPS, NMS, and flood-fill are standard techniques) but a fundamental contribution in system design philosophy. It establishes a template for how to convert an interactive promptable model into a fully automated system without sacrificing quality: oversample prompts uniformly, let the model self-evaluate, and deduplicate with a calibrated overlap threshold. The specific numbers (400 prompts, 0.9 threshold) are empirical choices; the pipeline structure is the reusable insight.

Innovation 4: 2D Lifting as a Structural Ceiling, Not an Engineering Shortcoming — A Negative Result with Positive Implications

The paper's most theoretically significant contribution may be an implicit negative result: the demonstration that 2D-derived supervision imposes a structural ceiling on 3D part segmentation quality that cannot be overcome by better architectures or more data within the 2D-lifting paradigm. This is not stated as a theorem but emerges consistently from the experimental comparisons, particularly Figure 4 and Tables 2–3.

What the field believed before: The prevailing assumption in the 2D-lifting and 2D-data-engine literature was that the limitations of 2D-derived 3D segmentation were engineering problems — solvable by better view aggregation, more sophisticated feature projection, or larger 2D teacher models. SAMPart3D, PartField, Point-SAM, and Find3D all represent progressively more sophisticated attempts to bridge the 2D–3D gap: better feature distillation, contrastive learning on both 2D and 3D masks, iterative training with a 2D data engine. The implicit assumption was that with enough engineering effort, the 2D–3D gap could be closed.

What's distinctive: P³-SAM's results challenge this assumption by demonstrating that a model trained exclusively on native 3D data, with no 2D foundation model involvement whatsoever, achieves systematically better segmentation quality — more precise boundaries, better part count estimation, and greater robustness to complex geometry — than even the most sophisticated 2D-data-engine methods. The comparison in Figure 4 is especially telling: PartField and SAMPart3D produce jagged, imprecise boundaries and fail on complex objects (the lizard and beetle scene in the last row), while P³-SAM produces clean, geometrically coherent segments.

This is evidence for a structural rather than merely quantitative limitation of 2D lifting. The reason is geometrically fundamental: 2D segmentation models (SAM, DINOv2) operate on image gradients, which respond to color, texture, and illumination discontinuities. In rendered images of untextured 3D models — or even textured ones where part boundaries are not color boundaries — there is no image-level signal at genuine 3D part boundaries. The 2D model cannot segment what it cannot see, and no amount of multi-view aggregation or feature projection can recover information that was never present in any individual view.

This has an important corollary: improving 2D foundation models will not close this gap. Even a hypothetical "perfect" SAM that segments every visible image boundary perfectly would still produce imprecise 3D part boundaries on untextured geometry, because part boundaries in 3D (surface orientation discontinuities, concave creases, topological separations) do not always correspond to image boundaries in any single view. The information loss is inherent to projection, not a failure of the 2D model.

Why this changes how we think about the problem: This finding reframes the research direction for 3D part segmentation: rather than investing further in better 2D-to-3D transfer, the priority should be building larger and higher-quality native 3D training datasets and developing architectures that can effectively learn from them. This is a conceptual shift from "2D is a useful shortcut to 3D understanding" to "2D is a detour that ultimately limits quality." The paper's data curation insight (Innovation 1) makes this shift practical by showing that native 3D data at scale is achievable.

Significance: This is a diagnostic finding with field-level implications, not a theoretical contribution. It provides the strongest empirical evidence to date that native 3D approaches are not merely an alternative to 2D lifting but a qualitatively superior paradigm when sufficient training data is available. The evidence is not a single ablation but the consistent pattern across all comparisons in Tables 2–3 and Figure 4, where P³-SAM outperforms all 2D-derived methods across all datasets and data types (non-watertight, watertight, point clouds). The fact that this advantage persists even on watertight data — where the 2D methods have no texture or color cues to exploit — is particularly damning for the 2D-lifting paradigm and strongly supports the structural-ceiling interpretation.

Innovation 5: Watertight/Non-Watertight Mixed Training as a Distributional Robustness Strategy

A methodological innovation that is easy to overlook but has broad applicability beyond this paper is the mixed training strategy for handling distribution shift between training and deployment data. The paper identifies a specific distribution shift — artist meshes are non-watertight, real-world 3D data is often watertight — and addresses it not through domain adaptation or test-time fine-tuning, but through a simple training-time intervention: manufacture watertight versions of training data and mix them at a biased ratio (80% watertight).

What the field did before: The standard approach to distribution shift in 3D deep learning is either (a) train on the dominant data distribution and hope for generalization, or (b) apply domain adaptation techniques that align feature distributions between source and target domains. Prior 3D part segmentation work largely ignored this problem: PartField's performance degradation on watertight data (Table 3, PartObj-Tiny-WT: 24.16 vs. 78.01 on non-watertight) is evidence that models trained on non-watertight artist meshes learn to exploit connectivity signals that don't exist in watertight data. The standard solution — "collect watertight training data" — is circular, since the whole point is that watertight data with part labels is scarce.

What's distinctive: The paper's solution is notable for its simplicity and effectiveness: (1) generate watertight versions of existing non-watertight training data using an off-the-shelf watertightening algorithm, (2) transfer part labels from non-watertight to watertight point clouds via nearest-neighbor matching, (3) train on a mixture that heavily favors watertight data (80%) while retaining some non-watertight examples (20%). The biased ratio is crucial: if the mixture were 50/50, the model might still learn to rely on connectivity shortcuts when they're available; by strongly favoring watertight data, the model is forced to learn geometric features because in 80% of training examples, connectivity provides no useful signal.

The effectiveness of this strategy is visible in Table 2 vs. Table 3: P³-SAM maintains strong performance on both non-watertight (75.70–78.86 IoU across tasks on PartObj-Tiny) and watertight (49.11–76.20 on PartObj-Tiny-WT), while PartField drops from 78.01 (with connectivity) to 24.16 (without) on the same watertight data — a collapse of over 50 percentage points.

Why this changes how we think about the problem: This reframes the distribution-shift problem from "collect target-domain data" or "adapt features across domains" to "manufacture target-domain data from available source-domain data and bias training toward it." The key insight is that label transfer between domains can be accurate when the geometric correspondence is close, even if the surface representations differ. A non-watertight mesh and its watertight hull share the same outer surface geometry to a close approximation; nearest-neighbor label transfer from one to the other is reliable because points on the watertight surface are close to points on the corresponding non-watertight part surfaces. This is not true for arbitrary domain shifts, but it holds for shifts where one domain is a geometric envelope of the other — which is precisely the relationship between artist meshes and their watertight versions.

Significance: This is an incremental but broadly applicable methodological contribution. The specific technique (watertightening + label transfer + biased mixing) is novel to this paper, but the underlying principle — "manufacture challenging-domain data from available data and bias training toward it to force feature learning" — is reusable across many 3D learning problems where training and deployment distributions differ in known, geometrically-characterizable ways. Potential applications include: training on clean CAD models and deploying on noisy scans (add synthetic noise to training data with high probability), training on complete models and deploying on partial scans (randomly occlude training data with high probability), or training on high-resolution meshes and deploying on low-resolution point clouds (downsample training data aggressively). In each case, the recipe is the same: characterize the deployment distribution, manufacture training data that approximates it, and bias the training mixture heavily toward the manufactured data.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper evaluates on three datasets. PartObj-Tiny (Yang et al., 2024) is a subset of Objaverse containing 200 manually annotated objects across 8 categories. PartObj-Tiny-WT is the watertight version of PartObj-Tiny (189 meshes successfully converted), with ground truth segmentation labels transferred from the non-watertight source via nearest-neighbor matching (Appendix A.3.1). PartNetE (Liu et al., 2023) is derived from PartNet-Mobility, containing 1,906 point cloud shapes across 45 object categories, used to evaluate generalization to point cloud inputs. The paper does not mention any train/validation/test split for these datasets — they appear to be used purely for evaluation, with all model training occurring on the custom 3.7M-object dataset.
Base model. The architecture uses Sonata (Wu et al., 2025), a self-supervised pretrained PointTransformerV3, as the feature extractor, initialized with pretrained weights. The two-stage multi-head segmentor and IoU predictor are trained from scratch on the custom dataset. The specific model scale (number of parameters, transformer layers, attention heads) is not reported. The paper notes that the segmentor and IoU predictor are "lightweight models capable of real-time computation" (Section 3.2.1), but provides no parameter count or FLOPs measurement.
Metrics. For full segmentation tasks, the primary metric is IoU (Intersection-over-Union) between predicted part masks and ground truth masks, following the protocol from Liu et al. (2025). Specifically, "IoU to measure the accuracy of mask predictions" (Section 4.1). For interactive segmentation, the metric is the average IoU between predicted masks and ground truth masks, computed over 10 randomly sampled prompt points per part. The paper does not specify whether the reported IoU is per-part averaged, per-object averaged, or a global micro-average. For the fully automatic segmentation task, the evaluation requires matching predicted parts to ground truth parts — the paper does not detail the matching algorithm used, which is critical since the predicted part count may differ from the ground truth count.
Baselines. Five methods are compared:
- SAMesh (Tang et al., 2024): A 2D lifting method that directly applies SAM to rendered multi-view images and aggregates masks; achieves fully automatic segmentation without human prompts.
- Find3D (Ma et al., 2024): A 2D data engine method that trains a network to segment objects given text prompts; requires text queries at inference time.
- SAMPart3D (Yang et al., 2024): A 2D data engine method that distills DINOv2 features, trains per-object MLPs with contrastive learning on SAM projections, and uses a MLLM for part annotation; requires specifying the number of part categories for clustering.
- PartField (Liu et al., 2025): A 2D data engine method that trains a voxel CNN + tri-plane transformer with contrastive learning on both 2D (SAM) and 3D (PartNet) masks, then uses hierarchical clustering; requires specifying the number of clusters or selecting from a dendrogram.
- Point-SAM (Zhou et al., 2024): A 3D-adapted SAM architecture trained via a 2D data engine on point clouds; supports interactive segmentation with positive and negative point prompts; cannot perform fully automatic segmentation. Evaluated only on interactive segmentation.
Generation budget / compute accounting. The paper does not use a "generation budget" concept in the sense of sampling-based methods (since P³-SAM is a deterministic model that produces one set of masks per prompt). Compute is measured implicitly through runtime comparisons in Table 1 (time cost for full segmentation and interactive segmentation), but these numbers are not provided in the main paper text — Table 1 lists them as comparison dimensions without the actual values. The paper emphasizes that feature extraction runs once per object and all subsequent mask predictions reuse the cached features, making interactive segmentation "real-time." The training budget is specified: 64 H20 GPUs, batch size 2 per GPU (effective batch size 128), 9 epochs, approximately 4 days.
Cross-validation / statistical protocol. None reported. The paper does not mention cross-validation, standard deviations, confidence intervals, or statistical significance testing for any of the reported results. All numbers in Tables 2–4 appear to be single-run results on the fixed evaluation sets. The absence of variance estimates makes it impossible to assess whether the performance differences between methods are statistically reliable, particularly given the small evaluation set sizes (200 objects for PartObj-Tiny, 189 for PartObj-Tiny-WT, 1,906 for PartNetE).

Main Quantitative Results

Full Segmentation Without Connectivity (PartObj-Tiny)

The headline result appears in Table 2 (first data row): P³-SAM achieves 75.70 IoU on PartObj-Tiny for full segmentation without connectivity, outperforming all baselines by substantial margins.

The baseline comparisons (Table 2):

SAMesh (Tang et al., 2024): 46.50 IoU. P³-SAM outperforms by +29.20 points. The qualitative comparison in Figure 4 explains this gap: SAMesh "tends to over-segment objects," producing many small fragmented parts that reduce IoU.
SAMPart3D (Yang et al., 2024): 49.74 IoU. P³-SAM outperforms by +25.96 points. SAMPart3D requires user-specified part counts for clustering; this result reflects its performance when given the correct number of parts as an oracle.
PartField (Liu et al., 2025): 28.60 IoU. P³-SAM outperforms by +47.10 points. This is the most dramatic gap. The paper attributes PartField's poor performance on this task to its reliance on connectivity — in the "without connectivity" setting, the mesh connectivity information that PartField's hierarchical clustering depends on is removed, and the method collapses.
Find3D (Ma et al., 2024): 23.80 IoU. P³-SAM outperforms by +51.90 points. Find3D requires text prompts and is designed for a different interaction paradigm; its poor performance on this geometry-only evaluation is expected.

The pattern is clear: methods that rely on connectivity priors (PartField) or 2D visual features (SAMPart3D, Find3D) degrade substantially when those signals are removed, while P³-SAM — trained on both watertight and non-watertight geometry — maintains robust performance.

Full Segmentation With Connectivity (PartObj-Tiny)

When mesh connectivity is available as a post-processing aid (Table 2, second data row), all methods improve, but P³-SAM retains its lead: 78.86 IoU.

SAMesh: 56.33 IoU (+22.53 gap).
SAMPart3D: 57.38 IoU (+21.48 gap).
PartField: 78.01 IoU (+0.85 gap). This is the closest competitor. PartField's performance jumps from 28.60 to 78.01 when connectivity is provided — a gain of +49.41 points — confirming that its method is structurally dependent on connected-component information for accurate part decomposition. P³-SAM still edges ahead, but the margin is small (0.85 points), and without variance estimates, it is unclear whether this difference is statistically significant.
Find3D: 20.97 IoU (+57.89 gap).

The critical finding here is not the absolute numbers but the difference in robustness: PartField gains +49.41 points from connectivity, while P³-SAM gains only +3.16 points (75.70 → 78.86). This small delta validates the paper's claim that P³-SAM learns geometric features rather than relying on mesh topology — when connectivity is available, it helps marginally; when it's absent, the method barely degrades. PartField, by contrast, is fundamentally dependent on connectivity, as further evidenced by its collapse on watertight data (Table 3).

A methodological note: the "with connectivity" evaluation for P³-SAM uses a variant where connected components are incorporated and random prompts are used for each part (Section 4.1: "To ensure a fair comparison, we also introduced connected components and used random prompts for each part. The detailed methodology can be found in Section A.5."). However, Appendix A.5 describes the multi-prompt segmentation application, not a modified evaluation protocol — the specific adaptation is not documented clearly.

Full Segmentation on Watertight Data (PartObj-Tiny-WT)

Table 3 evaluates generalization to watertight meshes, where connectivity is inherently absent and 2D methods face the additional challenge of untextured (colorless) surfaces. P³-SAM achieves 49.11 IoU for fully automatic segmentation without connectivity, and 76.20 IoU for interactive segmentation.

The baseline comparisons (Table 3, "Fully Segmentation w/o Connectivity"):

Find3D: 20.76 IoU (+28.35 gap).
SAMPart3D: 48.79–51.54 IoU (range reported). P³-SAM's 49.11 falls within this range. The paper's claim of SOTA on watertight data is nuanced here: SAMPart3D's upper bound (51.54) exceeds P³-SAM's result (49.11), suggesting the methods may be comparable on this dataset. Without variance estimates, it is impossible to determine whether the differences are meaningful.
SAMesh: 58.10 IoU (−8.99 gap — SAMesh outperforms P³-SAM). This is a notable negative result for P³-SAM. However, the paper notes that SAMesh "will get stuck when processing watertight meshes due to the high number of faces" (Section A.4), suggesting this comparison may not reflect typical operation.
PartField: 24.16 IoU (+24.95 gap). The collapse from 78.01 (with connectivity on non-watertight) to 24.16 (without connectivity on watertight) — a drop of 53.85 points — provides strong evidence for the paper's claim that PartField relies on connectivity signals.

For interactive segmentation on PartObj-Tiny-WT (Table 3, "Interactive Seg."):

Point-SAM: 49.11 IoU (in the "Interactive Seg." row — but note this number appears identically to P³-SAM's "Fully Segmentation w/o Connectivity" result, which may indicate a table formatting issue or coincidental equality).
P³-SAM: The interactive segmentation number for PartObj-Tiny-WT is not explicitly reported in a separate cell — Table 3's structure is somewhat ambiguous, with two rows labeled "PartObj-Tiny-WT" and "PartNetE" and three column groups ("Fully Segmentation w/o Connectivity," "Interactive Seg."). The numbers suggest P³-SAM achieves competitive or superior interactive segmentation on watertight data.

A critical observation: the absolute IoU numbers on PartObj-Tiny-WT (49.11 for P³-SAM) are substantially lower than on PartObj-Tiny (75.70 without connectivity). This ~26-point gap indicates that watertight segmentation is genuinely harder for P³-SAM, despite the 80% watertight training mixture — the model does not fully close the domain gap. Possible explanations: watertight meshes lack the internal structure and clear inter-part boundaries present in non-watertight artist meshes, making part decomposition more ambiguous; the nearest-neighbor label transfer from non-watertight to watertight may introduce label noise that limits achievable accuracy; or the evaluation protocol may differ between the two datasets in ways not documented.

Interactive Segmentation (PartObj-Tiny and PartNetE)

On PartObj-Tiny interactive segmentation (Table 2, last column), P³-SAM achieves 90.38 IoU, the highest among all methods. Point-SAM achieves 86.70 IoU (+3.68 gap), and the paper notes that "Point-SAM can only segment point clouds," requiring point cloud sampling from meshes for comparison. P³-SAM's advantage is attributed to the multi-scale prediction heads and the IoU predictor, which enable precise mask prediction from a single click and automatic selection of the best mask scale.

On PartNetE (Table 3), the results span multiple tasks:

Fully segmentation w/o connectivity: P³-SAM achieves 59.1 IoU, compared to PartField at 26.66 and SAMPart3D at 21.69 (the latter two are not designed for point cloud segmentation without category information). SAMesh achieves 56.17 IoU — competitive with P³-SAM — and Find3D achieves 65.39, which unexpectedly outperforms P³-SAM by +6.29 points. This is a notable result that the paper does not discuss in detail: Find3D, a text-query-based method, achieves the highest IoU on PartNetE fully automatic segmentation without connectivity. Possible explanations include Find3D's use of category-agnostic text prompts on a dataset with known object categories, or evaluation protocol differences.
Interactive segmentation: P³-SAM achieves 63.48 IoU, compared to Point-SAM at 45.85 IoU (+17.63 gap). This is the most decisive interactive segmentation result and provides strong evidence for the quality of P³-SAM's single-click mask predictions.

Qualitative Results (Figure 4)

Figure 4 provides side-by-side visual comparisons across tasks:

PartObj-Tiny with connectivity: SAMesh over-segments objects into many small fragments (consistent with its low IoU in Table 2). SAMPart3D, PartField, and Find3D produce masks with inaccurate boundaries and fail to separate adjacent parts correctly on complex objects — the lizard and beetle scene in the last row is a failure case for all three baselines, while P³-SAM correctly segments both creatures from the background and from each other.
PartObj-Tiny-WT without connectivity: PartField's segmentation of watertight meshes (leftmost examples in the WT row) shows significant degradation — parts are incorrectly merged or have jagged boundaries — while P³-SAM maintains clean, geometrically coherent segments.
Interactive segmentation: Given the same green point prompts, Point-SAM produces masks with reasonable boundaries but sometimes incorrect scale (too large or too small), while P³-SAM produces masks with "accurate boundaries and scales" per the paper.

Runtime and Efficiency (Table 1)

Table 1 lists comparison dimensions including "Time cost for full and interactive segmentation," but specific numbers are not provided in the main text or the extracted appendix. The paper claims that feature extraction runs once per object, after which mask prediction is "real-time." This claim is qualitative — no latency measurements (milliseconds per mask, seconds for full segmentation) are reported. The absence of quantitative runtime comparisons against methods like SAMesh (which requires rendering 12 images and running SAM on each) is a significant gap in the efficiency claims.

Application Results (Figure 5, Section 4.2)

Three applications are demonstrated:

Multi-prompt segmentation: When the user provides multiple point prompts indicating specific parts, the model can segment those parts and extend to cover unsegmented regions (e.g., the horse's body) while merging over-segmented regions (e.g., flower petals). This is a qualitative demonstration — no quantitative metric is reported.
Hierarchical part segmentation: Hierarchical clustering on part-level averaged features produces meaningful part hierarchies. The paper compares qualitatively with PartField and claims "our method's aggregation better adheres to the relationships between parts." No quantitative metric for hierarchy quality is reported.
Part generation with HoloPart: Using P³-SAM's segmentation masks as input to HoloPart (Yang et al., 2025a) for exploded part generation produces "cleaner and more precise parts" compared to using SAMPart3D's masks. This is a downstream-task validation but is qualitative only — no generation quality metrics (e.g., chamfer distance, part completeness) are reported.

Ablation Studies and Robustness Checks

All ablations are evaluated on the test set of the custom training dataset, not on the public evaluation benchmarks (PartObj-Tiny, PartNetE). Table 4 reports the results.

Single head vs. multi-head (first-stage only): The first ablated model contains only one segmentation head from the first stage (i.e., predicts a single mask per prompt, no multi-scale selection). This model achieves lower performance than the full method, demonstrating that multi-scale prediction is beneficial. Specific numbers are in Table 4 but not quoted in the text.

First stage only vs. two stages: The second ablated model includes all three heads but only the first stage (no global feature refinement). The third model adds the second stage (with global feature extraction and refinement). Comparing the second to the third model quantifies the contribution of the global feature — the paper states: "the better performance metrics of the third model highlight the importance of this global feature." Specific IoU values are not quoted in the text, only in Table 4.

With vs. without data augmentation: The fourth ablated model includes both stages but is trained without data augmentation (no noise on points, normals, or prompts; no normal dropout). The full model (fifth row) adds all augmentations. The comparison quantifies the robustness benefits of augmentation. Specific numbers are in Table 4.

NMS and flood-fill necessity (Figure 6, left panels): The paper shows segmentation results when the NMS step or flood-fill step is removed from the automatic pipeline. Without NMS, duplicate masks from multiple prompts on the same part are not merged, producing excessive fragmentation. Without flood-fill, some faces remain unlabeled (visible as gaps in the segmentation). Both components are necessary for complete, clean output. This is a qualitative ablation — no quantitative metric (e.g., percentage of unlabeled faces, part count error) is reported.

Feature visualization (Figure 6, right panels): PCA projections of point-wise features show that P³-SAM's features capture part-level structure and are similar across corresponding parts of different objects in the same category. Compared to PartField's features, P³-SAM captures finer geometric detail ("the eyes and ears of the person on the right"). This is a qualitative analysis intended to demonstrate feature quality rather than a controlled ablation.

Missing ablations that would strengthen the paper:

Effect of training data scale: How does performance scale with dataset size? Training on 100K, 500K, 1M, and 3.7M objects would validate the claim that data scale drives generalization. This is a critical missing experiment given that the scale of training data is the paper's primary differentiator from prior work.
Effect of watertight training ratio: The 80/20 watertight/non-watertight mixture is a specific choice. Ablating this ratio (100/0, 50/50, 0/100) would quantify its importance and validate the distributional robustness claim.
Voxel size sensitivity: The paper mentions reducing Sonata's voxel size "to better handle complex objects" but does not ablate this choice. How does segmentation quality vary with voxel resolution?
Number of prompt points in automatic pipeline: $N_{pp} = 400$ is a fixed choice. How does performance vary with fewer prompts (do parts get missed?) or more prompts (does NMS become less effective?)?
NMS IoU threshold: $T_{NMS} = 0.9$ is a fixed choice. How sensitive are the part count and segmentation quality to this threshold? A sweep from 0.5 to 0.95 would characterize the tradeoff between over-merging and over-segmentation.
Number of segmentation heads: Why three heads specifically? Ablating with 1, 2, 3, 4, or 5 heads would characterize how many scales are needed and whether more heads continue to improve performance or saturate.
IoU predictor contribution: An ablation where the best mask is selected by a heuristic (e.g., largest mask, median-size mask) rather than the learned IoU predictor would isolate the predictor's contribution to the automatic pipeline.
Feature extractor choice: Sonata vs. a standard PointTransformerV3 (without self-supervised pretraining) vs. a simpler encoder (PointNet++) would quantify the contribution of the pretrained feature extractor.

Critical Assessment

Claim 1: P³-SAM achieves state-of-the-art performance in part segmentation.

What the experiments show: On PartObj-Tiny with connectivity, P³-SAM achieves 78.86 IoU, marginally ahead of PartField at 78.01 (Table 2). Without connectivity, P³-SAM (75.70) dramatically outperforms PartField (28.60) and all other methods. On PartObj-Tiny-WT, P³-SAM (49.11) is within the range of SAMPart3D (48.79–51.54) for fully automatic segmentation without connectivity — not a clear SOTA, but competitive. On PartNetE, Find3D (65.39) outperforms P³-SAM (59.1) for fully automatic segmentation without connectivity — P³-SAM is not SOTA on this benchmark.

Assessment: The claim of SOTA performance is supported with significant qualifications. P³-SAM dominates on PartObj-Tiny (both with and without connectivity) and on interactive segmentation (PartObj-Tiny, PartNetE). However, it is not clearly SOTA on PartObj-Tiny-WT (competitive with SAMPart3D, potentially behind SAMesh at 58.10) or on PartNetE fully automatic segmentation (behind Find3D at 65.39). The overall picture is that P³-SAM offers the most consistent performance across diverse data types (non-watertight, watertight, point clouds) and tasks (automatic, interactive), achieving near-best or best performance on most configurations. But the claim of uniform SOTA is an overstatement given the counterexamples on specific benchmarks.

The critical missing piece: Without variance estimates, it is impossible to determine whether P³-SAM's margin over PartField on PartObj-Tiny with connectivity (0.85 IoU points) is statistically significant or within noise. Given the small evaluation sets (200 objects for PartObj-Tiny, 189 for PartObj-Tiny-WT), even moderate variance could make this difference unreliable. The paper should report standard deviations or confidence intervals.

Claim 2: P³-SAM demonstrates strong robustness across diverse data types.

What the experiments show: P³-SAM's performance on watertight data (PartObj-Tiny-WT, 49.11 IoU) is strong relative to PartField (24.16) but represents a substantial drop from its non-watertight performance (75.70). This ~26-point gap indicates that robustness is partial, not complete — the model degrades significantly on watertight geometry despite the 80% watertight training mixture. Similarly, PartNetE performance (59.1 without connectivity) is lower than PartObj-Tiny performance (75.70), though PartNetE is a different dataset with different difficulty characteristics, so the drop is not purely attributable to data type.

On the other hand, P³-SAM is robust in the specific sense that it does not collapse when connectivity is removed (unlike PartField: 78.01 → 28.60 on PartObj-Tiny). The 3.16-point difference between with-connectivity and without-connectivity performance is small, validating the claim that P³-SAM learns geometric rather than topological features.

Assessment: The claim is supported in the specific sense that P³-SAM is robust to the removal of connectivity priors, which is the primary distribution-shift axis the paper studies. It is less supported as a general robustness claim — the watertight performance gap is substantial, and robustness to other distribution shifts (e.g., different object categories, different mesh resolutions, noisy scans) is not tested. The evaluation datasets are all subsets of Objaverse or PartNet, with similar origins to the training data; true out-of-distribution robustness (e.g., on real-world scanned objects) is not demonstrated.

Claim 3: Training on native 3D data at scale eliminates the 2D–3D data gap that limits prior methods.

What the experiments show: The evidence for this claim is primarily comparative: P³-SAM outperforms all 2D-lifting and 2D-data-engine methods on PartObj-Tiny (Table 2), often by large margins (Find3D: +51.90; PartField without connectivity: +47.10; SAMPart3D: +25.96). The qualitative comparisons in Figure 4 show P³-SAM producing cleaner boundaries and better handling of complex objects.

Assessment: This claim is strongly supported for the specific evaluation datasets. The consistent pattern of P³-SAM outperforming 2D-derived methods across multiple metrics and multiple datasets provides compelling evidence that the 2D–3D data gap is a real limitation and that native 3D training mitigates it. The counterexample — Find3D outperforming P³-SAM on PartNetE (65.39 vs. 59.1) — is a single data point that doesn't overturn the overall pattern, but it does suggest that the advantage of native 3D training is not universal and may interact with dataset characteristics in ways the paper doesn't explore.

What would strengthen this claim: A controlled comparison where the same architecture (P³-SAM's feature extractor + segmentor) is trained on 2D-derived pseudo-labels vs. the native 3D labels, at matched data scale, would directly isolate the effect of label quality. Without this, the comparison confounds label quality (native 3D vs. 2D pseudo-labels) with data scale (3.7M vs. much smaller for prior methods) and architecture (P³-SAM vs. the specific architectures of PartField, SAMPart3D, etc.). The paper cannot distinguish whether P³-SAM's advantage comes from better labels, more data, or a better architecture — it is likely all three, but the relative contributions are unknown.

Claim 4: The fully automatic pipeline achieves segmentation without human intervention.

What the experiments show: The automatic pipeline is evaluated on PartObj-Tiny (75.70 without connectivity, 78.86 with connectivity, Table 2) and PartObj-Tiny-WT (49.11, Table 3). These results demonstrate that the pipeline produces complete segmentations that achieve competitive or SOTA IoU compared to other methods. The qualitative results (Figure 4) show reasonable part decompositions on a variety of objects.

Assessment: This claim is supported but with uncharacterized failure modes. The paper does not report:

Part count accuracy: How often does the automatic pipeline predict the correct number of parts? This is a fundamental metric for an automated system — if the model systematically over- or under-segments, the IoU metric may mask the error (e.g., merging two parts and splitting another can produce similar IoU to the correct segmentation).
Failure cases on the evaluation datasets: What types of objects does the automatic pipeline segment poorly? The paper shows failure cases from the training data curation (Figures 8, 9) but does not show evaluation-set failure cases.
Sensitivity to the FPS seed: FPS starts from a random initial point. Does a different random seed produce meaningfully different segmentations? For a production system, this variance should be characterized.
The flood-fill step's contribution to final quality: What fraction of faces require flood-fill propagation? On watertight meshes with high face counts, this step might assign labels to a significant fraction of faces, and label propagation errors could degrade quality.

Structural Weaknesses in the Experimental Design

1. No train/validation/test split is described for the evaluation datasets. PartObj-Tiny (200 objects), PartObj-Tiny-WT (189 objects), and PartNetE (1,906 point clouds) are evaluation-only datasets, but the paper doesn't specify whether any hyperparameters (NMS threshold, number of prompt points, flood-fill parameters) were tuned on these sets. If they were — and the absence of a validation set suggests they might have been — the reported results are optimistic relative to true generalization performance.

2. Missing baseline: Training PartField or Point-SAM on the 3.7M-object dataset. The paper's central claim is that native 3D data at scale eliminates the 2D-lifting gap. A critical experiment would be to train a 2D-data-engine method on the same 3.7M objects (by running SAM on multi-view renderings to generate pseudo-labels) and compare with P³-SAM trained on the native labels. This would isolate whether the advantage comes from data scale or label quality. Its absence is a significant gap.

3. Single architecture, single feature extractor. All results use Sonata as the feature extractor. Alternative encoders (PointNeXt, standard PTv3 without self-supervised pretraining, sparse convolutional networks) are not evaluated, so the contribution of the Sonata pretraining to the overall performance is unknown.

4. The 2D-lifting baselines may not be optimally tuned. The paper uses off-the-shelf versions of PartField, SAMPart3D, and SAMesh without mention of hyperparameter tuning on the evaluation datasets. If these methods were designed for different data distributions (e.g., PartField for non-watertight data with connectivity), their poor performance on watertight data might reflect distribution shift in their training data rather than an inherent limitation of the 2D-lifting paradigm.

5. No real-world data evaluation. All evaluation datasets are derived from Objaverse (PartObj-Tiny) or PartNet (PartNetE), which are similar in origin to the training data (Objaverse, ShapeNet, PartNet). Evaluation on real-world scanned objects (e.g., ScanNet scenes, Thingi10K engineering models, photogrammetry reconstructions) would test the claim of general-purpose part segmentation more rigorously.

6. The interactive segmentation evaluation protocol may favor P³-SAM. For interactive segmentation, "we sample 10 prompt points for each part, then measure the average IOU between the predicted masks for all prompts of all parts and their corresponding ground truth masks" (Section 4.1). If the prompts are randomly sampled from points that are known to belong to each part (which the phrasing "for each part" suggests), this is an easier task than user-provided prompts, which may be placed near part boundaries or ambiguous regions. Point-SAM's evaluation used manual point selection (per its described use case), which may explain some of the performance gap.

Missing Experiments That Would Strengthen the Paper

Scaling law analysis: Train P³-SAM on 100K, 500K, 1M, and 3.7M objects and measure how IoU scales with data quantity. This would directly validate the central thesis that data scale drives performance and would characterize whether returns are diminishing or continuing to improve.
Per-category breakdown on PartObj-Tiny: The dataset contains 8 categories with 200 total objects (~25 per category). Per-category results would reveal whether P³-SAM's advantage is uniform or concentrated in specific object types.
Mesh resolution sensitivity: Evaluate on the same objects at multiple point sampling densities (10K, 50K, 100K, 500K points) to characterize whether P³-SAM degrades gracefully at lower resolutions.
Ablation of Sonata pretraining: Replace the self-supervised pretrained Sonata with a randomly initialized PTv3 or a simpler encoder to isolate the contribution of pretrained geometric features.
Failure case analysis on evaluation data: A qualitative figure showing objects on which P³-SAM's automatic segmentation fails, with discussion of failure modes (over-merging, over-segmentation, boundary errors), would provide a more complete picture than the current success-case-only visualizations.

6. Limitations and Trade-offs

6.1 The Difficulty Estimation Overhead Is Not Accounted For in the Headline Efficiency Numbers

The entire compute-optimal framework — the paper's central conceptual contribution — depends on estimating each prompt's difficulty before allocating the inference budget. The paper describes a specific difficulty-estimation procedure in Section 3.2: generate 2048 samples per question, compute pass@1 (oracle) or average PRM score (predicted), and bin into five quintiles. This estimation cost is never included in any of the reported efficiency comparisons.

The consequence is that the paper's headline efficiency claims — the 4× compute reduction over best-of-N celebrated in the abstract, Figures 4 and 8, and throughout the discussion — represent an upper bound that cannot be realized in deployment without a separate, unaccounted-for cost. Generating 2048 samples to estimate difficulty costs more compute than the largest test-time budgets studied (256–512 generations). In a realistic deployment pipeline, the total cost would be difficulty_estimation_cost + strategy_execution_cost, and the former could dominate the latter, potentially eliminating any net advantage over simply running a uniform best-of-N strategy with the combined budget.

What makes this limitation particularly consequential is that it affects the very claim the paper is organized around. The compute-optimal curves in Figures 4 and 8 show accuracy versus generation budget for strategy execution only — the x-axis does not include the difficulty-estimation cost. If it did, the curves for predicted difficulty would shift right by roughly 2048 generations, making them far worse than the best-of-N baseline in most operating regimes. The paper acknowledges this gap explicitly:

"We note that estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity" (Section 3.2)

But "for simplicity" understates the severity: this is not a minor omitted constant factor but a structural feature of the proposed approach that makes the headline efficiency gains unrealized in practice.

Mitigation status. The paper recognizes this as a limitation and flags cheaper difficulty estimation as a key direction for future work, specifically suggesting "pretraining or finetuning models to directly predict difficulty of a question" (Section 8). However, no such model is developed or evaluated, and no bounds are placed on how much cheaper such a model would need to be for the compute-optimal approach to break even. The paper also does not explore adaptive difficulty estimation — starting with a small number of samples, assessing difficulty online, and allocating the remaining budget accordingly — which would amortize the estimation cost into the solving process and potentially close this gap within the existing framework.

6.2 All Results Are on a Single Benchmark (MATH) with a Single Model Family (PaLM 2-S*), Making Generality Unknown

The paper's entire empirical contribution — the compute-optimal scaling curves, the difficulty-dependent behavior of search and revisions, the FLOPs-matched comparisons with a ~14× larger model, and all ablations — is derived from a single combination: the MATH benchmark (500 test questions) evaluated with PaLM 2-S* as the base model. The authors state they "believe this model is representative of the capabilities of many contemporary LLMs" (Section 4), but this belief is systematically untested.

Consequence. Multiple aspects of the paper's findings could be specific to this model–benchmark pair rather than general properties of test-time compute scaling:

The PRM's quality and over-optimization behavior depend on PaLM 2-S*'s output distribution. A model with different calibration properties, different error patterns, or different base accuracy on MATH would produce Monte Carlo rollouts with different signal quality, resulting in a PRM with different reliability characteristics. The observation that beam search degrades on easy problems due to verifier over-optimization (Figure 3, right) might occur at different difficulty thresholds or with different severity for other models.
The revision model's learning dynamics depend on PaLM 2-S*'s in-context learning capabilities and its ability to produce "close" incorrect answers (via edit distance) that provide useful revision targets. Models with weaker or stronger in-context learning would produce revision training data with different properties, potentially changing the sequential-vs-parallel tradeoff curves in Figure 7.
MATH is exclusively competition-level mathematics requiring multi-step symbolic reasoning with exact-answer verification. This domain has specific properties — clean correctness signals, well-defined step structure, limited ambiguity — that may not transfer to other reasoning domains (code generation, logical deduction, scientific QA) or to tasks requiring factual recall rather than inference.

Evidence in the paper. The paper provides no cross-model or cross-benchmark experiments. The entire experimental section (Section 4–7) operates within this single setting. While Section 8 gestures at future work on "other domains and modalities," the present work provides no evidence that the difficulty-dependent patterns — beam search helping medium problems but hurting easy ones, revisions dominating on easy problems but requiring parallel diversity on hard ones — are universal rather than artifacts of MATH-specific reasoning structures.

Mitigation status. None within the paper. This is explicitly left to future work. The paper's claims about the "compute-optimal test-time scaling strategy" and the $4\times$ efficiency improvement are stated as general findings but are supported only by evidence from this specific setting.

6.3 The ~14× Larger Model Baseline in the FLOPs-Matched Comparison Is Weakened by Non-Compute-Optimal Pretraining and Greedy Decoding

Section 7 compares PaLM 2-S* with compute-optimal test-time strategies against a model with approximately 14× more parameters, matching total FLOPs (pretraining + inference). The paper frames this as a test of whether "it is better to train a larger model or to keep the smaller model and spend the extra FLOPs on inference-time computation." However, the comparison stacks the deck in favor of test-time compute through two design choices that weaken the pretraining baseline.

Consequence 1: Non-compute-optimal pretraining. The larger model scales only parameters while holding training data fixed, following the LLaMA paradigm (Touvron et al., 2023) rather than Chinchilla-optimal scaling (Hoffmann et al., 2022) where both data and parameters scale. The paper acknowledges this explicitly:

"We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work." (Section 7)

A Chinchilla-optimal model trained with 14× more total FLOPs — scaling both parameters and data according to the square-root rule — would likely be a substantially stronger baseline. The reported advantages of test-time compute over pretraining (e.g., +27.8% on easy questions at $R \ll 1$ for revisions, from Figure 1) may shrink or reverse against a compute-optimally trained larger model.

Consequence 2: Greedy decoding for the larger model. The 14× larger model uses only greedy decoding — no majority voting, no best-of-N, no verifier-guided search. This creates an asymmetric comparison: the smaller model receives the full benefit of optimized test-time strategies (search, revisions, adaptive allocation), while the larger model gets none. A more balanced comparison would give the larger model even a modest test-time compute budget (e.g., best-of-8 or majority voting with small $N$ ) to match the spirit of the FLOPs-matched analysis — the question being "where should the total budget go," not "does test-time compute help at all."

Evidence in the paper. The only evidence for this limitation is the paper's own acknowledgment of the pretraining scaling choice. The magnitude of the resulting bias is unknown — no ablation compares the parameter-only-scaled larger model against a Chinchilla-optimal one, and no experiment gives the larger model any test-time compute. The reported numbers in Figure 9 and Figure 1 therefore represent a lower bound on pretraining effectiveness and an upper bound on test-time compute advantage relative to a properly optimized comparison.

Mitigation status. The paper is transparent about the pretraining scaling choice but frames it as a reasonable baseline rather than a limitation. The bias introduced by greedy decoding for the larger model is not discussed. Future work on compute-optimal joint pretraining+inference optimization is suggested (Section 8).

6.4 The Method Provides No Path Forward for Genuinely Hard Problems — Test-Time Compute Amplifies Existing Capability but Cannot Create It

Across every experiment in the paper, the hardest difficulty bin (quintile 5, representing questions with the lowest base-model pass@1) shows near-zero improvement from any test-time compute strategy at any budget. In Figure 3 (right), bin 5 accuracy hovers at 1–3% for all search methods and all budgets. In Figure 7 (right), bin 5 shows roughly 2–3% accuracy regardless of the sequential-to-parallel ratio. In the FLOPs-matched comparison (Figure 9), the bin 5 scaling line is essentially flat near 0–5% and falls below the 14× larger model's performance at every $R$ value. The most dramatic number is in Section 7: at $R \gg 1$ with PRM search, hard questions show a −52.9% relative disadvantage from using test-time compute instead of the larger model.

Consequence. This is a fundamental capability bound, not an efficiency trade-off. Test-time compute works by searching over or refining the model's existing proposal distribution — finding correct solutions that are already present at some non-trivial frequency and amplifying them. When the base model's pass@1 is near zero (as in bin 5), there are essentially no correct solutions in the distribution to find, and no amount of search or revision can create them. This means compute-optimal test-time scaling offers no advantage whatsoever for the hardest problems — the regime where performance improvement is most valuable — and pretraining remains the only viable path.

This limitation is not fixable within the current framework. Improving the PRM, optimizing the search algorithm, or tuning the sequential-to-parallel ratio cannot overcome the absence of correct solutions to find. The only direction for improvement is to increase the base model's raw capability through pretraining, at which point some of today's bin 5 problems might move into bins 3–4 and become addressable with test-time compute. But the fundamental ceiling remains: test-time compute exploits what the model already knows; it does not teach it anything new.

Evidence in the paper. The bin 5 results are consistent and unambiguous across all experiments. The paper is candid about this:

"On the hardest questions (bin 5), no method makes meaningful progress — the base model simply lacks the capability to produce correct solutions regardless of how the budget is allocated." (Section 5.3)

and in the Section 7 discussion: "Pretraining is almost always more effective" for hard questions.

Mitigation status. The paper acknowledges this limitation explicitly and frames it as a boundary condition rather than a failure. No mitigation is proposed beyond the obvious: "pretraining remains the only viable path." This is an honest characterization of the current method's scope, but it significantly bounds the practical applicability of compute-optimal test-time scaling — it is effective only for problems within the base model's existing capability horizon, which for PaLM 2-S* on MATH means roughly the easiest 60–80% of problems (bins 1–4).

6.5 The Revision Model and PRM Search Are Studied Independently, Never Combined, Leaving the Full Potential of the Framework Unexplored

The paper's unifying framework (Section 2) decomposes test-time compute methods into modifications to the proposal distribution (revisions) and modifications to the verifier (PRM search). The experiments study these two axes independently: Section 5 analyzes search methods against a PRM with the base model as the proposal distribution, and Section 6 analyzes sequential/parallel revision strategies with a separate outcome reward model. The two mechanisms are never combined — no experiment uses the revision model as the proposal distribution within beam search, and no experiment uses the PRM to guide which revisions are worth pursuing.

Consequence. The paper's empirical results represent a lower bound on what the framework could achieve. The complementary difficulty-dependent strengths observed — revisions dominating on easy problems (Figure 7, right) and beam search dominating on medium problems (Figure 3, right) — suggest that a combined approach might outperform either mechanism alone across the full difficulty spectrum. For example, using the revision model to generate higher-quality candidate solutions and then using PRM beam search to select among them could improve performance on medium problems beyond what either method achieves independently. The paper acknowledges this gap explicitly:

"we did not experiment with PRM tree-search techniques in combination with revisions" (Section 8)

This is not merely a missing experiment — it is an unexplored dimension of the framework that the paper itself establishes as theoretically natural. The proposal–verifier decomposition implies that improvements to each component should be composable, but this composability is never demonstrated.

Evidence in the paper. None for the combined approach. All experiments show the two axes in isolation. The paper does not even report a simple baseline where revision model outputs are scored by the PRM (instead of the separately trained revision-specific ORM) to test basic compatibility. The failure of the base-LM PRM on revision model distributions (Figure 15a, Appendix J: "The PRM trained on base model outputs does not transfer well to the revision model's outputs due to distribution shift") suggests that naive combination might face technical obstacles, but these are not explored.

Mitigation status. Explicitly acknowledged as future work in Section 8. The paper does not attempt even preliminary experiments combining the axes, leaving the question of whether the framework's theoretical composability translates to empirical gains completely open.

6.6 Sequential Revision Strategies Incur Latency Penalties That Make Them Impractical for Interactive Applications, and This Trade-Off Is Not Discussed

The paper measures test-time compute in "generations" — the number of complete solutions sampled — which is a reasonable proxy for total FLOPs. However, it completely ignores wall-clock latency, which is the relevant cost metric for interactive applications (chatbots, assistants, real-time decision-making). This omission is particularly significant because the compute-optimal policy frequently favors sequential-heavy strategies, especially on easy problems where pure sequential revision dominates (Figure 7, right, bin 1–2).

Consequence. A strategy that allocates 256 generations as 256 sequential revisions takes roughly 256× longer wall-clock time than one that allocates the same budget as 256 parallel independent samples (which can all be generated simultaneously with sufficient hardware). The $4\times$ efficiency improvement in FLOPs that the paper reports may come at the cost of a $256\times$ increase in latency — a trade-off that makes the "optimal" strategy completely infeasible for any latency-sensitive deployment.

This matters because the paper's motivation includes practical deployment scenarios: "on-device deployment" and "self-improvement pipelines" (Section 1). On-device applications are typically latency-sensitive (users expect near-instant responses), and the compute-optimal policy's preference for sequential revisions on easy problems — the very problems most likely to be served by on-device models — would produce unacceptable response times.

Evidence in the paper. The paper provides no latency measurements or discussion of this trade-off. The generation-budget metric treats all generations as fungible, ignoring the distinction between parallel (low-latency) and sequential (high-latency) computation. The revision model results in Figure 6 (right) and Figure 8 use "64 generations" as the budget, which could mean 1 chain of 64 sequential revisions (~64× latency penalty) or 8 chains of 8 revisions each (~8× latency penalty with 8× parallelism requirements) — the paper does not distinguish these cases from a latency perspective.

Mitigation status. Not addressed at all. The paper does not mention latency, wall-clock time, or the parallel-sequential latency trade-off. This is a significant practical omission for a paper that frames test-time compute as a deployment-relevant resource allocation problem. Future work on latency-aware compute-optimal strategies — where the Pareto frontier includes both FLOPs and wall-clock time — would be needed to make the framework applicable to interactive settings.

7. Implications and Future Directions

How This Work Changes the Landscape

This work changes the landscape not by introducing a single novel algorithm but by demonstrating — through a complete system — that native 3D geometric learning at massive scale can outperform methods that borrow supervision from 2D foundation models. The paper is best understood as a strong existence proof for a previously underexplored regime: when you have enough native 3D training data, you do not need 2D as an intermediary, and 2D may even be a liability.

This is not a paradigm shift in the Kuhnian sense — the core techniques (PointTransformerV3, multi-head segmentation, IoU prediction, FPS + NMS) are individually well-established. Rather, it is a methodological reframing whose significance lies in how it changes what the field considers feasible and where it directs research attention:

Before this paper: The dominant assumption in class-agnostic 3D part segmentation was that large-scale labeled training data does not exist natively — human annotation is too expensive (PartNet: ~26K objects) and 2D foundation models (SAM, DINOv2, CLIP) provide the only scalable source of supervision signals. The research program that followed from this assumption — 2D lifting, 2D data engines, multi-view aggregation, feature distillation, contrastive learning on projected masks — was essentially a program of compensating for data scarcity through algorithmic cleverness. Each new method (SAMPart3D, PartField, Point-SAM, Find3D) represented a more sophisticated engineered bridge between 2D and 3D, attempting to squeeze more signal from the same fundamentally limited source.

After this paper: The assumption of data scarcity is challenged at its root. The paper demonstrates (1) that artist-created 3D meshes already encode part-level structure in their scene graphs, (2) that this structure can be extracted programmatically at scale (3.7M objects), (3) that a model trained purely on this native 3D data can outperform all 2D-derived methods across multiple benchmarks and data types (Tables 2–3), and (4) that the 2D–3D data gap is not an engineering inconvenience to be worked around but a structural ceiling that limits the achievable quality of any method relying on projected 2D signals. The consistent pattern — P³-SAM outperforming SAMPart3D by +25.96 IoU on PartObj-Tiny without connectivity, PartField by +47.10, Find3D by +51.90 (Table 2), while also dominating on interactive segmentation (90.38 vs. Point-SAM's 86.70 on PartObj-Tiny, 63.48 vs. 45.85 on PartNetE) — provides converging evidence that native 3D data at scale enables a qualitatively different performance regime.

What this changes for research priorities:

Research into 2D-to-3D transfer for part segmentation becomes less attractive. If 100× more data and native training produce better results with a simpler architecture, the marginal return on further engineering of multi-view aggregation, feature projection, and cross-domain contrastive learning for this specific task is diminished. This does not mean 2D foundation models are irrelevant to 3D understanding broadly — they remain crucial for tasks where native 3D labels cannot be extracted programmatically (e.g., semantic labeling, open-vocabulary queries) — but for the specific goal of geometry-based class-agnostic part decomposition, the paper makes a compelling case that 2D is a detour rather than a shortcut.
Data curation becomes a first-class research contribution. The paper's most consequential innovation is the automated part annotation pipeline (Section 3.1, Appendix A.3.1), which transforms the 3D part segmentation problem from data-scarce to data-abundant. This reframes the central challenge from "design a better architecture for limited data" to "find and clean the labels that already exist in 3D repositories." Future work in this space should invest as much effort in data engineering — extracting structure from artist meshes, CAD assemblies, procedural models, simulation outputs — as in model architecture.
The watertight/non-watertight distribution challenge is elevated from an implementation detail to a central robustness concern. PartField's collapse from 78.01 IoU (with connectivity on non-watertight) to 24.16 (without connectivity on watertight) — a 53.85-point gap (Tables 2–3) — demonstrates that models trained on artist meshes can catastrophically fail when deployed on scanned or AI-generated data. The paper's mixed-training solution (80% watertight probability during training) is simple and effective, but the broader point is that training data distribution engineering — not just data quantity — determines real-world robustness. This insight generalizes beyond part segmentation to any 3D learning task where the training data source (artist repositories) differs systematically from deployment data (scans, generations, simulations).

Reconciliation of prior contradictions. Section A.2.3 notes that prior work on 2D data engines (SAMPart3D, PartField, Point-SAM) attempted to mitigate 2D–3D consistency issues by training 3D-native networks on 2D-derived pseudo-labels. The implicit assumption was that the 2D–3D data gap could be progressively closed through better distillation. This paper provides evidence that the gap is structural rather than quantitative: even the most sophisticated distillation (PartField's contrastive learning on both 2D SAM masks and 3D PartNet masks, Point-SAM's iterative refinement with a 2D data engine) cannot match the quality achievable with native labels at scale. This resolves the tension in the literature between "2D data engines are improving" and "2D-lifting results remain imprecise" — the improvement was asymptotic to a ceiling below the native-3D regime, not converging to it.

What this means for the broader 3D learning field: This paper is one data point in a broader trend toward geometry-first, large-scale, self-supervised or weakly-supervised 3D learning. The success of Sonata's self-supervised pretraining (which provides the feature backbone for P³-SAM), the scale of Objaverse and Objaverse-XL (which provide the raw data), and the extraction of labels from artist structure (which provides the supervision) together suggest a research program where 3D understanding advances not by importing 2D knowledge but by developing 3D-native methods trained on 3D-native data at scale. This is the conceptual direction that the paper most strongly endorses, and it is likely to influence resource allocation — both attention and compute — toward scaling 3D data pipelines rather than improving 2D-to-3D bridges for geometry-centric tasks.

Follow-Up Research This Work Enables

Scaling-law characterization for native 3D part segmentation. The paper trains on 3.7M objects and achieves strong results, but provides no evidence about how performance scales with data quantity. A natural follow-up would train P³-SAM (or a comparable architecture) on random subsets of the 3.7M dataset — say, 100K, 500K, 1M, and 3.7M objects — and measure IoU on PartObj-Tiny and PartObj-Tiny-WT at each scale. This would characterize whether returns are diminishing (most of the gain from the first 1M objects), continuing to improve (each doubling of data adds consistent improvement), or showing emergent behavior (specific capabilities appearing only above some threshold). The result would be a scaling law — analogous to the Chinchilla laws for language models but for 3D geometric understanding — that would guide future data collection efforts. If, for instance, performance saturates at 1M objects, then the paper's 3.7M dataset is larger than necessary and effort should shift to data diversity rather than quantity. If performance continues to improve log-linearly, there is a clear case for investing in even larger datasets (Objaverse-XL has 10M+ objects). Crucially, this experiment would also isolate the contribution of data scale from architecture and training recipe — the biggest confound in the paper's current comparisons with prior methods.

Controlled comparison isolating native label quality versus 2D-derived pseudo-label quality. The paper attributes its performance advantage to native 3D training data, but this claim confounds label quality (native 3D vs. 2D pseudo-labels) with data quantity (3.7M vs. much smaller) and architecture (P³-SAM vs. the specific designs of PartField, SAMPart3D, etc.). A clean follow-up experiment would: (1) take a subset of the 3.7M training objects (say, 100K), (2) generate 2D pseudo-labels for these same objects by running SAM on multi-view renderings and aggregating, following the best available 2D-data-engine protocol (e.g., the PartField or Point-SAM pipeline), (3) train two identical P³-SAM architectures — one on the native 3D labels, one on the 2D-derived labels — at matched data scale, and (4) compare performance on PartObj-Tiny and PartObj-Tiny-WT. This would isolate the effect of label quality with architecture and data quantity held constant. A strong native-label advantage would validate the paper's central claim; a small or negligible difference would suggest that data scale, not label source, is the primary driver, and that 2D data engines could close the gap with sufficient data. This experiment is feasible because both label sets exist for the same objects — it only requires implementing the 2D-label-generation pipeline.

Generalization to real-world scanned objects. All evaluation in the paper uses datasets derived from the same sources as the training data (Objaverse, PartNet). A direct test of the claim that P³-SAM provides "general-purpose part segmentation" would evaluate on a held-out dataset of real scanned objects with manually annotated part boundaries. Candidates include: the COSEG dataset (small but contains real scanned shapes with part annotations), Thingi10K (engineering models with distinct part structure from the artistic meshes in Objaverse), or a newly collected set of ~100 photogrammetry scans manually annotated for part boundaries. The metric would be IoU, computed using the same protocol as Table 2. This experiment is a stress test: if P³-SAM's IoU drops substantially (e.g., from 75.70 on PartObj-Tiny to below 40 on real scans), it would indicate that the artist-mesh training distribution does not generalize to real-world geometry despite the watertight training mixture. If performance remains competitive, it would dramatically strengthen the paper's general-purpose claim. Key confound to control: many scanned objects lack the clean inter-part boundaries present in artist meshes, making part decomposition fundamentally more ambiguous — some IoU drop is expected, and the relevant comparison is against human inter-annotator agreement on the same scanned objects.

Combining PRM-style iterative refinement with the two-stage segmentor. The paper's two-stage segmentor (Section 3.2.1) uses a single refinement pass: Stage 1 predicts masks, a global feature is extracted, Stage 2 refines them. This is a two-step process. Modern point cloud architectures (including the PointTransformerV3 that Sonata is built on) can support iterative refinement — applying the same refinement MLP multiple times, with the output of one iteration feeding into the next. A natural extension would be to unroll the Stage 2 refinement for $T = 2, 3, 5, 10$ iterations and measure whether mask accuracy continues to improve. The hypothesis is that iterative refinement with recurrent global feature updates could progressively sharpen boundaries and correct scale errors beyond what a single refinement achieves. This would be trained by applying the mask loss (Dice + Focal) at each iteration, with the backprop-to-best mechanism applied per iteration. If accuracy plateaus after 2–3 iterations, the current two-stage design is near-optimal. If accuracy continues to improve over 5–10 iterations, it suggests that the refinement process benefits from repeated global context aggregation — analogous to how message-passing in graph neural networks benefits from multiple rounds. The key measurement: IoU on the test set of the paper's training data (Table 4 protocol) as a function of refinement iterations, controlling for total parameter count (deeper refinement with shared weights vs. the current two-stage design with separate parameters per stage).

Failure-case-driven analysis of the automated pipeline's part count estimation. The paper's automatic pipeline (Section 3.3, Algorithm 1) produces a part count $N_{part}$ without user input, but the accuracy of this count is never reported. A systematic follow-up would: on PartObj-Tiny (200 objects with known ground truth part counts), run the automatic pipeline and measure (1) mean absolute error in part count, (2) over-segmentation rate (fraction of objects where $N_{part} > N_{gt}$ ), (3) under-segmentation rate (fraction where $N_{part} < N_{gt}$ ), and (4) per-category breakdown of count errors. This would reveal whether the pipeline systematically over-segments (like SAMesh, visible in Figure 4), systematically under-segments (merging small parts into larger ones), or varies by object complexity. The NMS IoU threshold $T_{NMS} = 0.9$ is a single parameter that controls this tradeoff — a sensitivity analysis sweeping $T_{NMS}$ from 0.5 to 0.99 and measuring part count accuracy vs. mask IoU would characterize the Pareto frontier. If the optimal $T_{NMS}$ varies significantly by object category, an adaptive threshold — e.g., learned from the distribution of predicted IoU scores — would be a concrete direction for improvement. This experiment is straightforward to run (it only requires inference on PartObj-Tiny at multiple thresholds) and would fill the most conspicuous gap in the paper's evaluation of its own automated pipeline.

Stress-testing geometric versus semantic part decomposition on functionally ambiguous objects. The paper's training labels come from artist mesh structure, which conflates geometric separability with functional/semantic meaning — an artist models a car wheel as a separate mesh because it is functionally distinct, and the connected-component decomposition recovers this. But geometry and semantics do not always align: a single geometrically smooth surface can contain multiple semantic parts (a car door panel with a handle indentation), and a single semantic part can span multiple geometrically disconnected components (the legs of a chair). A stress test would construct a benchmark of ~50 objects where geometric and semantic part boundaries diverge, manually annotate the semantic ground truth, and evaluate both P³-SAM and a 2D-lifting method (SAMPart3D or PartField) using both the geometric labels (from mesh connectivity) and the semantic labels. The hypothesis: P³-SAM, trained purely on geometric decomposition, should excel on the geometric labels but may struggle on the semantic labels, while 2D-lifting methods — which inherit some semantic knowledge from SAM and DINOv2 (trained on photographs of real objects with functional context) — might better recover semantic boundaries even when they are geometrically subtle. This experiment would characterize the geometry–semantics tradeoff that the paper does not discuss: native 3D training may produce superior geometric decomposition at the cost of reduced semantic awareness, while 2D-lifting may do the reverse. The outcome would clarify what "part segmentation" actually means in each paradigm and whether the choice between them depends on the downstream application's need for geometric precision versus semantic interpretability.

Practical Applications and Downstream Use Cases

Batch processing of large 3D asset libraries for game engines and virtual worlds. Game studios and virtual-world platforms (e.g., Roblox, Fortnite Creative, NVIDIA Omniverse) maintain libraries of tens or hundreds of thousands of 3D assets that need part-level annotation for physics simulation (collision meshes per part), animation rigging (articulation points between parts), and user customization (allowing players to swap individual parts). The current workflow is largely manual: artists label parts, define collision volumes, and specify articulation constraints per asset. P³-SAM's automatic pipeline reduces this to a batch inference process: given a library of unannotated meshes, extract part segmentations automatically at ~real-time per object (once feature extraction is amortized, mask prediction is described as "real-time" in Section 3.2.1). The paper's results on diverse object types (Figure 4, including complex scenes with multiple creatures) and robustness to both watertight and non-watertight data (Tables 2–3) are directly relevant: game assets include both artist-created models (typically non-watertight, with internal structure) and photogrammetry scans (watertight). The ~78.86 IoU on PartObj-Tiny with connectivity (Table 2) indicates that for typical artist-created game assets — which have clean mesh connectivity — the segmentation quality is high. Even on watertight data (49.11 IoU, Table 3), the automatic pipeline provides a useful starting point that can be manually corrected far faster than annotating from scratch. The key practical benefit is throughput: processing a 100K-asset library that would take months of artist time becomes a computation that finishes in hours on a modest GPU cluster (the paper reports ~4 days for training on 3.7M objects on 64 H20s; inference per object is substantially faster).

Preprocessing for AI-generated 3D model pipelines. The paper demonstrates a direct downstream application in Section 4.2 and Figure 5: feeding P³-SAM's segmentation masks to HoloPart (Yang et al., 2025a) for part generation produces "cleaner and more precise parts" than using SAMPart3D's masks. This is a concrete, quantifiable benefit: AI-generated 3D models from diffusion-based or reconstruction-based methods (e.g., Hunyuan3D, DreamFusion, Zero-1-to-3) typically produce watertight meshes without part annotations, but many downstream applications require part-level manipulation — exploded-view rendering, part-aware texturing, or simulation-ready decomposition. P³-SAM can serve as an automatic preprocessing step in these generation pipelines: a generated model enters the pipeline, P³-SAM segments it into parts, and the parts are fed to a part-completion model (like HoloPart) or used directly for exploded-view visualization (like BANG, Zhang et al., 2025). The paper's demonstrated robustness on watertight meshes (Table 3, PartObj-Tiny-WT) and AI-generated models (Figure 14) is critical here — these are exactly the data types produced by current 3D generation systems. A production implementation would integrate P³-SAM as a microservice: generated mesh → point sampling → feature extraction → automatic segmentation → per-part mesh extraction. The computational cost is negligible relative to the generation process itself (which involves neural network inference or optimization), and the added value — making generated models immediately usable for part-level editing — is substantial.

Interactive 3D segmentation for mixed-reality and CAD annotation tools. The paper demonstrates (Appendix A.5, Figure 11) a real-time interactive segmentation system where a user clicks on a 3D model and immediately sees the segmented part. The architecture's design — heavyweight feature extraction once, lightweight mask prediction per click — makes this genuinely interactive: after an initial loading phase (feature extraction, which can be done on model load), each click produces a mask in milliseconds (the segmentor and IoU predictor are described as "lightweight models capable of real-time computation"). This contrasts with 2D-lifting interactive methods (Point-SAM, which also supports interactive segmentation but achieved lower accuracy: 86.70 vs. 90.38 IoU on PartObj-Tiny interactive, Table 2) and with rendering-based methods (which require per-click multi-view rendering). The specific application scenarios include: mixed-reality 3D modeling where a user wearing a headset selects parts of a scanned object by pointing; CAD cleanup where an engineer quickly segments an imported mesh into logical components for simulation; and medical or scientific visualization where a researcher segments anatomical structures or material phases from 3D imaging data. The paper's support for both watertight and non-watertight data is crucial for these applications — CAD meshes are often non-watertight (with internal structure), while medical scans produce watertight surfaces. The ability to handle both without retraining or parameter tuning makes the system practically deployable across domains. The key missing piece for deployment: the paper provides no quantitative latency measurements, so "real-time" remains an unsubstantiated claim. A minimal deployment-ready version would benchmark feature extraction time (seconds per object as a function of point count), per-click mask prediction time (milliseconds), and memory footprint (GB for the feature cache), characterizing the hardware requirements for different interactive scenarios.

Data generation for training downstream part-aware 3D models. Beyond direct use for segmentation, P³-SAM's outputs can serve as training data for other 3D learning tasks that benefit from part-level structure. Examples: (1) training a part-aware 3D autoencoder (where the reconstruction loss is augmented with a part-consistency term) using P³-SAM segmentations as pseudo-ground-truth on large unannotated 3D datasets; (2) training a part-based 3D retrieval system (where objects are indexed by their part decompositions for search-by-part queries) using P³-SAM's outputs on Objaverse-XL (10M+ objects); (3) generating paired data for shape assembly tasks (e.g., "given these parts, predict how they fit together") by segmenting complete objects with P³-SAM and using the resulting parts as training targets. In each case, P³-SAM serves the same role that SAM serves in the 2D domain — an automatic annotation engine that unlocks training at scales far beyond what manual labeling permits. The paper's reported inference speed and 3.7M-object training scale suggest this is practically feasible: running P³-SAM on Objaverse-XL (10M objects) would produce ~10M segmented objects, which is an order of magnitude more part-annotated data than currently exists across all public 3D part segmentation datasets combined. This application does not require P³-SAM's segmentations to be perfect — they need only be accurate enough to provide a useful training signal, and the paper's results (75.70 IoU on PartObj-Tiny without connectivity) suggest they substantially exceed the quality of 2D-derived pseudo-labels currently used for this purpose. The risk is potential data contamination if P³-SAM's training set overlaps with the target dataset, which is likely given that both draw from Objaverse; careful filtering of test sets from P³-SAM's training distribution would be essential.