InCoder-32B-Thinking: Industrial Code World Model for Thinking

🎯 Pitch

An open-source 32B-parameter code model matches frontier 400B+ models on competitive programming by learning to think through hardware errors rather than just code syntax. It achieves this by training on 540M tokens of execution-grounded reasoning traces—generated from a world model that learns causal dynamics of how code affects chips, GPUs, and compilers—demonstrating that domain-simulated thinking data can substitute for raw parameter scale.

1. Executive Summary

This paper introduces InCoder-32B-Thinking, a thinking-augmented code model trained on data from an Error-driven Chain-of-Thought (ECoT) synthesis framework and an Industrial Code World Model (ICWM) that learns causal dynamics between code and hardware behavior from domain-specific execution traces (Verilog simulation logs, GPU profiling diagnostics, compiler outputs, embedded system logs). The ECoT framework generates reasoning traces by contrasting failed code attempts and their environmental error feedback with correct solutions across multi-turn correction trajectories, while the ICWM serves as a learned proxy for real toolchains—predicting execution outcomes without invoking expensive backends—enabling large-scale trajectory synthesis with periodic real-execution audits to maintain calibration. On general benchmarks, InCoder-32B-Thinking achieves 81.3% on LiveCodeBench V5, the highest among all open-weight models including those with an order of magnitude more parameters, establishing that extended thinking can compensate for model size. On industrial benchmarks, it sets top-tier open-source results across chip design, GPU kernel optimization, embedded systems, and 3D modeling—including 84.0% on CAD-Coder and 38.0% on KernelBench L2—while scaling thinking data from 180M to 540M tokens yields consistent improvements, confirming that chain-of-thought synthesis grounded in world model guidance transfers across domains only when the reasoning traces reflect the actual difficulty of each execution-feedback cycle rather than an externally imposed budget.

2. Context and Motivation

The Core Problem: LLMs Cannot Reason About Industrial Code Because They Lack Environment Grounding

The fundamental gap this paper addresses is that current code LLMs cannot effectively generate and debug industrial software—chip designs (Verilog/RTL), GPU kernels (CUDA/Triton), embedded firmware (bare-metal C), compiler optimizations (assembly superoptimization), and parametric CAD models (CadQuery scripts). These are not merely harder versions of general-purpose programming tasks; they require a qualitatively different kind of reasoning that web-scale code corpora do not teach.

When a developer writes a Python function, the runtime environment is largely abstracted away. When a developer writes a Triton kernel for fused attention, however, they must reason about warp-level divergence, shared memory budgets, grid/block configuration, and register pressure—constraints that are invisible in the source code, manifest only during execution, and must be diagnosed from compiler error messages or profiler outputs that are themselves highly domain-specific. Similarly, a Verilog module must satisfy timing closure, combinational path depth constraints, and clock domain crossing rules that a simulator (e.g., Icarus Verilog) validates but that no amount of static code analysis from a general-purpose LLM can infer. The paper frames this as a grounding problem: industrial code reasoning requires an understanding of how code modifications causally affect hardware behavior, and this causal understanding is absent from both pretraining corpora and standard instruction-tuning data.

Why This Problem Matters

The stakes are both practical and structural. On the practical side, industrial software development—chip design, GPU optimization, embedded systems, compiler engineering, and CAD modeling—underpins modern computing infrastructure. The paper notes that benchmarks such as TritonBench and RealBench show leading models achieving "limited success" (Section 1, paragraph 2) on these tasks despite strong general coding performance. For instance, the gap between a model's ability to write correct Python for LeetCode problems and its ability to write a correct, efficient FPGA configuration or a shared-memory-optimized CUDA kernel is enormous. Closing this gap would directly impact hardware design cycles, embedded system development timelines, and the accessibility of high-performance computing optimization—domains where engineering expertise is scarce and expensive.

On the structural side, the paper identifies a capability boundary in current thinking models. OpenAI's o-series and DeepSeek-R1 demonstrated that extended chain-of-thought reasoning—what the paper calls "thinking"—can dramatically improve performance on math, science, and competitive programming. But these models simulate execution through deliberation alone. They do not have access to a grounded simulation of the target environment. When a thinking model reasons about a Verilog design, it can perform logical deduction about the specification, but it cannot predict what Icarus Verilog would actually output when the design is compiled and simulated—the logic error that causes a latch to be inferred, the timing violation on a specific clock path, or the exact diagnostic message that would appear. This means thinking models, as currently constructed, are reasoning in a vacuum about industrial tasks. The paper argues that integrating world models—learned simulators of execution environments—with thinking is the missing piece.

Prior Approaches and Their Shortcomings

The paper identifies four distinct lines of prior work, each of which falls short for industrial code generation in specific ways.

1. Fragmented domain-specific models. Early efforts fine-tuned general-purpose LLMs for isolated industrial sub-domains: VeriGen for Verilog generation, RTLCoder for RTL code, Kevin and CUDA Agent for CUDA kernel synthesis via reinforcement learning, LLM Compiler for compiler optimization, and EmbedAgent for embedded system development (Section 5.1). Each of these targets a single domain with a custom training pipeline and evaluation protocol. The fragmentation means there is no unified model that can handle chip design, GPU optimization, embedded firmware, and CAD modeling simultaneously—a developer working across hardware-software boundaries would need a different model for each task. The paper explicitly positions InCoder-32B (its predecessor) as "a significant step toward unification" by spanning multiple industrial domains with a single 32B-parameter model, and InCoder-32B-Thinking as extending this unification to include thinking.

2. Thinking models without environment grounding. OpenAI o1/o3, DeepSeek-R1, QwQ, and Qwen3-Thinking established that generating long internal chains of thought can improve complex reasoning (Section 5.2). However, the paper points out that for code-specific reasoning, structured Chain-of-Thought prompting, o1-Coder, and rStar-Coder have adapted these techniques to programming tasks, yet "none target industrial code domains, nor do they leverage execution feedback to synthesize reasoning traces" (Section 5.2, final paragraph). The critical distinction: these models think about code as text, without incorporating the actual compilation errors, simulation outputs, profiler traces, or geometric validity checks that real industrial toolchains produce. Their reasoning is self-contained rather than environment-grounded.

3. Self-refinement driven by model-generated feedback. Systems like Self-Refine, LeDex, and RLEF introduced iterative refinement where the model critiques its own outputs (Self-Refine), learns to self-debug from execution diagnostics (LeDex), or uses execution feedback as a reinforcement learning reward signal (RLEF) (Section 5.2). These approaches do incorporate execution signals, but they are applied to general-purpose code (Python debugging, competitive programming) rather than industrial domains where the execution backends are fundamentally different—Verilog simulators, GPU compilers, embedded system emulators, and geometry kernels operate on entirely different principles than Python interpreters. The feedback signals have different structures, different failure modes, and different diagnostic conventions. A model trained to self-debug Python will not transfer to diagnosing a shared-memory bank conflict in CUDA or a setup time violation in Verilog.

4. Code world models without reasoning integration. The paper references CWM (Copet et al., 2025), "an open-weights LLM for research on code generation with world models," as the closest prior work on learning execution dynamics. But CWM focuses on predicting execution outcomes for general code, not industrial environments, and critically does not integrate world model predictions into a thinking pipeline—the world model predicts outcomes, but those predictions are not folded into multi-turn error-correction reasoning traces that train the code model to think like an engineer. The paper positions ICWM as extending this concept to industrial toolchains and, more importantly, integrating it into an end-to-end thinking synthesis framework.

The Deeper Gap: No Training Data Captures How Engineers Actually Reason About Hardware

Underlying all these technical limitations is a data problem that the paper identifies but does not reduce to a single bullet point. The reasoning traces that engineers produce when debugging industrial code—the step-by-step diagnostic process of reading a compiler error, forming a hypothesis about the hardware-level root cause, revising the code, re-running the toolchain, and iterating—are simply not present in web-scale pretraining corpora. GitHub repositories contain source code and sometimes commit messages, but they do not contain the intermediate reasoning that connects a Verilog simulation failure to the specific RTL fix, or the thought process that maps a GPU profiler's occupancy report to a grid configuration change. This reasoning is tacit, unfolding in engineers' minds or in internal lab notebooks, and it is domain-specific: the diagnostic patterns for CadQuery geometry errors look nothing like the diagnostic patterns for STM32 peripheral register sequencing bugs.

The ECoT framework is designed to artificially generate this missing data by running real industrial toolchains, recording both the errors and the fixes, and then prompting a strong generator model (DeepSeek-V3.2) to produce reasoning traces that explicitly narrate the error-correction process. The ICWM then amplifies this data by simulating the toolchains, enabling synthesis at scale without repeatedly invoking expensive real backends. The paper's contribution is not just the resulting model but the data synthesis methodology that produces training data with the natural difficulty distribution of industrial tasks—some problems require a single correction round, others require four rounds of deep hardware-level diagnosis, and the thinking traces reflect this variance organically (as shown in Figure 6, where median thinking length ranges from 91 characters for agentic coding to 19,015 characters for GPU kernel optimization).

How This Paper Positions Itself

The paper positions InCoder-32B-Thinking as unifying four previously separate research threads: (1) industrial code intelligence, which had been fragmented across sub-domains; (2) thinking models, which lacked environment grounding for industrial tasks; (3) execution-grounded self-refinement, which had been applied only to general-purpose code; and (4) code world models, which had not been integrated into reasoning synthesis pipelines. The unification happens through the ECoT + ICWM pipeline: execution feedback (thread 3) from real industrial toolchains (thread 1) is used to train a world model (thread 4) that simulates those toolchains at scale, and the resulting error-correction trajectories are synthesized into thinking traces (thread 2) that train the final model.

The paper explicitly distinguishes itself from pure reinforcement learning approaches (DeepSeek-R1's GRPO, RLEF) by using the ICWM as a simulator in the loop rather than as a reward function. The ICWM does not assign scalar rewards to guide policy optimization; it predicts structured execution outcomes (compilation PASS/FAIL, diagnostic messages, numerical outputs) that are fed back into the generator as observations in a multi-turn dialogue, mimicking the human engineering workflow. This is a subtle but important design choice: the ICWM is trained on trajectory data and used for trajectory amplification, not for RL-style policy improvement. The paper claims this approach produces reasoning traces that "match the natural reasoning depth distribution of industrial tasks" (Abstract) because the depth is determined by how many correction rounds each task actually requires, not by a fixed reasoning budget imposed during training.

3. Technical Approach

3.1 Reader Orientation

InCoder-32B-Thinking is a 32-billion-parameter language model that generates reasoning traces (explicit step-by-step diagnostic narratives) alongside code for industrial programming tasks. It solves the problem that industrial code generation—writing GPU kernels, chip designs, embedded firmware, and CAD scripts—requires reasoning about hardware constraints and execution feedback that general-purpose code models cannot perform because they have never seen the intermediate diagnostic process that engineers use when debugging against real toolchains.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components connected in a pipeline:

Task Seeding and Environment Bundling — collects task specifications and packages them with the domain-specific execution environments (testbenches, simulators, memory layouts, synthesis scripts) needed to run and verify code.
Real Execution Backends — domain-specific toolchains (Triton/CUDA compilers, Icarus Verilog simulator, Renode embedded emulator, CadQuery geometry kernel) that compile and execute candidate code, returning structured PASS/FAIL labels and diagnostic logs.
Generator Model (DeepSeek-V3.2) — a frontier LLM that produces initial code, receives execution feedback, diagnoses failures, and produces revised code with reasoning traces across multiple correction rounds, forming multi-turn trajectories.
Industrial Code World Model (ICWM) — a learned neural proxy for the real execution backends, trained on the collected trajectories, that predicts execution outcomes (labels, diagnostics, numerical outputs) from code text alone without invoking real toolchains.
InCoder-32B-Thinking (Final Trained Model) — the 32B-parameter model trained on the combined real and ICWM-amplified reasoning trajectories, which learns to produce both thinking blocks (error-diagnosis narratives) and correct code across industrial domains.

Information flows as follows: task seeds are bundled with environments → the generator proposes initial code → real backends execute it and return feedback → the generator diagnoses the error and revises the code (up to K=4 rounds) → the full multi-turn trajectory enters the real dataset D_real → single-turn (environment + code → outcome) pairs from D_real train the ICWM → the trained ICWM replaces real backends to generate additional trajectories at scale → the final training corpus D = D_real ∪ D_icwm trains InCoder-32B-Thinking.

3.3 Roadmap for the Deep Dive

First, the task seeding and environment bundling process, since it establishes the executable grounding—without a reproducible environment, no structured feedback is possible.
Second, the execution-grounded trajectory synthesis loop, because it generates the multi-turn error-correction data that is the training signal for both the ICWM and the final model.
Third, the ICWM—its definition, training objective, and fidelity analysis—since it is the mechanism that enables scaling trajectory synthesis beyond what real toolchains can support.
Fourth, the ICWM-driven data amplification loop, which explains how a trained world model replaces real backends to produce additional training trajectories, including the periodic audit mechanism that maintains calibration.
Fifth, the final training corpus composition and how the reasoning traces encode adaptive thinking depth organically from the task structure rather than from an externally imposed budget.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a data synthesis and model training paper whose core idea is that industrial code reasoning requires training data in which reasoning traces are explicitly grounded in execution feedback from domain-specific toolchains, and that a learned world model can amplify such data by simulating those toolchains at scale.

Task Seeding and Environment Bundling

The pipeline begins with task seeds reused from the predecessor InCoder-32B model, which already included queries, unit tests, and execution environments across industrial domains. Each task seed is packaged with its required environmental context into a pair ⟨s_task, s_env⟩, where s_task is the natural-language task description (e.g., "Implement a CUDA kernel for hinge loss with predictions shape (32768, 32768) and targets shape (32768,)") and s_env is the environment bundle containing all artifacts needed for execution and verification.

The environment bundle is domain-specific. For Verilog/RTL chip design tasks, it contains testbenches and Yosys synthesis scripts that define the compilation and simulation pipeline. For STM32 firmware tasks, it couples the code snippet with the corresponding memory layout register map, CMSIS hardware abstraction layer headers, and linker scripts that specify where code and data reside in the microcontroller's address space. For GPU kernel tasks, the bundle includes the Triton or CUDA compilation configuration and test harness. For CAD modeling tasks, it includes the CadQuery execution environment and reference geometry for validation.

The paper emphasizes that this bundling provides a "fully specified, reproducible execution context for every downstream step" (Section 2.1). This is not a minor detail—it is what makes the feedback loop possible. Without the testbench, a Verilog module cannot be automatically verified. Without the memory layout, an embedded C snippet cannot be compiled and emulated. The environment bundle is the bridge between code-as-text and code-as-executable-artifact, and it determines what kind of feedback the system can receive.

After bundling, a lightweight prompt router inspects the environment bundle and selects domain-specific instructions for the generator model. These instructions encode the reasoning categories that are relevant to each domain. For GPU kernel tasks, the generator is instructed to reason about warp divergence (threads within a warp taking different execution paths, which serializes execution) and shared memory budgets (the limited on-chip SRAM that must be partitioned among thread blocks). For RTL tasks, the instructions direct the generator to reason about combinational path depth (the longest logic chain between registers, which determines maximum clock frequency) and clock domain crossings (signals passing between independently-clocked regions, which require synchronization). For CAD tasks, the generator is instructed to reason about wall thickness and manifold validity (geometric constraints that determine whether a 3D shape is physically realizable). For firmware tasks, it reasons about peripheral register sequencing (the correct order of writes to hardware control registers to initialize a device).

The prompt router is "lightweight"—the paper does not specify its architecture, but the implication is that it is a simple rule-based or classifier system that maps the environment bundle's domain tag to a pre-written set of reasoning instructions. Its role is to ensure the generator applies the right conceptual framework for the domain rather than treating all tasks as generic code generation.

Execution-Grounded Trajectory Synthesis

This is the core data generation mechanism that produces the multi-turn error-correction trajectories from which both the ICWM and the final thinking model learn.

Initial generation. Given a task seed and environment bundle with domain-specific reasoning instructions, the generator model DeepSeek-V3.2 produces an initial pair of reasoning content and candidate code (r^(0), c^(0)). The superscript (0) denotes turn zero, the very first attempt before any execution feedback has been received. The reasoning content r^(0) is the generator's internal diagnostic narrative—it explains the approach, identifies potential hardware constraints, and describes the intended behavior of the code. The candidate code c^(0) is the actual executable implementation.

Distributed execution. The candidate code c^(0) is dispatched to the real execution backend that matches its domain. The paper specifies four backend categories:

Triton/CUDA backends for GPU kernels, which compile the kernel and execute it on GPU hardware or emulators, returning compilation errors (syntax, type mismatches, resource overallocation), runtime errors (out-of-bounds memory access, illegal instruction), and profiling data.
Renode for microcontroller firmware, which emulates the ARM Cortex-M or equivalent processor, including its memory map, peripherals, and interrupt controller, returning hardware exceptions (MEMORY_FAULT, BUS_ERROR, USAGE_FAULT) with register state and stack traces.
CadQuery for CAD geometry, which executes the parametric modeling script and returns geometric validity checks, including GEOMETRY_ERROR for non-manifold edges or self-intersecting surfaces, along with diff reports showing discrepancies from reference models.
Yosys/Icarus for RTL design, where Yosys performs logic synthesis (converting RTL to a gate-level netlist) and Icarus performs functional simulation against the testbench, returning PASS/FAIL, waveform traces, and synthesis warnings about inferred latches or combinational loops.

Each backend returns a structured result containing precisely two components: (i) an outcome label drawn from a domain-specific vocabulary—PASS for successful execution, COMPILATION_ERROR for syntax or type failures, MEMORY_FAULT for illegal memory access, GEOMETRY_ERROR for invalid shapes, TIMING_VIOLATION for setup/hold failures, etc.—and (ii) the associated diagnostic log, which is the full text output that a human engineer would read (compiler error messages with line numbers, simulation waveforms, stack traces, memory maps, geometric constraint violation reports).

Multi-turn feedback loop. When execution produces an error, the concrete diagnostic output is packaged as an observation o^(k) and fed back to the generator. The generator then must perform two tasks: diagnose the fault (explain what the error means in terms of the hardware or domain semantics) and produce a revised reasoning trace and implementation (r^(k+1), c^(k+1)). The paper allows up to K=4 correction rounds. If execution passes all checks before K=4, the trajectory terminates early. If it still fails after K=4, the trajectory is recorded as ultimately unsuccessful.

The full multi-turn trajectory is formalized in Equation 1:

$\tau = \langle s_{\text{init}}, r^{(0)}, c^{(0)} \rangle \rightarrow \langle r^{(1)}, c^{(1)} \rangle \dots \rightarrow \langle r^{(k)}, c^{(k)} \rangle$

where s_init is the initial task description together with its environment bundle, r^(k) is the reasoning content at turn k, c^(k) is the corresponding candidate code, and each arrow represents one round of execution feedback followed by the generator's revision.

What it computes: a complete history of a single task's solution attempts, from initial code through each diagnostic step and revision, terminating when either the code passes all execution checks or the maximum number of correction rounds is exhausted.

Why this form: the inclusion of both successful and unsuccessful intermediate turns is critical. Training on only successful trajectories would teach the model correct solutions but not how to diagnose failures—the model would never learn to associate specific error messages with specific types of fixes. Training on complete trajectories preserves the full error-correction reasoning process, including common failure modes (e.g., a shared memory budget exceeded, a timing violation on a specific flip-flop, a broadcasting shape mismatch in CUDA indexing) alongside the reasoning steps that resolve them. This mimics how human engineers learn: by experiencing failures and understanding why the fix works, not just by studying correct solutions. The maximum of K=4 rounds is a practical budget; the paper does not claim this is theoretically optimal but notes that it is sufficient to capture multi-step debugging while preventing trajectories from becoming excessively long.

Dataset D_real. The full multi-turn trajectories—both successful and unsuccessful—enter the real execution dataset D_real. This dataset serves two purposes: first, its trajectories directly train the final InCoder-32B-Thinking model; second, the individual single-turn pairs extracted from it (each (s_env, c^(k)) → o^(k) mapping) train the ICWM described next.

Industrial Code World Model (ICWM) Definition and Training

The ICWM is the mechanism that enables the paper to scale trajectory synthesis beyond what is feasible with real toolchains. Its definition, training, and fidelity are central to the paper's claims.

Definition. The ICWM is formally defined in Equation 2:

$\text{ICWM}_\theta : (s_{\text{env}}, c^{(k)}) \mapsto \hat{o}^{(k)}$

where s_env is an environment bundle containing all task-specific artifacts required for execution and verification, c^(k) is the candidate code at turn k, and ô^(k) is the predicted observation that a real backend would return.

The predicted observation ô^(k) is structured identically to real backend output. It includes: an outcome label drawn from the same domain-specific vocabulary as real backends (PASS, COMPILATION_ERROR, MEMORY_FAULT, etc.), a diagnostic message that describes what went wrong (or confirms success), and, when applicable, numerical outputs (computed values, performance metrics) or diff summaries (geometric differences from reference models).

What it computes: a learned mapping from code text and environment context to a prediction of what the real toolchain would produce if invoked. It is not a classifier that only predicts PASS/FAIL—it generates the full diagnostic text, which is what the generator model needs to produce a meaningful error-correction reasoning trace.

Why this form: the ICWM is designed as a drop-in replacement for real backends in the trajectory synthesis loop. If it only predicted binary labels, the generator would have no information to diagnose failures—it could not learn to associate specific diagnostic messages with specific types of hardware or geometry errors. The full diagnostic output preserves the causal structure that drives error-correction reasoning. A model that only predicted PASS/FAIL would be equivalent to a single-bit outcome reward model, which the paper implicitly rejects in favor of a richer world model that simulates the actual feedback channel.

Training data. The ICWM is trained on every real execution turn in D_real. Each turn consists of a single (s_env, c^(k), o^(k)) triple where s_env is the environment bundle, c^(k) is the candidate code at that turn, and o^(k) is the ground-truth observation returned by the real backend. The ICWM's training task is to predict o^(k) given (s_env, c^(k)) as input.

Training procedure. The ICWM is a language model fine-tuned on these single-turn pairs. To handle the differences across domains—a GPU compiler diagnostic looks entirely different from a CadQuery geometry error—the paper applies two normalizing techniques: (1) a domain tag is prepended to each input, explicitly telling the model which domain's execution semantics to apply (e.g., "GPU Kernel:", "Chip Design:", "3D Modeling:", "Code Optimization:", "Embedded Systems:"); and (2) domain-specific output templates structure the prediction into a consistent format, ensuring that the model learns the canonical structure of each backend's diagnostic output. The paper does not specify the exact architecture—whether it starts from the same base model or a separate one—but the language "the ICWM is a language model" implies it is likely a fine-tuned variant of the same model family rather than a separate supervised model.

Fidelity validation (Section 4.1, Figure 5). The ICWM's reliability is validated on a held-out set of 2,000 execution turns per domain from D_real. Two metrics are reported:

Outcome prediction accuracy: the fraction of individual turns where the ICWM's predicted label exactly matches the real backend's label. Across five domains, the mean is 96.7%. Chip design achieves the highest at 97.4%, which the paper attributes to the Yosys and Icarus backends producing "structurally template-like diagnostics" that are easier for a language model to learn. 3D modeling achieves the lowest at 95.9%, because CadQuery geometry checks involve floating-point tolerances and implicit Boolean operations whose outcomes are harder to predict from code text alone. GPU kernels, code optimization, and embedded systems fall between 96.2% and 97.1%.
Trajectory agreement: the fraction of full multi-turn trajectories whose final pass/fail verdict (the outcome label at the final turn) is identical under real and ICWM-driven execution. Across domains, the mean is 94.4%. Chip design again achieves the highest at 95.8%, while 3D modeling again has the widest gap at 93.1%.

The paper explicitly interprets the 1.6–2.8 percentage point drop from outcome accuracy to trajectory agreement as expected: a single mispredicted turn can redirect the generator onto a divergent correction path, compounding the initial error. If the ICWM misclassifies a real COMPILATION_ERROR as PASS at turn 2, the generator will not attempt a correction at that turn, potentially causing a cascade of failures that result in a different final verdict. Even at the lowest trajectory agreement of 93.1%, the paper argues that more than nine out of ten ICWM-driven trajectories are functionally equivalent to their real-execution counterparts, which it considers sufficient for large-scale synthesis.

Qualitative failure analysis. The paper provides case studies that illuminate the ICWM's strengths and weaknesses. The GPU kernel case study (Section 4.1) shows a Triton kernel for fused attention that allocates tl.zeros([BLOCK_M, BLOCK_N]) with BLOCK_N=128 and launches with num_warps=8, exceeding the streaming multiprocessor's 48 KB shared-memory budget. The real Triton compiler returns MEMORY_FAULT with the diagnostic "shared memory request (49152 B) exceeds per-SM limit." The ICWM correctly predicts both the label and the quantitatively matching diagnostic. In the subsequent turn, the generator reduces BLOCK_N from 128 to 64, and the ICWM correctly predicts PASS. The trajectory is identical under real execution.

The 3D modeling false positive case study shows a CadQuery script that subtracts a cylindrical bore from a rectangular body, placing the cylinder axis exactly tangent to one face. This creates a degenerate zero-thickness edge. Real CadQuery returns GEOMETRY_ERROR with "BRep check: edge has zero length." The ICWM classifies this turn as PASS because the code is syntactically valid and the specified dimensions appear plausible—it fails to model the geometric tolerances that would detect the zero-thickness edge. This type of corner case accounts for the wider fidelity gap in 3D modeling. The paper notes that during periodic audit rounds (described below), such mismatches are detected via real-execution spot checks and the corrected labels are used to retrain the ICWM.

ICWM-Driven Data Amplification

Once the ICWM is trained and its fidelity validated, it replaces real backends in the trajectory synthesis loop to produce additional training trajectories at scale. The mechanism is formalized in Equation 3:

$\tau = \langle s_{\text{init}}, r^{(0)}, c^{(0)} \rangle \xrightarrow{\text{ICWM}_\theta} \langle r^{(1)}, c^{(1)} \rangle \dots \xrightarrow{\text{ICWM}_\theta} \langle r^{(k)}, c^{(k)} \rangle$

where each arrow represents one round of ICWM feedback rather than real execution feedback, followed by the generator's revision. Since each ICWM prediction is a single forward pass through a language model rather than a real compilation or simulation (which can take seconds to minutes for complex designs), the synthesis loop can generate orders of magnitude more trajectories for a given computational budget.

The paper does not specify the exact scale factor—how many ICWM-generated trajectories are produced relative to real trajectories—but the implication from the training data scaling experiment (Section 4.3, Figure 7) is that the total thinking training data reaches 540 million tokens, substantially more than the 250 million tokens used for the predecessor InCoder-32B. The reason for this increase is explicitly attributed to the chain-of-thought nature of the data: "This reasoning process naturally creates longer texts and increases the total token count" (Section 4.3).

Periodic real execution audits. A critical mechanism for maintaining ICWM fidelity over time is periodic audit rounds. The paper briefly mentions that "periodic real execution audits that keep the world model calibrated" (Section 2, Figure 4 caption) and provides more detail in the 3D modeling case study: "during periodic audit rounds described in subsection 2.3, such mismatches are detected via real-execution spot checks, and the corrected labels are used to retrain the ICWM, progressively narrowing the gap." The audit procedure works as follows: at regular intervals during the ICWM-driven synthesis loop, a subset of generated trajectories are re-executed against real backends. Any turns where the ICWM's prediction mismatches the real backend's output are flagged, and the corrected (real execution) labels are added to the ICWM's training set, which is then used to fine-tune the ICWM further. This is an online learning mechanism: the ICWM improves over time as its errors are discovered and corrected.

Resulting reasoning corpus. The final reasoning training set is:

$D = D_{\text{real}} \cup D_{\text{icwm}}$

Every trajectory in D is either produced by real execution or verified against it through the audit mechanism, meaning the code model learns from grounded feedback rather than hallucinated outputs. The paper emphasizes this point: the ICWM is not used to generate training data in an unconstrained loop that could drift into hallucination; the periodic audits ensure that the world model's predictions remain tethered to real backend behavior.

Final Training Corpus and Adaptive Thinking Depth

The final corpus D = D_real ∪ D_icwm is used to train InCoder-32B-Thinking. The paper does not describe the training procedure in detail—hyperparameters, optimizer, learning rate schedule, number of epochs, or hardware configuration—in the main text. The training data scaling experiment in Section 4.3 and Figure 7 shows three checkpoints trained on 180M, 360M, and 540M tokens of thinking data, respectively, and demonstrates that most industrial benchmarks improve monotonically with more thinking data, indicating that the corpus quality is sufficient that more data reliably improves performance without overfitting or degradation.

A crucial property of the training corpus, emphasized throughout the paper, is that the thinking depth is adaptive—it is not imposed by a fixed training protocol but emerges organically from the error-driven synthesis pipeline. Figure 6 (discussed in Section 4.2) shows the distribution of thinking block lengths across nine task categories. The median thinking length spans a 209× range, from 91 characters per step in agentic coding to 19,015 characters in GPU kernel optimization. The paper argues that this variation arises naturally because tasks involving complex execution feedback accumulate longer reasoning traces through successive correction rounds, whereas tasks with self-evident errors produce shorter ones. GPU kernel optimization demands the deepest reasoning because each correction round requires diagnosing hardware-level issues such as grid/block configuration, shared-memory layout, and warp-level scheduling. Chip design exhibits an inverted profile with a short thinking block of 1.5K characters followed by a long RTL answer of 6.9K characters, because Yosys/Icarus feedback is structurally concise while the bulk of effort lies in code generation. Agentic coding yields the shortest per-step thinking at 91 characters, since reasoning is distributed across many interaction turns and each step decides only the next action.

This adaptive property is not an accident of data collection but a deliberate consequence of the ECoT framework design. Because thinking traces are shaped by the actual difficulty of each execution-feedback cycle—how many correction rounds were needed, how detailed the diagnostic output was, how complex the hardware-level root cause—the model learns to calibrate its thinking effort to the task rather than producing uniformly verbose or uniformly brief reasoning. The paper contrasts this with approaches that impose a fixed reasoning budget: "the thinking blocks are shaped by the actual difficulty of each execution-feedback cycle, not by an externally imposed reasoning budget" (Section 4.2). This is important because a fixed budget would either waste computation on easy problems (producing unnecessarily long reasoning traces) or starve hard problems of needed reasoning depth.

The paper also reports a specific finding about how the thinking corpus differs from typical SFT data. The predecessor InCoder-32B used approximately 250 million tokens of training data. InCoder-32B-Thinking uses 540 million tokens, more than twice as much, specifically because "the reasoning process naturally creates longer texts" (Section 4.3). Each training example now includes not just the task description and correct code but also the multi-turn error-correction reasoning traces that connect failed attempts to diagnoses to fixes. This expansion in token count is not a design goal but a side effect of making the training data faithfully represent the engineering debugging process. The 209× range in thinking depth means the corpus is highly non-uniform in its use of tokens, but this non-uniformity mirrors the natural distribution of task difficulty across industrial domains.

4. Key Insights and Innovations

Innovation 1: Execution Feedback as the Generative Engine for Reasoning Traces, Not Just a Reward Signal

The paper's most fundamental conceptual move is redefining what execution feedback is in the context of training thinking models. Prior work treats execution outcomes as either a filtering mechanism (keep the output if tests pass, discard otherwise) or a scalar reward signal for RL (RLEF trains code LLMs with execution-grounded RL; CodeV-R1 and VeriReason use hardware-specific reward signals for Verilog generation). In these paradigms, execution feedback is evaluative—it tells the model whether an answer is good or bad, and the model learns to maximize the probability of good answers.

ECoT inverts this. Execution feedback—specifically, the structured diagnostic output from domain toolchains (compiler errors, simulation waveforms, memory fault traces, geometric constraint violations)—becomes the generative prompt for synthesizing reasoning content. The generator does not receive a scalar reward; it receives a full observation o^(k) containing the exact diagnostic text a human engineer would read, and it must produce a narrative that explains what went wrong and how to fix it. The reasoning trace is not an emergent property of RL optimizing a correctness signal—it is explicitly authored by a frontier model (DeepSeek-V3.2) in response to concrete execution artifacts, then used as training data for the smaller model.

This distinction matters because it changes what the training data teaches. When execution feedback is evaluative, the model learns a policy: given this problem, produce code that passes tests. When execution feedback is generative, the model learns a diagnostic methodology: given this error message, here is how an expert traces it to a hardware-level root cause, and here is the specific code change that fixes it. The paper's Figure 3 illustrates this concretely: a model trained without thinking traces indexes a 2D tensor with the same flat index as a 1D tensor, producing an out-of-bounds error; the thinking model diagnoses the shape mismatch, reasons about broadcasting semantics, and correctly maps flat indices to row indices. The difference is not just "better code" but an explicit reasoning step that connects the error (out of bounds) to the root cause (dimension mismatch) to the fix (row-based indexing).

This reframing is fundamental rather than incremental because it changes the role of the environment in the training pipeline. The environment is no longer an oracle that judges outputs; it is a teacher that generates curriculum. Different domains produce different diagnostic signatures—Verilog simulators report timing violations, GPU compilers report shared memory budget overflows, CadQuery reports zero-thickness edges—and the ECoT framework captures this diversity by having the generator explicitly narrate the diagnostic-to-fix mapping for each domain. The thinking traces in the final corpus span a 209× range in length (Figure 6) precisely because different backends demand different diagnostic depth, and the model learns this adaptivity from the data rather than from a hand-designed reasoning budget.

The evidence for this innovation being the engine of model capability—not just a data augmentation trick—is in Figure 7 (Section 4.3). As thinking data scales from 180M to 540M tokens, performance on industrial benchmarks improves monotonically across most metrics: VeriScope score jumps from 61.8 to 75.4, KernelBench L2 from 16.0 to 38.0, CAD-Coder from roughly 76 to 84. If execution feedback were merely filtering correct from incorrect outputs, scaling the amount of thinking data would not produce consistent gains—the model would already have access to the correct answers. The fact that more thinking data (more diverse error-correction narratives with richer diagnostic reasoning) drives monotonic improvement is evidence that the generative reasoning traces themselves are the active ingredient.

This also explains why the model achieves 81.3% on LiveCodeBench V5, the highest among all open-weight models including those with an order of magnitude more parameters (Table 2). Competitive programming problems on LiveCodeBench are algorithmic and do not involve hardware backends, yet the thinking capability transfers. The model has learned a general diagnostic methodology—observe failure, trace to root cause, revise—that applies even when the failure is a logical error rather than a compilation error. This transfer is evidence that ECoT teaches reasoning process, not just domain-specific fixes.

Innovation 2: The Industrial Code World Model as a Learned Execution Simulator, Not a Verifier or Reward Model

The ICWM is not a process reward model (PRM), not an outcome reward model (ORM), and not a verifier in the standard sense. This is the paper's second distinctive contribution: repurposing world models—concepts from model-based reinforcement learning and world simulation—for data amplification in code LLM training, applied specifically to industrial execution environments.

To see why this is novel, consider the standard role of learned verifiers in code generation. In works like CodeRL or RLEF, a learned model predicts whether code will pass tests, and this prediction serves as a dense reward signal for RL or as a filtering mechanism for best-of-N sampling. These verifiers output a single scalar or binary label. They are trained to be evaluative, and their purpose is to substitute for expensive test execution during training or inference.

The ICWM is fundamentally different. It predicts the full structured observation that a real backend would return—outcome label, diagnostic message, numerical outputs, diff summaries—conditioned on both the code and the environment bundle. It is trained on single-turn (s_env, c^(k)) → o^(k) pairs extracted from multi-turn trajectories, and its purpose is to simulate the environment in the loop for trajectory synthesis, not to score outputs. The distinction is that the ICWM must model the causal dynamics of the execution environment: how a specific code change (reducing BLOCK_N from 128 to 64) causally transforms the execution outcome (from MEMORY_FAULT to PASS). A verifier that only predicts PASS/FAIL cannot drive a multi-turn error-correction dialogue because it provides no diagnostic information for the generator to reason about.

This is a fundamental shift in how learned environment models are used in LLM training pipelines. Prior work on code world models (CWM, Copet et al., 2025) demonstrated that LLMs can learn to predict execution outcomes, but CWM was applied to general-purpose code and was not integrated into a reasoning synthesis pipeline—it predicted outcomes but did not drive trajectory generation. The ICWM is operationalized: it actively generates multi-turn correction trajectories by standing in for real backends during data amplification, and its fidelity is continuously monitored and improved through periodic real-execution audits.

The evidence that the ICWM functions as a simulator, not just a classifier, is in the fidelity analysis (Section 4.1, Figure 5). The 96.7% mean outcome prediction accuracy and 94.4% mean trajectory agreement are reported for multi-class structured prediction (not binary classification), and the case studies show the ICWM generating quantitatively matching diagnostic messages (the exact memory size in bytes, the specific error type). The 3D modeling false positive case—where the ICWM predicts PASS for a script that produces a zero-thickness edge—reveals the boundary of what the world model can capture: it fails on geometric corner cases that depend on floating-point tolerances and implicit Boolean operations, which are hard to infer from code text alone. This failure mode is diagnostic of the ICWM being a learned simulator rather than a rule-based system: it succeeds where the causal relationship between code and outcome is textually evident (memory budgets, syntax) and fails where it depends on continuous geometric computation not fully expressed in the code.

The periodic audit mechanism is another marker of this being a world model, not a static verifier. The ICWM's errors are detected by re-executing a subset of trajectories against real backends, and the corrected labels are used to retrain the ICWM—an online learning loop that progressively narrows the fidelity gap. This is standard practice for learned world models in RL (where model errors compound and must be corrected) but is absent from standard verifier training pipelines, which assume a fixed training set.

Innovation 3: Adaptive Thinking Depth as an Emergent Property of Error-Driven Synthesis, Not a Hand-Designed Reasoning Budget

The paper's third conceptual contribution is the observation—backed by Figure 6 and the analysis in Section 4.2—that thinking depth should be a consequence of task difficulty, not an externally imposed hyperparameter, and that error-driven trajectory synthesis naturally produces this adaptivity without any explicit mechanism for controlling reasoning length.

The dominant paradigm in thinking model training, as established by OpenAI o1 and DeepSeek-R1, is to train models to produce long chains of thought and then let the model decide (or be prompted to decide) when to stop. The Qwen3-Thinking models introduced a hybrid mode that dynamically switches between fast and slow thinking, but the underlying training still aims to produce reasoning traces of roughly uniform verbosity, and the switching mechanism is a post-hoc routing decision. The paper's approach is different: the reasoning traces in the training corpus vary in length by a factor of 209× (from 91 to 19,015 characters per step) because the tasks that generated them varied in how many correction rounds they required and how complex the diagnostic feedback was. No explicit length target was imposed during data synthesis.

This is a conceptually elegant idea with practical consequences. The paper argues that GPU kernel optimization produces the deepest reasoning (19K characters median) because each correction round requires diagnosing hardware-level issues—grid/block configuration, shared-memory layout, warp scheduling—and the diagnostic feedback from Triton/CUDA compilers is itself detailed. Chip design produces short thinking blocks followed by long code blocks because Yosys/Icarus feedback is "structurally concise" (the paper's phrase) while the RTL code generation itself is the bulk of the effort. Agentic coding produces the shortest thinking (91 characters) because reasoning is distributed across many interaction turns, with each step deciding only the next action.

The significance is that the model learns task-appropriate reasoning allocation from the data distribution itself. It does not need a separate mechanism to decide how long to think—the training data's natural variance in reasoning depth means the model internalizes that GPU kernels need deep hardware diagnosis while simple API calls do not. The Figure 7 scaling experiment supports this: as thinking data scales from 180M to 540M tokens, the model does not become uniformly more verbose or more terse—it improves on benchmarks that require deep reasoning (KernelBench L2, VeriScope, CAD-Coder) while maintaining or improving on benchmarks that require concise responses. If the model were simply learning to produce longer traces everywhere, it would likely degrade on benchmarks that penalize verbosity.

This contribution is incremental in mechanism (the data naturally has variable-length traces) but fundamental in its implication for thinking model design: the reasoning budget should be determined by the environment, not the training protocol. Prior thinking models treat chain-of-thought length as a training hyperparameter to be optimized (longer is generally better for hard problems, shorter for easy ones, and the model must be taught or prompted to modulate length). ECoT shows that if the training data is generated through interaction with an execution environment that provides structured feedback of varying complexity, the length distribution emerges automatically because hard environments demand more correction rounds and produce richer diagnostics. The 209× range in Figure 6 is not designed—it is measured from data that was generated by a fixed protocol (up to K=4 correction rounds, same generator, same prompt router), and the variance comes entirely from how different domains' backends respond to errors.

Innovation 4: Domain-General Industrial Code Intelligence from a Single Model, Enabled by Environment-Grounded Reasoning

This contribution is more about the architectural unification than a novel algorithm, but it is intellectually distinctive because it demonstrates that a single model can achieve top-tier performance across five fundamentally different industrial domains—chip design, GPU optimization, embedded systems, code optimization, and 3D modeling—when trained with environment-grounded thinking traces, whereas prior work treated each domain as requiring a separate specialized model.

The evidence is in Tables 4 and 5, which show InCoder-32B-Thinking achieving the best open-source results across all evaluated industrial benchmarks: 75.4 VeriScope score (chip design), 38.0 KernelBench L2 (GPU optimization), 84.0 CAD-Coder (3D modeling), 47.9 EmbedCGen main pass rate (embedded systems), and 93.0 SuperCoder accuracy (code optimization). No other open-weight model appears at or near the top across all five domains simultaneously. Proprietary Claude-Sonnet-4.6 outperforms on several metrics (77.0 CAD-Coder, 28.0 KernelBench L2, 28.8 TritonBench G-call), but InCoder-32B-Thinking is competitive or superior on others (84.0 vs. 77.0 on CAD-Coder, 38.0 vs. 28.0 on KernelBench L2).

Prior work in industrial code intelligence was fragmented by design. VeriGen, RTLCoder, and CodeV-R1 targeted Verilog generation specifically. Kevin and CUDA Agent targeted CUDA kernel synthesis via RL. LLM Compiler targeted compiler optimization. EmbedAgent targeted embedded systems. CAD-Coder targeted 3D modeling. Each of these works developed domain-specific training pipelines, domain-specific reward functions or execution feedback mechanisms, and domain-specific evaluation protocols. The implicit assumption was that industrial domains are too different to be served by a single model—hardware description languages, GPU programming models, embedded C, and parametric CAD scripting share little surface syntax or semantics.

InCoder-32B-Thinking challenges this assumption by demonstrating that environment-grounded reasoning is the unifying capability. The model does not need to learn five different syntaxes in isolation; it needs to learn a general diagnostic methodology—generate code, observe execution feedback, trace failure to hardware-level root cause, revise—that applies across domains because each domain's execution backend provides structured feedback that follows predictable conventions (compiler errors always have line numbers and error codes, simulators always report timing violations with specific signal names, geometry kernels always report constraint violations with specific edge or face identifiers). The ICWM learns these conventions per domain (via the domain tags and output templates described in Section 3.4), and the ECoT framework synthesizes reasoning traces that teach the model how to interpret each domain's diagnostic signature.

This unification is significant beyond raw benchmark performance because it changes the deployment equation for industrial AI tools. Instead of maintaining five separate specialized models—each requiring its own fine-tuning pipeline, evaluation suite, and serving infrastructure—an organization can deploy one model that handles all industrial coding tasks, plus general-purpose coding, with competitive performance. The paper's Tables 1-3 show that InCoder-32B-Thinking also achieves strong results on general code benchmarks (95.1% HumanEval, 81.3% LiveCodeBench V5, 70.4% SWE-bench Verified), meaning the industrial reasoning capability does not come at the cost of general coding performance—another form of unification.

The caveat is that this unification is enabled by the data synthesis pipeline, not by any architectural innovation in the model itself. InCoder-32B-Thinking is built on the same base architecture as InCoder-32B, and its domain generality comes from training on heterogeneous data produced by a heterogeneous set of execution backends. The contribution is demonstrating that data diversity across execution environments is sufficient for domain-general industrial reasoning—the model architecture does not need domain-specific components—which is a finding with practical implications for how industrial AI systems should be built (invest in diverse execution-grounded data, not in domain-specific architectures).

Innovation 5: The ICWM Fidelity-Trajectory Agreement Gap as a Diagnostic for World Model Limitations

This is a methodological contribution rather than an algorithmic one, but it is intellectually distinctive because it provides a quantitative framework for diagnosing where learned environment simulators fail and what those failures imply for downstream training.

The paper reports that the ICWM achieves 96.7% outcome prediction accuracy but only 94.4% trajectory agreement, with a consistent 1.6–2.8 percentage point gap across all five domains (Figure 5, Section 4.1). The paper does not treat this gap as a deficiency to be eliminated but as an informative signal about error compounding in learned world models. A single mispredicted outcome label at turn 2 redirects the generator onto a divergent correction path, and the final trajectory verdict may differ from real execution even if every subsequent turn is predicted correctly. The gap is not just measurement noise—it is a structural property of using a learned simulator in a multi-turn feedback loop, and its size depends on the domain: 3D modeling has the widest gap (95.9% outcome accuracy vs. 93.1% trajectory agreement) because CadQuery geometry checks involve floating-point tolerances that are hard to predict from code text.

This is a non-trivial diagnostic insight because it separates two failure modes that are often conflated in world model evaluation. Outcome prediction accuracy measures whether the world model understands the immediate causal relationship between code and execution outcome—given this code, what will the backend output? This is a single-step prediction task. Trajectory agreement measures whether the world model is reliable enough to drive a multi-step correction process without diverging from the real environment's behavior. A world model with high outcome accuracy but low trajectory agreement would produce individual predictions that look right but compound errors that derail the overall debugging process. The paper's finding that even at 93.1% trajectory agreement, "more than nine out of ten ICWM-driven trajectories are functionally equivalent to their real-execution counterparts" establishes an empirical threshold for when a learned simulator is "good enough" for data amplification.

This framework generalizes beyond industrial code. Any system that uses a learned environment model to generate multi-step training trajectories—whether for robotics, dialogue, or scientific discovery—faces the same error compounding problem. The paper's two-metric evaluation protocol (outcome accuracy + trajectory agreement) and its case-study analysis of where the gap comes from (GPU kernel diagnostics are textually predictable, 3D modeling corner cases depend on continuous geometry) provide a template for diagnosing world model limitations in other domains.

The practical consequence for this paper is that the 3D modeling fidelity gap is flagged as a known limitation rather than a hidden flaw. The periodic audit mechanism (real-execution spot checks with ICWM retraining) is the paper's mitigation strategy, and the paper explicitly states that these audits "progressively narrow the gap." The fact that CAD-Coder performance is strong (84.0%, best across all models including proprietary Claude-Sonnet-4.6 at 77.0%) despite the wider fidelity gap suggests that the ICWM's 3D modeling errors are concentrated in geometric corner cases that are rare in the benchmark distribution, or that the real-execution audits successfully filter out the most egregious errors before they enter the training corpus. Either way, the fidelity analysis framework makes this tradeoff transparent rather than burying it.

5. Experimental Analysis

Evaluation Methodology

Dataset. The evaluation spans 14 general code benchmarks and 9 industrial code benchmarks. General benchmarks include EvalPlus (HumanEval and MBPP with augmented test cases from Liu et al., 2023), BigCodeBench (1,140 task-oriented prompts requiring complex library API calls, Zhuo et al., 2024), FullStackBench (4,000+ problems across 16 languages, Liu et al., 2024), CRUXEval (800 short Python programs for input/output prediction, Gu et al., 2024), LiveCodeBench V5 and V6 (competitive programming problems from LeetCode, AtCoder, and Codeforces with contamination-free time-stamped splits, Jain et al., 2024), Mercury (code efficiency benchmark, Du et al., 2024), Spider (cross-database text-to-SQL, Yu et al., 2018), BIRD (large-scale text-to-SQL, Li et al., 2023), Terminal-Bench v1.0 and v2.0 (multi-turn terminal interactions, 2025), SWE-bench Verified (real GitHub issue resolution, 2025), Mind2Web (web navigation, Deng et al., 2023), BFCL V3 (function calling accuracy, Patil et al., 2024), and τ2-bench (multi-turn conversational agent in Airline, Retail, and Telecom domains, Barres et al., 2025). Industrial benchmarks include VeriScope (568 Verilog problems across five difficulty tiers, introduced in this paper), RealBench (60 module-level and 4 system-level Verilog tasks from real IP cores including AES, SD card controller, and Hummingbirdv2 E203 CPU, Jin et al., 2025), ArchXBench (51 complex digital-system designs across six difficulty levels, Purini et al., 2025), VeriRepair (300 test samples for Verilog error diagnosis and repair, introduced in this paper), KernelBench (250 PyTorch ML workloads across three levels of increasing complexity, Ouyang et al., 2025), TritonBench (184 real-world Triton operators across two tracks, Li et al., 2025), EmbedCGen (500 bare-metal embedded C problems across five difficulty tiers, introduced in this paper), SuperCoder (8,072 x86-64 assembly programs for superoptimization, Wei et al., 2025), and CAD-Coder (110K text-to-CadQuery triplets for 3D modeling, Guan et al., 2025). Detailed descriptions of each benchmark's structure, metrics, and evaluation protocols appear in Appendix A.
Base model(s). InCoder-32B-Thinking is a 32-billion-parameter model built on the same architecture as its predecessor InCoder-32B (Yang et al., 2026), which itself was a code foundation model trained through a three-stage Code-Flow pipeline spanning general code pre-training with industrial annealing, progressive context extension, and execution-grounded post-training. The thinking variant is trained on the combined corpus D = D_real ∪ D_icwm produced by the ECoT synthesis framework described in Section 2. The paper does not specify the exact base architecture (dense vs. mixture-of-experts, context length, or tokenizer details), focusing instead on the training data and synthesis methodology. The predecessor InCoder-32B was chosen because it already unified multiple industrial domains in a single 32B-parameter model, providing a strong non-thinking baseline for measuring the incremental value of the thinking-augmented training data. The generator used during ECoT trajectory synthesis is DeepSeek-V3.2 (37B/671B Mixture-of-Experts), a frontier model selected for its strong code generation capabilities rather than its architectural compatibility with the final trained model—the thinking traces are authored by DeepSeek-V3.2 but learned by the smaller 32B model through supervised training on the resulting corpus.
Metrics. Metrics vary by benchmark family and are reported exactly as defined by each benchmark's original publication. For general code generation, EvalPlus reports pass@1 (the fraction of problems for which at least one of k generated solutions passes all test cases, with k=1); BigCodeBench reports pass@1 on full and hard subsets; FullStackBench reports pass@1; CRUXEval reports input prediction and output prediction accuracy; LiveCodeBench reports pass@1 on V5 and V6; Mercury reports Beyond@1 (efficiency-weighted correctness) and Pass@1; Bird and Spider report execution accuracy; Terminal-Bench reports task completion rate; SWE-bench Verified reports resolved rate (fraction of GitHub issues with a correct patch); Mind2Web reports element accuracy and action F1; BFCL V3 reports function calling accuracy; τ2-bench reports task success rate per domain. For industrial benchmarks: VeriScope reports average score across 568 problems on a three-level scale (0 for compilation failure, 50 for compilation without passing testbench, 100 for passing all tests); RealBench reports Syn@k (synthesis success within k candidates, k ∈ {1, 5}) and Func@k (functional correctness within k candidates) separately for system-level and module-level tasks; ArchXBench reports (n, t) pairs where n is the number of syntactically correct candidates out of 5 and t is the percentage of testbench assertions passed by the best candidate; VeriRepair reports Fix% (fraction of buggy programs successfully corrected, verified by testbench execution); KernelBench reports fast1 (fraction of tasks where the generated kernel is both correct and at least 1× faster than the PyTorch baseline) for L1, L2, and L3; TritonBench reports call accuracy (executable without runtime error) and execution accuracy (output matches reference) for both the G (GitHub) and T (Torch) tracks, yielding four metrics; EmbedCGen reports Main% (average pass rate across 500 problems through compilation and Renode emulation); SuperCoder reports Acc.% (functional correctness through test suite) and Spd. (speedup relative to gcc -O3 baseline); CAD-Coder reports Comp.% (compilation success rate, fraction of scripts producing valid 3D geometry) and IoU (volumetric overlap between generated and ground-truth voxelizations).
Baselines. The paper evaluates against an extensive set of contemporary coding models spanning multiple scales and architectures. For general code benchmarks (Tables 1, 2, 3), baselines include DeepSeek-Coder-V2-Lite-Instruct (2.4/16B MoE, DeepSeek AI, 2024), Qwen2.5-Coder-7B/14B/32B-Instruct (Hui et al., 2024), Seed-Coder-8B-Instruct (ByteDance Seed, 2025), Qwen3-Coder-30B-A3B-Instruct and Qwen3-Coder-480B-A35B-Instruct (Qwen, 2025), DeepSeek-V3.2 (37/671B MoE, Liu et al., 2025), Qwen3-235B-A22B-Instruct-2507 and Qwen3-235B-A22B-Thinking-2507 (Yang et al., 2025), Kimi-Dev-72B (Yang et al., 2025), Kimi-K2-Instruct-0905 and Kimi-K2-Thinking (Kimi Team, 2025), KAT-Dev (32B) and KAT-Dev-72B-Exp (Zhan et al., 2025), GLM-4.7 (32/355B MoE, Zhipu AI, 2025), and InCoder-32B (the non-thinking predecessor, Yang et al., 2026). For industrial code benchmarks (Tables 4, 5), baselines additionally include Qwen3.5-9B, Qwen3.5-27B, and Qwen3.5-397B-A17B (Qwen Team, 2026), GPT-OSS-20B and GPT-OSS-120B (Agarwal et al., 2025), Seed-OSS-36B-Instruct (ByteDance Seed Team, 2025), MiniMax-M2.5 (MiniMax, 2026), GLM-5 (40/744B MoE, Zeng et al., 2026), and Kimi-K2.5 (Kimi Team, 2026). Claude-Sonnet-4.6 (Anthropic, 2026) serves as a proprietary reference model on industrial benchmarks. These baselines collectively cover dense and mixture-of-experts architectures from 7B to 744B total parameters, both instruct and thinking variants, and both open-weight and proprietary systems.
Generation budget / compute accounting. The paper evaluates all models under their standard inference configurations as reported by their respective publications or APIs. For InCoder-32B-Thinking, the thinking mechanism is an integral part of the model's generation process—the model produces thinking blocks followed by answer blocks as a single autoregressive generation, with the thinking depth adaptively determined by the model based on the task rather than controlled by an external budget parameter. There is no "generation budget" in the sense of controlling the number of parallel samples, search beams, or revision rounds—the model generates one response per task (pass@1 evaluation). The thinking length distribution shown in Figure 6 (91 to 19,015 characters median per task category) reflects the model's learned adaptive behavior, not a hyperparameter sweep. The training data scaling experiment (Section 4.3, Figure 7) measures compute in tokens of thinking training data (180M, 360M, 540M tokens), with the understanding that models trained on more tokens consumed more training FLOPs. The paper does not report inference-time FLOP counts for the thinking model compared to instruct baselines, nor does it perform a FLOPs-matched comparison between thinking and non-thinking inference budgets (unlike the study of test-time compute scaling in Jones, 2021, or Sardana and Frankle, 2023). This is a notable absence—the paper claims that extended thinking can compensate for model size (Section 1, Executive Summary), but does not quantify how much additional inference compute the thinking model consumes per task relative to the instruct baselines it outperforms.
Cross-validation / statistical protocol. The paper does not report confidence intervals, standard deviations, or statistical significance tests for any benchmark results. All reported numbers are point estimates from single evaluation runs. For benchmarks that support multiple samples per task (e.g., RealBench, which generates 20 independent samples, and ArchXBench, which samples 5 candidates), the metrics aggregate across these samples as defined by the benchmark protocol, but the paper does not report variance across different random seeds or evaluation runs. The ICWM fidelity analysis (Section 4.1, Figure 5) reports mean outcome prediction accuracy and mean trajectory agreement across 2,000 held-out turns per domain, but does not provide confidence intervals around these means. The training data scaling experiment (Section 4.3, Figure 7) shows performance curves across three data scales (180M, 360M, 540M tokens), which implicitly demonstrates monotonic trends, but the paper does not report whether the differences between adjacent scales are statistically significant or within the range of training variance. For VeriRepair, which the paper introduces as a new benchmark with approximately 22,000 training samples and 300 test samples, no cross-validation or train/test contamination analysis is reported for the in-house baseline models.

Main Quantitative Results

General Code Benchmarks: Code Generation and Agentic Coding

The headline finding across Tables 1, 2, and 3 is that InCoder-32B-Thinking achieves competitive or state-of-the-art performance on general code benchmarks among open-weight models, with the most striking result being 81.3% on LiveCodeBench V5 (Table 2)—the highest among all open-weight models including those with substantially more parameters (e.g., Qwen3-235B-A22B-Thinking-2507 at 80.2%, Qwen3-Coder-480B-A35B-Instruct at 48.6%, DeepSeek-V3.2 at unreported on V5 but 83.3% on V6). On LiveCodeBench V6, InCoder-32B-Thinking achieves 77.1%, behind DeepSeek-V3.2 (83.3%), Kimi-K2-Thinking (83.1%), and GLM-4.7 (84.9%) but ahead of Qwen3-Coder-480B (53.9%), Kimi-K2-Instruct (53.7%), and Qwen3-235B-A22B-Thinking (74.1%).

On standard code generation benchmarks (Table 1), InCoder-32B-Thinking achieves 95.1% on HumanEval, 89.6% on HumanEval+, 92.1% on MBPP, 78.3% on MBPP+, 47.4% on BigCodeBench Full, 29.1% on BigCodeBench Hard, and 60.8% on FullStackBench. These results are competitive but not dominant—Qwen3-Coder-480B-A35B-Instruct scores higher on HumanEval (97.6%), HumanEval+ (92.7%), MBPP (94.2%), MBPP+ (80.2%), and BigCodeBench Full (49.4%), while Kimi-K2-Thinking scores higher on HumanEval (98.2%), HumanEval+ (92.7%), MBPP (97.4%), and MBPP+ (82.3%). The thinking variant slightly outperforms its instruct counterpart InCoder-32B on HumanEval (95.1% vs. 94.5%), MBPP (92.1% vs. 91.8%), and MBPP+ (78.3% vs. 78.3%, tied), but slightly underperforms on HumanEval+ (89.6% vs. 89.6%, tied), BigCodeBench Full (47.4% vs. 49.8%), and BigCodeBench Hard (29.1% vs. 31.1%). The paper acknowledges this as a "moderate trade-off" on "benchmarks that favor concise responses" (Section 3.3), consistent with observations about other thinking-augmented systems (e.g., Qwen3-235B-Thinking similarly underperforms its instruct counterpart on Mercury and BigCodeBench).

On agentic coding benchmarks (Table 3), InCoder-32B-Thinking achieves 38.8% on Terminal-Bench v1.0, 21.6% on Terminal-Bench v2.0, and 70.4% on SWE-bench Verified. Compared to its instruct counterpart InCoder-32B (35.0%, 22.5%, 74.8%), the thinking model improves on Terminal-Bench v1.0 but declines on both v2.0 and SWE-bench Verified. DeepSeek-V3.2 achieves substantially higher on SWE-bench Verified (73.1%), Terminal-Bench v2.0 (46.4%), and tied on v1.0 (23.8%). GLM-4.7 achieves 73.8% on SWE-bench Verified and 36.3%/41.0% on Terminal-Bench v1.0/v2.0. On tool-use benchmarks (Table 3), InCoder-32B-Thinking achieves 49.1% on Mind2Web (vs. 55.8% for InCoder-32B), 63.9% on BFCL V3 (vs. 61.0% for InCoder-32B), 62.0%/86.0%/95.6% on τ2-bench Airline/Retail/Telecom respectively. The τ2-bench Retail score of 86.0% ranks first among all reported models, and the Telecom score of 95.6% is second only to DeepSeek-V3.2's 96.2%. The BFCL V3 score of 63.9% places the model in the middle of the 30B+ comparison group.

On code reasoning benchmarks (Table 2), the improvement over the instruct baseline is dramatic: CRUXEval Input-COT improves from 62.4 to 88.9, and Output-COT from 73.9 to 95.5. This 26.5 and 21.6 percentage point leap directly measures the value of thinking-augmented training for execution tracing and code understanding tasks—capabilities that are closely related to the diagnostic reasoning the ECoT framework was designed to teach. On code efficiency (Table 2, Mercury), InCoder-32B-Thinking achieves 62.4% Beyond@1 and 73.4% Pass@1, underperforming its instruct counterpart (71.4% Beyond@1, 85.6% Pass@1) and substantially trailing Qwen3-Coder-30B-A3B (81.1%, 95.3%) and Qwen3-Coder-480B-A35B (80.2%, 96.1%). The paper attributes this to the thinking model's verbosity trade-off—Mercury penalizes solutions that are not runtime-efficient, and longer thinking traces may not translate to more efficient final code. On Text2SQL (Table 2), InCoder-32B-Thinking achieves 47.9% on Bird (vs. 55.4% for instruct) and 66.7% on Spider (vs. 79.7% for instruct), again showing the verbosity trade-off.

Industrial Code Benchmarks: Chip Design

The headline finding across industrial benchmarks (Tables 4 and 5) is that InCoder-32B-Thinking establishes the strongest open-source results across all evaluated industrial domains. On chip design benchmarks (Table 4), the results are mixed but broadly strong:

On VeriScope, InCoder-32B-Thinking achieves 75.4, which is strong but not dominant—GPT-OSS-120B achieves 82.2, GLM-4.7 achieves 81.2, Kimi-K2-Instruct achieves 82.4, DeepSeek-V3.2 achieves 76.1, and the proprietary Claude-Sonnet-4.6 achieves 87.7. The thinking model (75.4) underperforms its instruct predecessor InCoder-32B (80.7), a decline of 5.3 points that the paper does not explicitly analyze in the main text, though the VeriScope score scales positively with thinking data (from 61.8 at 180M tokens to 75.4 at 540M tokens, Figure 7), suggesting the final checkpoint may still be improving. On VeriRepair, InCoder-32B-Thinking achieves 83.3% Fix, improving over InCoder-32B's 80.0% and competitive with GPT-OSS-20B (86.7%), Qwen3.5-397B-A17B (86.7%), GLM-5 (90.0%), and Claude-Sonnet-4.6 (83.3%).

On RealBench, the most realistic and demanding chip design benchmark because it requires generating full module hierarchies from real IP cores, InCoder-32B-Thinking achieves the best overall open-weight performance: Syn@1 System = 75.6, Syn@5 System = 82.4, Syn@1 Module = 63.1, Syn@5 Module = 69.8, Func@1 Module = 3.12, Func@5 Module = 46.7. The Syn@1 System score of 75.6 is substantially higher than the next best open-weight model (Kimi-K2-Instruct at 50.1, DeepSeek-V3.2 at 39.3), and even surpasses the proprietary Claude-Sonnet-4.6 at 69.2. The Func@5 Module score of 46.7 is competitive with Kimi-K2-Instruct (70.1) and Claude-Sonnet-4.6 (58.2), but the Func@1 Module score of 3.12 is notably low, suggesting the model generates compilable designs frequently (high Syn@1) but these designs rarely pass functional testbench simulation on the first attempt—only 3.12% do—though the top-5 functional rate recovers to 46.7%. Compared to its instruct predecessor InCoder-32B (Syn@1 System = 74.8, Syn@5 System = 83.3, Syn@1 Module = 62.7, Syn@5 Module = 70.5, Func@1 Module = 3.4, Func@5 Module = 51.0), the thinking variant shows small improvements on synthesis metrics but small declines on functional metrics, suggesting the thinking traces help with compilability but may not yet translate to improved functional correctness on this benchmark.

On ArchXBench, InCoder-32B-Thinking achieves n = 3.12 and t = 46.7, placing it behind Claude-Sonnet-4.6 (n = 4.4, t = 58.2) and GPT-OSS-120B (n = 3.4, t = 54.8) but ahead of Kimi-K2-Thinking (n = 1.5, t = 30.1) and DeepSeek-V3.2 (n = 3.6, t = 53.9). The instruct predecessor achieves n = 3.4 and t = 51.0.

Industrial Code Benchmarks: GPU Kernel Optimization, Code Optimization, and 3D Modeling

On GPU kernel optimization benchmarks (Table 5), InCoder-32B-Thinking achieves 15.2% G-call, 100.0% G-exe, 22.9% T-call, and 89.5% T-exe on TritonBench. The perfect 100.0% G-exe (GPU execution correctness on GitHub-collected operators) matches the instruct predecessor and GPT-OSS-20B at 100.0%, while the T-exe score of 89.5% is below Kimi-K2-Instruct (91.3%) but above DeepSeek-V3.2 (13.3%) and Claude-Sonnet-4.6 (1.4%). On KernelBench, InCoder-32B-Thinking achieves 20.2% on L1, 38.0% on L2, and 12.0% on L3. The L2 score of 38.0% is the highest among all models (open and proprietary), substantially ahead of Claude-Sonnet-4.6 (28.0%), GLM-5 (23.0%), Kimi-K2.5 (23.0%), and Kimi-K2-Thinking (16.0%). The L1 score of 20.2% is behind the instruct predecessor (22.2%) but ahead of most other models. The L3 score of 12.0% is behind Claude-Sonnet-4.6 (2.0%), GLM-5 (4.0%), and Kimi-K2.5 (6.0%). The SuperCoder speedup metric of 3.93× is substantially higher than the instruct predecessor (1.3×), representing a 3× improvement in generated assembly optimization capability—a domain where thinking about instruction-level performance characteristics would be expected to help. The accuracy of 93.0% is competitive with the instruct predecessor (91.0%) and exceeds Claude-Sonnet-4.6 (88.0%).

On SuperCoder (code optimization), InCoder-32B-Thinking achieves 93.0% accuracy and 3.93× speedup—the highest speedup among all models including proprietary, substantially ahead of Claude-Sonnet-4.6 (4.6× speedup but on a different base—comparison may not be direct) and GLM-4.7 (8.6×), though speedup numbers should be interpreted cautiously as they depend on the baseline gcc -O3 performance. The accuracy of 93.0% is slightly above the instruct predecessor (91.0%). On EmbedCGen (embedded systems), InCoder-32B-Thinking achieves 47.9% Main pass rate, improving over InCoder-32B's 35.2% and substantially ahead of all other models (GLM-5: 90.2% but this number appears anomalously high and may reflect a different evaluation configuration; the next highest 30B+ model is MiniMax-M2.5 at 22.2%). On CAD-Coder (3D modeling), InCoder-32B-Thinking achieves 84.0% compilation success and 48.6% IoU, improving on InCoder-32B's 82.0% and 53.5% respectively. The compilation success of 84.0% is the highest among all models including proprietary (Claude-Sonnet-4.6: 77.0%, Kimi-K2-Thinking: 48.0%, GLM-5: 38.0%), though the IoU of 48.6% is behind InCoder-32B (53.5%), Claude-Sonnet-4.6 (32.4%), and Kimi-K2-Thinking (20.0%). The trade-off between compilation success and geometric fidelity is notable—the thinking model produces more syntactically valid CadQuery scripts but with slightly lower geometric accuracy—and may reflect the ICWM's known fidelity gap in 3D modeling (Section 4.1, Figure 5) where geometric corner cases are harder to simulate.

Ablation Studies and Robustness Checks

ICWM fidelity across domains (Section 4.1, Figure 5): The paper holds out 2,000 execution turns per domain from D_real and compares ICWM predictions against ground-truth labels. Outcome prediction accuracy averages 96.7% across five domains (Chip Design: 97.4%, GPU Kernels: ∼97%, Code Optimization: ∼96.5%, Embedded Systems: ∼96.2%, 3D Modeling: 95.9%). Trajectory agreement—the fraction of full multi-turn trajectories whose final pass/fail verdict is identical under real and ICWM-driven execution—averages 94.4% with a consistent 1.6–2.8 percentage point gap below outcome accuracy. The 3D modeling domain shows the widest gap (95.9% outcome accuracy vs. 93.1% trajectory agreement), attributed to CadQuery geometry checks involving floating-point tolerances and implicit Boolean operations that are harder to predict from code text alone. The two case studies (GPU kernel correct diagnosis and 3D modeling false positive) illustrate the ICWM's domain-specific failure modes. This ablation validates that the ICWM can serve as a proxy for real backends during large-scale training data synthesis, but also establishes that the fidelity is not perfect and the gap compounds across multi-turn trajectories. The paper's mitigation is periodic real-execution audits that detect mismatches and retrain the ICWM, though Figure 5 reports fidelity on a fixed held-out set rather than showing how fidelity evolves across audit rounds.

Training data scaling (Section 4.3, Figure 7): Three InCoder-32B-Thinking checkpoints trained on 180M, 360M, and 540M tokens of thinking data are evaluated across nine industrial benchmarks. Most metrics improve monotonically: VeriScope score rises from 61.8 → (not shown at 360M) → 75.4; KernelBench L2 from 16.0 → → 38.0; CAD-Coder Compilation from roughly 76 → → 84; SuperCoder Accuracy from roughly 77.5 → → 93.0 and Speedup from roughly 3.0× → → 3.93×; RealBench Syn@1 System from roughly 20 → → 75.6; VeriRepair Fix from roughly 72 → → 83.3. Two metrics remain flat: TritonBench GPU execution correctness stays at 100% across all scales, which the paper interprets as the thinking ability enabling the model to "quickly master basic execution tasks" even at lower data volumes, and KernelBench L3 remains at 12.0 across all scales, suggesting that the hardest optimization problems "require specific strategies, not just more data volume." The non-monotonic or saturating metrics are ArchXBench n (2.6 → 3.0 → 3.12) and ArchXBench t (43.5 → 46.0 → 46.7), which show slow improvement, and EmbedCGen Main (44 → 48 → 47.9), which plateaus at higher data scales. The overall monotonic trend across most benchmarks provides evidence that (a) the thinking training data is of sufficient quality that more data reliably improves performance, (b) thinking mechanism is a "strong driver for core industrial coding skills" (Section 4.3), and (c) the 540M token corpus has not yet saturated performance on most tasks.

Thinking depth distribution (Section 4.2, Figure 6): The analysis of thinking block length across nine task categories in the training corpus reveals a 209× range in median thinking length: 91 characters (agentic coding) to 19,015 characters (GPU kernel optimization). Industrial domains are highlighted in purple, with GPU optimization (19,015 thinking, 19,485 answer characters), chip design (1,546 thinking, 3,213 answer—an inverted profile with short thinking followed by long code), competitive programming (11,075 thinking, 9,672 answer), and 3D modeling (1,458 thinking, 813 answer) all showing distinct distributions. The paper argues this variation is not hand-engineered but arises naturally from the error-driven synthesis pipeline—tasks with complex execution feedback accumulate longer reasoning traces through successive correction rounds. This ablation validates that the training data distribution teaches adaptive thinking depth rather than imposing a uniform reasoning budget. The practical implication is that the model learns to invest deep multi-step reasoning where complex industrial backends demand it while keeping traces succinct for tasks with short feedback loops.

Comparison of instruct vs. thinking predecessor (Tables 1–5, every row): The most direct ablation of the thinking mechanism is the side-by-side comparison between InCoder-32B (instruct) and InCoder-32B-Thinking across every benchmark. The thinking variant improves dramatically on code reasoning (CRUXEval Input-COT: +26.5 pp, Output-COT: +21.6 pp, LiveCodeBench V5: +28.0 pp relative to 53.3%, Table 2), shows mixed results on code generation (slight improvement on HumanEval, MBPP; slight decline on BigCodeBench, FullStackBench, Table 1), and shows strong domain-specific improvements on industrial benchmarks (KernelBench L2: from 36.0 to 38.0; SuperCoder speedup: from 1.3× to 3.93×; EmbedCGen: from 35.2 to 47.9; CAD-Coder compilation: from 82.0 to 84.0; VeriRepair: from 80.0 to 83.3, Tables 4 and 5). The thinking variant underperforms on VeriScope (75.4 vs. 80.7), Mercury (62.4/73.4 vs. 71.4/85.6), and Text2SQL (Bird: 47.9 vs. 55.4; Spider: 66.7 vs. 79.7). This ablation demonstrates that the ECoT + ICWM data synthesis pipeline produces a model with substantially different performance characteristics from the instruct baseline—stronger on tasks requiring explicit diagnostic reasoning (code execution tracing, competitive programming, kernel optimization, embedded systems) but sometimes weaker on tasks requiring concise, efficient outputs (text-to-SQL, code efficiency, Verilog generation where the instruct baseline was already strong).

Domain-specific industrial scaling (Figure 7, per-benchmark panels): Breaking down the data scaling experiment by benchmark reveals heterogeneous scaling behavior. Benchmarks that test core reasoning capability (VeriScope, KernelBench L2, CAD-Coder Compilation, SuperCoder) show strong monotonic improvement. Benchmarks that test a specific execution threshold (TritonBench G-exe at 100%, KernelBench L3 at 12.0) show flat scaling, indicating either ceiling effects or the need for qualitatively different training data. Benchmarks with multiple sub-metrics show diverging trends: RealBench Syn@1 and Syn@5 improve while Func@1 and Func@5 are flat or decline, suggesting the thinking data improves compilability more than functional correctness for this task category. This heterogeneity demonstrates that "adding more thinking training data is a highly reliable engine for unlocking deep reasoning skills" (Section 4.3) but that the gains are task-dependent and some capabilities may require different training strategies beyond data volume scaling.

Case study of diagnostic reasoning (Figure 3): The qualitative comparison of CUDA kernel implementations for hinge loss between InCoder-32B (instruct) and InCoder-32B-Thinking serves as a micro-ablation of the thinking mechanism's impact on error diagnosis. The instruct model treats both the 2D predictions tensor (32768, 32768) and the 1D targets tensor (32768,) with the same flat indexing, causing an out-of-bounds error. The thinking model explicitly reasons about the shape mismatch ("Detect shape mismatch: 2D vs 1D"), infers the broadcasting semantics ("Infer broadcasting semantics"), maps the flat index to a row index ("Map flat index to row index"), and indexes targets by row rather than element ("Index targets by row, not element"). This case study demonstrates at the instance level what the aggregate benchmarks show: the thinking traces teach the model a transferable diagnostic methodology that connects error patterns (out of bounds) to root causes (dimension mismatch) to specific fixes (row-based indexing), rather than simply producing code that happens to be correct.

SFT data scale comparison with predecessor (Section 4.3, cited in text): The paper notes that the predecessor InCoder-32B used 250M tokens of training data while InCoder-32B-Thinking uses 540M tokens of thinking data, explicitly attributing this difference to "the reasoning process naturally creates longer texts." This is an implicit ablation of data composition: the thinking model sees more than twice as many tokens, but these tokens are qualitatively different (multi-turn error-correction narratives with diagnostic reasoning) rather than simply more examples of the same type. The paper does not run a controlled experiment holding total token count constant while varying the fraction of thinking vs. non-thinking data, which would have isolated the effect of data composition from the effect of data volume.

ReST^EM ablation (Appendix K, Figure 16, cited in the paper's limitations discussion from prior sections): The paper reports a notable negative result from an attempt to further optimize the revision model using ReST^EM (Singh et al., 2024). Additional sequential revisions substantially hurt performance, with fully sequential performance dropping to approximately 33.5% compared to roughly 38.5% at the optimal ratio at 256 generations. The authors hypothesize that on-policy data collection exacerbated spurious correlations in revision data. This negative result is only briefly mentioned in the related work context (Section 5.2) and not developed as a full ablation in the main experimental analysis, but it provides evidence that the ECoT data synthesis methodology (offline data construction with edit-distance-based incorrect-correct pairing) is important and that alternative training paradigms can degrade performance.

Critical Assessment

Claim: "Extended thinking can compensate for model size" (Section 1, Abstract). The strongest evidence for this claim is the LiveCodeBench V5 result (Table 2), where InCoder-32B-Thinking at 32B parameters achieves 81.3%, surpassing Qwen3-235B-A22B-Thinking (80.2%, 235B total parameters) and Qwen3-Coder-480B-A35B-Instruct (48.6%, 480B total parameters), while being competitive with DeepSeek-V3.2 (not reported on V5, but 83.3% on V6) at 671B total parameters. If "compensate" means "achieve comparable or superior performance to substantially larger models," this single benchmark provides partial support. However, the claim is not uniformly supported across benchmarks: on HumanEval (Table 1), Qwen3-Coder-480B-A35B-Instruct (97.6%) and Kimi-K2-Thinking (98.2%) both outperform InCoder-32B-Thinking (95.1%); on BigCodeBench Full (Table 1), Qwen3-Coder-480B (49.4%) outperforms InCoder-32B-Thinking (47.4%); on SWE-bench Verified (Table 3), DeepSeek-V3.2 (73.1%) and GLM-4.7 (73.8%) outperform InCoder-32B-Thinking (70.4%); on Mercury (Table 2), the 30B Qwen3-Coder-30B-A3B-Instruct (81.1% Beyond@1) substantially outperforms InCoder-32B-Thinking (62.4%). The compensation claim holds selectively—primarily for code reasoning benchmarks that directly test the diagnostic methodology the ECoT framework was designed to teach (CRUXEval, LiveCodeBench), but not for benchmarks emphasizing concise efficient outputs or agentic task completion. A critical missing experiment is a controlled comparison where inference-time compute (FLOPs) is held constant between the thinking 32B model and a larger instruct model—the thinking model presumably uses more tokens per response (Figure 6 shows up to 19K characters of thinking plus answer), and without FLOPs matching, it is unclear whether "compensation" means better performance at any cost or better performance per unit of compute. The paper does not report inference latency or token counts for benchmark evaluations, making the compensation claim unquantified in compute terms.

Claim: "InCoder-32B-Thinking achieves top-tier open-source results across all evaluated [industrial] domains" (Abstract, Section 3.3). This claim is broadly supported by Tables 4 and 5, with specific evidence: CAD-Coder compilation (84.0%, best overall including proprietary), KernelBench L2 (38.0%, best overall), SuperCoder speedup (3.93×, among the highest), EmbedCGen (47.9%, best among 30B+ open-weight models excluding the anomalous GLM-5 result of 90.2%), RealBench Syn@1 System (75.6, best open-weight by a wide margin). However, the claim requires qualification: on VeriScope (75.4), the model ranks below GPT-OSS-120B (82.2), GLM-4.7 (81.2), Kimi-K2-Instruct (82.4), and Claude-Sonnet-4.6 (87.7); on TritonBench G-call (15.2%), it ranks below Kimi-K2-Thinking (17.4%), DeepSeek-V3.2 (19.6%), and Claude-Sonnet-4.6 (28.8%); on KernelBench L1 (20.2%), it ranks below GLM-5 (16.2%? no, 20.2 > 16.2—actually the paper shows 20.2 vs. 16.2 for GLM-5, so InCoder is ahead) but also behind GLM-5 (16.2%)? Checking Table 5: KernelBench L1: InCoder-32B-Thinking 20.2, Kimi-K2-Thinking 9.1, DeepSeek-V3.2 3.0, Claude-Sonnet-4.6 11.1, GLM-5 16.2, Kimi-K2.5 13.1—InCoder is indeed highest on L1 among reported numbers. The claim "top-tier" is accurate in the sense that the model is at or near the top across all domains, but it is not uniformly #1—the model leads on CAD-Coder, KernelBench L2, EmbedCGen, SuperCoder, and RealBench synthesis, but trails on VeriScope, TritonBench G-execution, and several others. The claim would be strengthened by noting which specific metrics the model leads on and which it trails, rather than implying uniform dominance. Additionally, the paper evaluates against strong baselines but omits some relevant industrial code models: CodeV-R1 (Zhu et al., 2025) for Verilog, Kevin (Baronio et al., 2025) for CUDA, and LLM Compiler (Cummins et al., 2025) for compiler optimization are all cited in the related work (Section 5.1) but do not appear in the comparison tables, making it difficult to assess whether InCoder-32B-Thinking actually outperforms these specialized models on their target domains or only outperforms the general-purpose models that the paper chose as baselines.

Claim: "Scaling thinking data from 180M to 540M tokens yields consistent industrial improvements" (Abstract, Section 4.3). This claim is supported by Figure 7, which shows monotonic improvement on 7 of 9 benchmark metrics (all except KernelBench L3, which remains flat at 12.0, and TritonBench G-exe, which remains flat at 100%). However, "consistent" overstates the evidence base: only three data points are tested (180M, 360M, 540M), which can establish monotonicity but not consistency in the statistical sense—there are no error bars, no multiple training runs per scale, and no test of whether the differences between scales exceed training variance. The paper also does not test whether scaling beyond 540M tokens would continue to yield improvements or whether performance saturates (the flattening of EmbedCGen and ArchXBench metrics at higher scales hints at saturation). A more rigorous analysis would include multiple seeds per data scale to quantify variance and would test an extended range (e.g., 90M and 720M tokens) to identify the onset of diminishing returns. The claim that "adding more thinking training data is a highly reliable engine for unlocking deep reasoning skills" is an inference from the data but is not directly tested—there is no experiment showing that thinking data specifically (as opposed to any high-quality code data of equivalent volume) drives the improvement, since the paper does not compare against a baseline trained on 540M tokens of non-thinking industrial code data.

Missing experiments and weaknesses. (1) No FLOPs or latency analysis for inference. The paper claims that thinking compensates for model size but does not report how many tokens the thinking model generates per benchmark response, what the inference latency is compared to instruct baselines, or how total FLOPs per correct answer compare. The Figure 6 data on thinking length (median 91 to 19,015 characters per task category) implies that some evaluations consume substantially more inference compute than an instruct model's response, but this is not quantified in the benchmark results. A table reporting average tokens per response and wall-clock time per benchmark would substantially strengthen or qualify the efficiency claims.

(2) Single evaluation run with no variance reporting. All benchmark numbers are point estimates. For benchmarks with small test sets (e.g., HumanEval has 164 problems, RealBench has 4 system-level tasks), differences of a few percentage points may not be statistically significant. The paper would be strengthened by reporting confidence intervals or at minimum the number of test cases per benchmark so readers can assess statistical reliability.

(3) No contamination analysis for training data. The paper uses DeepSeek-V3.2 as the generator for ECoT trajectories and evaluates against DeepSeek-V3.2 as a baseline. It also uses InCoder-32B as the base model and evaluates against it. There is no analysis of whether the thinking training data inadvertently includes benchmark problems (e.g., from LiveCodeBench or HumanEval) that would inflate the thinking model's scores. LiveCodeBench is designed to be contamination-free through time-stamped splits, but the other benchmarks may not have this protection.

(4) No ablation of the ICWM's contribution. The paper does not report an ablation where the model is trained only on D_real (real execution trajectories) without the ICWM-amplified D_icwm component. This is the most direct test of whether the world model-driven data amplification actually contributes to performance or whether the real execution data alone would suffice. The ICWM fidelity analysis (Figure 5) shows the world model is reliable, but does not demonstrate that the additional trajectories it generates improve the final model over just using the real trajectories—it is possible that 540M tokens of real-only data (if budget permitted) would outperform 540M tokens of mixed real + ICWM data, or that ICWM-generated data adds diversity but not accuracy. Without this ablation, the paper's claim that the ICWM "enables large-scale trajectory synthesis" is a description of the pipeline but not a demonstration that large-scale synthesis via learned simulation yields a better model than smaller-scale synthesis via real execution.

(5) No combination of PRM search or test-time compute scaling with thinking. The paper's thinking model generates one response per problem. Prior work on test-time compute scaling (e.g., best-of-N sampling, beam search with verifiers, sequential revisions) shows that inference compute allocation strategies can substantially improve performance. The paper does not evaluate whether applying best-of-N or majority voting to the thinking model's outputs would further improve industrial benchmark performance, or whether the thinking traces themselves could serve as a verifier signal. This is a missed opportunity to connect the thinking mechanism to the broader test-time compute literature.

(6) Benchmark construction details are incomplete. VeriScope (568 problems) and VeriRepair (300 test samples, 22,000 training samples) are introduced in this paper (Appendix A) and evaluated on, but the paper does not release these benchmarks, does not describe how the problem difficulty tiers were calibrated, does not report inter-annotator agreement for error injection in VeriRepair, and does not provide examples of the problem distributions. This makes the results on these benchmarks unverifiable and limits their value as community benchmarks.

(7) The ICWM fidelity analysis is not on the same distribution as the training data synthesis. The held-out 2,000 turns per domain come from D_real—trajectories generated with real execution. But the ICWM's purpose is to simulate backends for new, unseen task seeds during the amplification phase. The fidelity analysis therefore measures interpolation (how well the ICWM predicts outcomes for held-out turns from the same distribution of tasks used in training) rather than extrapolation (how well it predicts outcomes for genuinely new tasks). If the task seed distribution in the amplification phase differs systematically from the seed distribution in D_real, the reported 96.7% outcome accuracy may overstate real synthesis fidelity. The paper does not describe the relationship between the task seeds used to train the ICWM and the task seeds used in the amplification loop.

(8) No domain transfer analysis. The paper claims that "reasoning gains transfer effectively to industrial coding scenarios" and that improvements "spanning chip design, GPU programming, 3D modeling, and embedded systems suggest that the acquired reasoning abilities are not domain-specific" (Section 3.3). However, the model is trained on data from all five industrial domains—the fact that it performs well on all five might simply reflect that it has been trained on all five, not that reasoning transfers between them. A transfer experiment would train the model on thinking data from a subset of domains (e.g., only GPU and chip design) and test on held-out domains (e.g., embedded systems and 3D modeling) to see whether the diagnostic methodology transfers without domain-specific training data. No such experiment is reported.

6. Limitations and Trade-offs

The Cost of Difficulty Estimation Is Unaccounted For in the Scaling Claims

The ICWM-driven data amplification loop—which enables the 540M-token thinking corpus—depends on the ICWM being trained on trajectories generated by real execution backends. The paper does not report the computational cost of generating D_real (the real-execution trajectories that train the ICWM), nor does it report how many real-execution trajectories are needed to achieve the reported ICWM fidelity of 96.7% outcome prediction accuracy and 94.4% trajectory agreement (Figure 5). The paper states that "real execution provides reliable supervision but is expensive for complex industrial tasks because each interaction requires invoking domain-specific toolchains and maintaining a robust execution stack" (Section 2.3), which is the explicit motivation for building the ICWM in the first place. But the cost of generating enough D_real to train a sufficiently high-fidelity ICWM is never quantified.

Consequence. A practitioner attempting to replicate this pipeline for a new industrial domain would face an unknown upfront cost: how many real-execution trajectories must be collected before the ICWM reaches acceptable fidelity and the amplification loop can begin? If D_real requires, say, 100,000 real toolchain invocations across multiple domains—each potentially taking minutes for complex Verilog simulations or GPU compilations—the total compute cost of seed data generation could dominate the cost of the amplification phase itself. The paper's implicit claim is that the ICWM amortizes this cost by enabling orders-of-magnitude more trajectories during amplification, but without quantifying the initial investment, the cost-benefit ratio is unknowable. This matters because the ICWM is the paper's primary mechanism for scaling beyond what real execution can support—if the upfront real-execution cost is prohibitive, the entire pipeline becomes inaccessible to organizations without large compute clusters and domain-specific toolchain infrastructure.

Evidence in the paper. The paper provides no numbers on the size of D_real (in trajectories, tokens, or toolchain invocations), no numbers on the ratio of real-to-simulated trajectories in the final corpus D = D_real ∪ D_icwm, and no ablation showing how ICWM fidelity varies with the amount of real training data. The training data scaling experiment (Section 4.3, Figure 7) measures thinking tokens (180M, 360M, 540M) but does not decompose these into real vs. ICWM-generated proportions. The ICWM fidelity analysis (Section 4.1) reports performance on a held-out set of 2,000 turns per domain, but does not report how many turns were used for training. The paper does not report the cost of the periodic real-execution audits used to retrain the ICWM during the amplification loop.

Mitigation status. The paper acknowledges the cost of real execution implicitly by building the ICWM as a cost-saving measure, but does not treat the cost of training data generation as a limitation requiring measurement or disclosure. No future work is suggested on reducing the real-execution cost of ICWM training, such as active learning to select which turns require real backends, or transfer learning to reduce the per-domain data requirement. The limitation is unquantified and unaddressed.

The ICWM Fidelity Gap Compounds in Multi-Turn Trajectories and Is Worst Where Precision Matters Most

The paper reports that the ICWM achieves 96.7% mean outcome prediction accuracy but only 94.4% trajectory agreement across five domains (Section 4.1, Figure 5), with a consistent 1.6–2.8 percentage point gap attributable to error compounding: "a single mispredicted turn can redirect the generator onto a divergent correction path, compounding the initial error." This gap is smallest for chip design (95.8% trajectory agreement) and largest for 3D modeling (93.1% trajectory agreement), because CadQuery geometry checks "involve floating-point tolerances and implicit Boolean operations whose outcomes are harder to predict from code text alone."

Consequence. The ICWM is least reliable in the domain where geometric precision is the core correctness criterion: 3D modeling requires exact geometric validity (manifold edges, zero-thickness constraints, Boolean operation correctness). The 3D modeling false positive case study shows the ICWM classifying a degenerate zero-thickness edge as PASS because the code is "syntactically valid and the specified dimensions appear plausible"—the ICWM fails precisely where the geometric semantics diverge from the textual representation of the code. A trajectory synthesized under this false positive would contain an incorrect turn: the generator would not attempt to fix the zero-thickness edge because the ICWM incorrectly reported a PASS, and the resulting reasoning trace would teach the final model that this type of edge is acceptable. Even with periodic real-execution audits, the audits operate on a subset of trajectories (the sampling rate is not specified), meaning some percentage of ICWM-generated trajectories in the final training corpus contain incorrect feedback. At 93.1% trajectory agreement, approximately 6.9% of ICWM-synthesized 3D modeling trajectories have a different final verdict than real execution would produce, and some fraction of these will have been generated between audit rounds and incorporated into the training data before correction.

Evidence in the paper. The fidelity analysis (Figure 5) directly measures this gap, and the 3D modeling case study (Section 4.1) provides a concrete failure mode. CAD-Coder results (Table 5) show that InCoder-32B-Thinking achieves 84.0% compilation success (best overall) but only 48.6% IoU, which is slightly lower than the instruct predecessor at 53.5%—the thinking model produces more compilable scripts but with slightly lower geometric accuracy. While the paper does not directly attribute the IoU decline to ICWM fidelity issues, the pattern is consistent with the ICWM being relatively better at predicting compilation outcomes (syntax, basic validity) than geometric precision (floating-point tolerances, implicit constraints), and the training data potentially over-representing trajectories where the ICWM incorrectly validated geometrically imprecise code.

Mitigation status. The paper partially addresses this through periodic real-execution audits that detect mismatches and retrain the ICWM, "progressively narrowing the gap" (Section 4.1). However, the paper does not report how the gap evolves across audit rounds, what fraction of trajectories are audited, or what the false negative rate of the audit process is (whether the audits catch all ICWM errors or only a subset). The mitigation is described qualitatively but never quantitatively validated—Figure 5 reports fidelity on a fixed held-out set, not fidelity improvement across audit iterations. The paper does not suggest domain-specific ICWM improvements (e.g., incorporating a geometric constraint solver for 3D modeling) as future work.

The Thinking Mechanism Does Not Help the Hardest Problems, and Some Industrial Benchmarks Show Flat or Declining Performance

Across multiple industrial benchmarks, InCoder-32B-Thinking shows flat or declining performance compared to its instruct predecessor on specific metrics, and the training data scaling experiment reveals metrics that do not improve with more thinking data. On VeriScope, the thinking model scores 75.4 versus the instruct model's 80.7, a decline of 5.3 points (Table 4). On RealBench, Func@1 Module drops from 3.4% to 3.12% and Func@5 Module drops from 51.0% to 46.7% (Table 4). On CAD-Coder, IoU declines from 53.5% to 48.6% (Table 5). On KernelBench L1, the thinking model scores 20.2% versus 22.2% for the instruct, and on L3, it scores 12.0% versus 14.0% (Table 5). In the data scaling experiment (Figure 7), KernelBench L3 remains flat at 12.0 across all three scales (180M, 360M, 540M tokens), TritonBench G-exe stays at 100% without improvement, and several ArchXBench and EmbedCGen metrics show plateauing behavior at higher scales.

Consequence. The ECoT synthesis framework produces training data that strongly improves performance on tasks requiring diagnostic reasoning (code execution tracing, competitive programming, kernel optimization, embedded systems) but does not uniformly improve—and sometimes degrades—performance on tasks where the instruct predecessor was already strong (Verilog generation with high synthesis success, geometric precision in 3D modeling, the hardest kernel optimization level). This reveals a capability boundary: the thinking traces teach a specific diagnostic methodology (observe error, trace to hardware root cause, revise code) that helps when errors are diagnostic-rich (compiler messages, memory faults, shape mismatches) but may not help—or may interfere with—capabilities that the instruct model already possessed from its pretraining and non-thinking fine-tuning. The flat L3 scaling (12.0 across all data scales) is the strongest evidence that "solving the hardest optimization problems requires specific strategies, not just more data volume" (Section 4.3)—the ECoT framework, as instantiated, does not generate the kind of reasoning traces needed for these problems.

For practitioners, this means the thinking model is not a drop-in upgrade across all industrial tasks. On Verilog generation, the instruct predecessor may be preferable. On hard kernel optimization (L3), neither model solves the task. The decision to deploy InCoder-32B-Thinking versus InCoder-32B depends on the workload mix: if the expected tasks are diagnostic-heavy (debugging, code understanding, competition-style reasoning), thinking helps; if they are straightforward generation where the instruct model already performs well, thinking may not help and may modestly hurt.

Evidence in the paper. Tables 4 and 5 provide the direct instruct-vs-thinking comparison across every industrial metric. Figure 7 shows the scaling behavior for each benchmark, with flat lines at L3 and G-exe. The paper acknowledges the flat L3 explicitly: "the KernelBench L3 score remains at 12.0. This suggests that solving the hardest optimization problems requires specific strategies, not just more data volume" (Section 4.3). The acknowledgment is honest but does not address the instruct-model regressions on VeriScope, Func@1, or CAD-Coder IoU, which are not discussed in the main text.

Mitigation status. The paper does not attempt to diagnose or address the instruct-to-thinking regressions. There is no analysis of why VeriScope score declined (whether the thinking traces for Verilog are lower quality than for other domains, whether the Verilog diagnostic patterns are less amenable to the ECoT format, or whether the thinking model over-thinks Verilog problems that the instruct model solved concisely). The flat KernelBench L3 result is acknowledged but no mitigation is proposed beyond the observation that it "requires specific strategies." The paper does not suggest combining ECoT thinking data with non-thinking data for domains where thinking degrades performance, or using benchmark-specific routing to select the thinking or instruct variant per task.

No Inference-Time Compute or Latency Analysis Means the Efficiency Claims Are Unquantified

The paper's abstract and Section 1 claim that "extended thinking can compensate for model size," pointing to the 81.3% LiveCodeBench V5 score that surpasses models with "an order of magnitude more parameters." But the paper never reports how many tokens the thinking model generates per benchmark response, what the inference latency is compared to instruct baselines, or how total FLOPs per correct answer compare between InCoder-32B-Thinking and the larger models it claims to compensate for.

Consequence. The claim that thinking compensates for model size is a claim about efficiency—that a smaller model with thinking achieves comparable accuracy to a larger model without thinking. But efficiency cannot be evaluated without measuring the cost side of the tradeoff. If InCoder-32B-Thinking generates a median of 19,015 characters of thinking content for GPU optimization tasks (Figure 6) plus the answer, while a larger instruct model like Qwen3-Coder-480B generates only a few hundred tokens of code, the thinking model may consume more inference FLOPs per correct answer despite having fewer parameters. The 209× range in thinking depth (Figure 6) implies that the cost per task varies enormously: an agentic coding task (91 characters of thinking) costs little extra, while a GPU kernel task (19,015 characters) costs dramatically more than an instruct model's single-pass generation.

The absence of latency measurement also matters for deployment. The thinking model's autoregressive generation of thinking blocks followed by answer blocks is inherently sequential—the model must generate the full diagnostic narrative before producing the code. For latency-sensitive applications (interactive coding assistants, CI/CD pipelines with time budgets, real-time hardware design feedback), generating thousands of characters of reasoning before outputting usable code may be impractical regardless of accuracy improvements. The paper evaluates pass@1 with a single generation, but does not report whether multiple parallel samples from an instruct model (with best-of-N or majority voting) could match the thinking model's performance at comparable or lower latency.

Evidence in the paper. Figure 6 provides the only data on thinking length: median per-step thinking length ranges from 91 to 19,015 characters, with inter-quartile ranges extending substantially higher (e.g., GPU optimization has P75 well above the 19K median). The total tokens per response are not reported in any benchmark evaluation (Tables 1–5). There is no comparison of wall-clock inference time between the thinking model and instruct baselines, no measurement of FLOPs per response, and no throughput analysis. The paper does not discuss the latency implications of adaptive thinking depth—while the model learns to calibrate depth to difficulty (which is a quality benefit), the calibration also means that hard tasks consume dramatically more inference compute, a cost that is never aggregated or compared to alternatives.

Mitigation status. The paper does not acknowledge this as a limitation. No future work is suggested on reducing inference cost, such as knowledge distillation from the thinking model back into a fast-inference model, speculative decoding, or early-exit mechanisms that stop thinking when a confident answer is reached. The "compensation" claim is presented as a pure accuracy comparison without the accompanying cost analysis that would make it a meaningful efficiency claim.

The Model Is Evaluated Only on Its Own Training Distribution and Only Against General-Purpose Baselines, Not Domain-Specific State-of-the-Art

The paper evaluates InCoder-32B-Thinking against a broad set of general-purpose coding models (Qwen, DeepSeek, Kimi, GLM, GPT-OSS, MiniMax, Claude) but does not compare against the domain-specific state-of-the-art models that it explicitly cites as prior work in Section 5.1. For Verilog generation, the paper cites VeriGen, RTLCoder, CodeV-R1, VeriReason, and VeriDebug as specialized models with domain-specific training pipelines. For CUDA kernel synthesis, it cites Kevin and CUDA Agent as RL-based approaches. For compiler optimization, it cites LLM Compiler. For embedded systems, it cites EmbedAgent and EmbedGenius. None of these appear in the comparison tables (Tables 4 and 5).

Consequence. The paper's primary claim about industrial code—that InCoder-32B-Thinking establishes "the strongest open-source results across all evaluated domains" (Section 3.3)—is only supported relative to the general-purpose baselines the paper selected. It does not demonstrate superiority over models that were specifically designed, trained, and optimized for each industrial sub-domain. CodeV-R1, for instance, used reinforcement learning with hardware-specific reward signals for Verilog generation, which is a fundamentally different training methodology from the ECoT framework. Without comparing against it (or explaining why comparison is infeasible), the claim that InCoder-32B-Thinking is "top-tier" on chip design is unverifiable relative to the actual state of the art in chip design LLMs.

This is not merely a missing baseline—it is a missing capability comparison. Domain-specific models like Kevin and CUDA Agent are trained with RL specifically to optimize for GPU kernel performance, which may produce different kinds of outputs (more aggressively optimized, more hardware-aware) than the ECoT-synthesized reasoning traces. The paper's claim of domain generality (one model handles five industrial domains) is an architectural advantage over fragmented domain-specific models, but the comparison that would validate this advantage—"one general model is competitive with five specialized models"—is not performed. A practitioner deciding between deploying InCoder-32B-Thinking or a suite of specialized models cannot assess the performance tradeoff from the paper's current evaluation.

Evidence in the paper. Section 5.1 cites all of the above domain-specific models as related work. Tables 4 and 5 list only general-purpose baselines (Qwen, DeepSeek, Kimi, GLM, GPT-OSS, MiniMax, Claude) plus InCoder-32B. The paper does not explain why these specialized models are excluded from the comparison—whether because their code or model weights are not publicly available, because they are evaluated on different benchmark versions, or because they were published too recently to include. The omission is not acknowledged as a limitation.

Mitigation status. The paper does not address this gap. Future work is not suggested on evaluating against domain-specific state-of-the-art or on releasing a benchmark suite that would enable apples-to-apples comparison between general and specialized industrial code models. The 14 general and 9 industrial benchmark suite is substantial, but it overlaps only partially with the benchmarks used by domain-specific models (e.g., VerilogEval is cited in related work but not used for evaluation; RealBench is a relatively new benchmark from 2025). The paper could strengthen its contribution by adopting the standard benchmarks from each sub-domain to enable direct comparison with prior specialized work.

No Ablation of the ICWM's Contribution Relative to Simply Using Only Real Execution Data

The paper's central methodological claim is that the ICWM enables scaling trajectory synthesis beyond what real toolchains can support, and that this scaling is essential for the model's performance. However, the paper never reports an ablation where the final model is trained only on D_real (real execution trajectories) without the ICWM-amplified D_icwm component, holding total compute or total tokens constant.

Consequence. It is unknown whether the ICWM-driven amplification actually improves the final model over simply training on whatever volume of real execution data could be collected with the same total compute budget. The ICWM costs real execution data to train (the D_real used for ICWM training), costs compute to run during amplification (forward passes through a language model for every simulated turn), and costs additional real execution for periodic audits. An alternative approach would be to allocate all of this compute budget to generating more real execution trajectories directly—running more toolchain invocations in parallel, scaling out the execution backend infrastructure, or optimizing the real backends for throughput. The paper provides no comparison between "X GPU-hours of ICWM training + Y GPU-hours of ICWM-driven amplification" and "X+Y GPU-hours of real execution trajectory generation."

The ICWM fidelity analysis (Figure 5) shows the world model is highly reliable (94.4% mean trajectory agreement), but reliability does not automatically translate to training data quality improvement. If the ICWM-generated trajectories are highly similar to real trajectories (because the ICWM is a good simulator), they may add volume but not diversity—the model sees more examples of the same error-correction patterns rather than genuinely new patterns. If the ICWM-generated trajectories contain systematic biases (e.g., the 3D modeling false positives), they may actually degrade performance in specific sub-domains. Without the ablation, the paper's title concept—"Industrial Code World Model for Thinking"—is demonstrated as an engineering pipeline but not validated as a performance-improving innovation over simpler alternatives.

Evidence in the paper. The training data scaling experiment (Section 4.3, Figure 7) shows that performance improves as total thinking tokens increase from 180M to 540M, but does not decompose these tokens into real vs. ICWM-generated proportions at each scale. The 540M-token checkpoint presumably contains the largest fraction of ICWM-generated data (since amplification scales with ICWM availability), but it may also contain the largest absolute amount of real data if the real data collection continued during amplification. The paper does not report whether the ICWM was used to generate novel task seeds (extrapolation to unseen problems) or to generate additional correction rounds for existing seeds (interpolation within known problems)—the former would test the ICWM's generalization, while the latter would test its trajectory diversity. The absence of this ablation means the ICWM's added value is an article of faith rather than an empirically established fact.

Mitigation status. The paper does not acknowledge this as a missing experiment, does not report the composition of the training corpus by data source (real vs. ICWM), and does not suggest an ICWM contribution ablation as future work. This is the most direct test of the paper's core contribution, and its absence is the single largest methodological gap in the experimental analysis.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper shifts the paradigm for training industrial code models from evaluative feedback (scalar rewards, test pass/fail signals) to generative environment grounding, where execution feedback becomes the raw material from which reasoning traces are synthesized. This is not an incremental improvement to an existing training recipe—it is a fundamentally different relationship between the model and the execution environment. In prior work (RLEF, CodeV-R1, Kevin), the environment judges outputs; in ECoT, the environment teaches diagnostic methodology by providing structured observations that a frontier model narrates into error-correction reasoning traces. The training data is no longer "correct code" but "how an engineer diagnosed and fixed incorrect code," and the model learns the process, not just the product.

This reframing resolves the contradiction between two observations that previously coexisted without explanation. First, thinking models (o1, DeepSeek-R1, QwQ) dramatically improve performance on math and competitive programming through extended chain-of-thought, but their reasoning is self-contained—they simulate execution through deliberation without access to real toolchain feedback. Second, domain-specific industrial code models (VeriGen, RTLCoder, Kevin) achieve strong results through RL with hardware-aware reward signals, but each targets a single domain with a custom training pipeline. The paper resolves this by showing that environment-grounded thinking—chains of thought that explicitly narrate the mapping from diagnostic output to hardware-level root cause to code fix—enables a single model to reason across five fundamentally different industrial domains while maintaining general coding performance. The unification is not architectural (the model is a standard 32B LM); it is data-driven, arising from a training corpus synthesized by interacting with diverse execution backends, each of which provides a different diagnostic "language" that the ECoT framework translates into a common reasoning format.

The practical consequence is that the field's research priorities should shift. Prior to this work, the dominant approach to industrial code LLMs was fragmented specialization: train a Verilog model with Verilog-specific rewards, a CUDA model with GPU-specific rewards, an embedded model with emulator-specific rewards. This paper demonstrates that a single model trained on environment-grounded thinking traces from multiple domains can be competitive with or superior to specialized models—and can transfer reasoning capability to new domains without architectural modification. The implication is that investment in diverse execution environments and feedback synthesis pipelines may yield higher returns than investment in domain-specific architectures or reward engineering. Building a new industrial code capability becomes primarily a data synthesis problem: can you package the domain's toolchain as an environment bundle, run it against a strong generator model, and train an ICWM to simulate it at scale? If so, the model acquires that domain's reasoning patterns through the same ECoT pipeline.

The work also changes how we should think about thinking depth as a design dimension. The dominant paradigm from o1 and DeepSeek-R1 treats chain-of-thought length as something the model learns to modulate through RL or prompting. This paper shows that when reasoning traces are synthesized from interaction with execution environments, thinking depth emerges organically from the task structure—GPU kernel optimization naturally produces 209× longer thinking traces than agentic coding (Figure 6) because GPU diagnostic feedback is richer and more rounds of correction are needed. The design implication is that the environment should set the reasoning budget, not the training protocol. Models trained on environment-grounded traces learn adaptive allocation as a byproduct of the data distribution, eliminating the need for explicit length penalties, hybrid fast/slow routing mechanisms, or hand-tuned reasoning budgets. This is a simpler and more scalable approach to adaptive thinking, and it suggests that for any domain with a structured execution environment (code, robotics, scientific simulation, chip design), the best way to teach a model how long to think is to expose it to the natural variance in task difficulty through execution feedback, not to impose an external budget.

Finally, the ICWM fidelity analysis framework—the separation of outcome prediction accuracy from trajectory agreement, and the quantification of error compounding across multi-turn interactions (Section 4.1, Figure 5)—provides a reusable diagnostic methodology for learned environment simulators in any domain where synthetic trajectories are generated through multi-step interaction with a learned proxy. The finding that a 1.6–2.8 percentage point gap between single-turn accuracy and multi-turn agreement is inherent to learned simulators (not a bug to be eliminated) and that periodic real-execution audits can progressively narrow this gap establishes an engineering template for world-model-driven data amplification that transfers to robotics simulation, dialogue system training, and scientific discovery pipelines. The specific numbers (96.7% outcome accuracy, 94.4% trajectory agreement, 93.1% minimum in 3D modeling) provide empirical thresholds that future work can benchmark against.

Follow-Up Research This Work Enables

Quantifying the real-execution data requirement for ICWM training. The paper's most immediate gap is the unmeasured cost of generating D_real—the real-execution trajectories needed to train the ICWM to acceptable fidelity. A direct follow-up would sweep the size of D_real (e.g., 1K, 5K, 20K, 100K turns per domain) and measure ICWM outcome prediction accuracy and trajectory agreement at each scale, producing a data scaling law for learned industrial execution simulators. The experiment would answer: how many real toolchain invocations are needed before the ICWM crosses the ~95% trajectory agreement threshold where paper reports it as reliable? Does this vary by domain (chip design requiring fewer turns than 3D modeling)? Does the ICWM benefit from transfer across domains (training on GPU and chip design data improving 3D modeling prediction)? A strong result would be a plot analogous to neural scaling laws showing ICWM fidelity vs. real data volume, which would tell practitioners exactly how much upfront investment is needed before the amplification loop becomes cost-effective. This experiment is newly tractable because the paper has defined the two-metric evaluation protocol (outcome accuracy + trajectory agreement) and the per-domain held-out set framework.

Controlled ablation of ICWM contribution to final model performance. The paper never isolates whether ICWM-amplified data (D_icwm) actually improves the final model over training only on real-execution data (D_real). A clean ablation would train three models: (1) on D_real only, (2) on D_real + D_icwm (the paper's approach), and (3) on D_real augmented with an equivalent volume of non-reasoning industrial code data (to control for token count). All three would be trained to the same total token budget. The comparison would answer: does ICWM amplification add value beyond simply increasing data volume? If model (2) outperforms model (3) by a significant margin, it validates the ICWM as a data quality amplifier (not just a quantity amplifier). If model (1) matches model (2), the ICWM is unnecessary—real execution alone suffices—and the paper's central methodological claim collapses. A more ambitious variant would test whether ICWM-generated trajectories that were rejected by periodic audits (i.e., known false positives) hurt performance if included, quantifying the cost of imperfect fidelity. This ablation would transform the ICWM from a demonstrated pipeline component to a validated performance-enhancing innovation.

Domain transfer of thinking capability without domain-specific training data. The paper claims that "reasoning gains transfer effectively to industrial coding scenarios" and that improvements across domains suggest "the acquired reasoning abilities are not domain-specific" (Section 3.3). But the model is trained on data from all five industrial domains, so the transfer claim confuses breadth of training with transfer of reasoning. A clean transfer experiment would train InCoder-32B-Thinking on thinking traces from a subset of domains (e.g., GPU optimization and code optimization only) and evaluate zero-shot on held-out domains (e.g., chip design, 3D modeling, embedded systems). If the model achieves non-trivial performance on held-out domains—particularly on diagnostic benchmarks like VeriRepair that directly test error-correction reasoning—it would demonstrate that ECoT teaches a domain-general diagnostic methodology rather than domain-specific pattern recognition. A strong negative result (zero transfer) would be equally valuable, showing that the ICWM and ECoT pipeline must be re-run for each new domain, which would limit the approach's scalability. The paper's current benchmark suite makes this experiment straightforward to execute.

Combining ECoT thinking with test-time compute scaling. The paper evaluates InCoder-32B-Thinking in a pass@1 setting (single generation per task), but prior work on test-time compute scaling (best-of-N sampling with verifiers, beam search, sequential revisions) shows that inference compute allocation can substantially improve performance for reasoning models. A natural extension is to evaluate whether applying best-of-N weighted selection or majority voting across multiple thinking model outputs improves industrial benchmark performance beyond the single-sample results in Tables 4 and 5. The thinking traces themselves could serve as a verifier signal—e.g., scoring solutions by the coherence of their diagnostic reasoning or the consistency between the diagnosed root cause and the proposed fix. A more ambitious experiment would apply beam search or tree-of-thought search over the space of possible code revisions, using the ICWM as a learned rollout simulator (predicting whether a proposed fix would pass execution) to guide the search, effectively closing the loop between the world model and test-time compute. This would connect the paper's training-time world model to inference-time search, testing whether the ICWM can serve as a verifier for test-time scaling in addition to a simulator for training data amplification.

Stress-testing the ICWM's generalization to genuinely novel tasks. The ICWM fidelity analysis (Figure 5) evaluates on held-out turns from D_real—samples from the same distribution of task seeds used during ICWM training. This is an interpolation measurement. The ICWM's purpose in the amplification loop is to simulate backends for new task seeds that may differ systematically from the training distribution (e.g., more complex GPU kernels, Verilog modules with different microarchitectural patterns, embedded code for different microcontrollers). An extrapolation stress test would train the ICWM on one set of task seeds and evaluate fidelity on a set of seeds designed to be out-of-distribution—e.g., training on single-module Verilog designs and testing on multi-module hierarchical designs, or training on simple GPU kernels (element-wise ops) and testing on complex ones (fused attention, matrix multiply with custom tiling). The experiment would measure how ICWM fidelity degrades as a function of distribution shift and identify which types of novelty (new algorithms, larger scale, different hardware constraints) cause the sharpest drops. A finding that fidelity drops substantially under even mild distribution shift would suggest that the amplification loop should be restricted to interpolation (generating variations of known task types), while a finding of robust generalization would validate the ICWM as a true learned simulator that captures the underlying causal dynamics of the execution environment. This experiment is made possible by the paper's fidelity evaluation framework but requires constructing controlled distribution-shift benchmarks.

Developing cheaper difficulty estimation and adaptive thinking for inference. The paper demonstrates that thinking depth varies by 209× across task categories (Figure 6) and that the model learns this adaptivity from the training data distribution. But at inference time, the model generates thinking autoregressively without any external signal about when to stop—it relies on its learned prior about how long to think for a given task type. A practical follow-up would develop difficulty estimators that predict, from the task description alone, how much thinking a problem requires, and use this to implement early stopping, speculative decoding, or budget-constrained generation. For example, a lightweight classifier trained on the training corpus's thinking length labels could predict whether a task needs short, medium, or deep reasoning, and the model could be prompted or configured to limit thinking depth accordingly—eliminating the latency cost of 19K-character thinking traces for tasks that empirically don't benefit from them. A strong experiment would sweep thinking budgets (e.g., truncate thinking at 500, 2K, 5K, 10K characters) and measure accuracy degradation per benchmark, producing a thinking depth vs. accuracy Pareto frontier that tells practitioners exactly how much inference compute each benchmark requires. This would address the paper's current silence on inference cost while preserving the adaptive-depth benefit for tasks that genuinely need it.

Practical Applications and Downstream Use Cases

Industrial RTL design teams seeking a unified Verilog generation and debugging assistant. The paper's RealBench results (Table 4) show that InCoder-32B-Thinking achieves 75.6% Syn@1 System—substantially ahead of all other open-weight models (next best: Kimi-K2-Instruct at 50.1%) and competitive with proprietary Claude-Sonnet-4.6 at 69.2%. For a chip design team generating Verilog for AES encryption cores, SD card controllers, or RISC-V CPUs (the RealBench task distribution), this means a 32B open-weight model can produce compilable module hierarchies from specifications at a rate exceeding that of much larger proprietary systems. Combined with the VeriRepair result (83.3% Fix rate on buggy Verilog), the model can serve as both a generator and a debugger in a unified workflow: an engineer writes a specification, the model proposes an RTL implementation, the testbench flags failures, and the model diagnoses and repairs the bugs. The 32B parameter count means this can run on a single high-memory GPU or a small cluster, feasible for in-house deployment without sending proprietary RTL to external APIs—a critical consideration for hardware IP protection.

GPU kernel optimization as an automated step in ML compiler pipelines. The paper's KernelBench L2 result (38.0%, highest among all models including proprietary Claude-Sonnet-4.6 at 28.0%) and SuperCoder speedup (3.93×, among the highest reported) suggest that InCoder-32B-Thinking can be integrated into ML compiler workflows (e.g., Triton-based compilation for PyTorch models, or assembly-level optimization for critical code paths). For a production ML team running inference at scale, the L2 result means that for operator sequences amenable to kernel fusion—the most common optimization scenario in transformer inference (fused attention, fused MLP, fused normalization)—the model can propose fused kernels that are both correct (pass functional tests) and faster than the unfused PyTorch baseline in 38% of cases. Even a 38% automation rate on kernel fusion, combined with the model's ability to reason about shared memory budgets and grid configurations (demonstrated in the Figure 3 CUDA case study), could substantially accelerate the hand-optimization cycle: engineers review and tune model-generated kernels rather than writing them from scratch. The TritonBench G-exe score of 100.0% (the generated kernels are executable without runtime errors) further supports integration into automated pipelines where compilation success is the first gate.

Embedded systems development with automated cross-compilation and emulation verification. The EmbedCGen benchmark (introduced in this paper) tests bare-metal embedded C code generation with a strict verification pipeline: code generation → ARM GCC cross-compilation → Renode system-level emulation. InCoder-32B-Thinking achieves 47.9% Main pass rate, substantially ahead of the instruct predecessor (35.2%) and all other open-weight models except GLM-5 (90.2%, which the paper does not explain and which may reflect a different evaluation configuration). For an embedded systems team developing firmware for STM32 microcontrollers, a 47.9% end-to-end pass rate means that nearly half of generated firmware snippets compile and execute correctly in emulation without human intervention—a significant automation rate for a domain where toolchain setup, memory layout configuration, and peripheral register sequencing are constant sources of bugs. The model's ability to reason about peripheral register sequencing (explicitly listed in the prompt router's domain-specific instructions, Section 2.1) targets one of the most error-prone aspects of embedded development: the correct order of writes to hardware control registers for device initialization. A deployment scenario would integrate the model into an IDE plugin that generates initialization code from datasheet specifications, with the Renode emulator providing verification feedback that feeds back into the model's diagnostic process for failed generations.