Mastering the game of Go without human knowledge

URL: https://www.nature.com/articles/nature24270.pdf

🎯 Pitch

Starting from random play with zero human data, AlphaGo Zero not only surpassed all previous Go-playing AI but did so using a single neural network trained purely through self-play—ultimately defeating the version that beat human champion Lee Sedol 100 games to zero after just 36 hours of training.

1. Executive Summary

This paper introduces AlphaGo Zero, a reinforcement learning system that learns to play Go at superhuman level tabula rasa—starting from random play with no human data, guidance, or domain knowledge beyond the game rules. The system uses a single neural network (combining policy and value networks into one architecture) trained through a novel policy iteration via Monte Carlo tree search procedure, where the network predicts both move probabilities and position values, and these predictions in turn guide the MCTS to produce improved move selections for the next self-play iteration. AlphaGo Zero defeated the previously published champion-defeating AlphaGo Lee 100–0 after just 36 hours of training on a single machine with 4 TPUs, and the fully trained 40-block version reached an Elo rating of 5,185—substantially surpassing AlphaGo Master (the version that defeated top human professionals 60–0) by winning 89–11 in a 100-game match—establishing that pure reinforcement learning can not only match but significantly exceed systems trained on human expertise, and that human data may actually impose a ceiling on performance rather than being necessary for superhuman play.

2. Context and Motivation

The Core Problem: Can an AI Learn Superhuman Expertise From Scratch?

The fundamental question this paper tackles is both philosophical and deeply practical: can a machine learning system achieve superhuman performance in an extraordinarily complex domain using nothing but the rules of the game and its own experience — with zero human input? By 2016, AlphaGo had already demonstrated that AI could defeat human champions at Go, widely considered a "grand challenge" for artificial intelligence due to its enormous search space ( $~10^{170}$ positions) and the need for sophisticated pattern recognition that had resisted computational approaches for decades. But that achievement, as impressive as it was, came with a significant asterisk: the published AlphaGo system required massive amounts of human data to bootstrap its capabilities.

This gap matters for reasons that go far beyond Go. Go is simply the testbed — the real question is about the nature of intelligence and the role of human knowledge in building intelligent systems. If the best AI systems can only reach superhuman levels by first imitating human experts, then they are inherently limited by what humans already know, do, and have recorded. They cannot discover genuinely novel strategies, cannot operate in domains where human expertise is scarce or unreliable, and ultimately represent a form of sophisticated mimicry rather than autonomous learning. The paper's opening paragraph makes this explicit:

"expert data sets are often expensive, unreliable or simply unavailable. Even when reliable data sets are available, they may impose a ceiling on the performance of systems trained in this manner."

This ceiling is not hypothetical. In Go, humans have accumulated knowledge over thousands of years, but that knowledge is incomplete — there exist strategies, sequences, and tactical patterns that no human has discovered. A system trained to imitate humans would never find them because it's optimizing to reproduce what humans already do. The authors frame this as a direct constraint: human data imposes an upper bound on what the system can achieve, and the only way to break through that ceiling is to learn from first principles.

The Broader AI Significance: Beyond One Game

The paper is clearly not just about making a better Go program. The abstract frames the ambition in sweeping terms: "A long-standing goal of artificial intelligence is an algorithm that learns, tabula rasa, superhuman proficiency in challenging domains." Go serves as the proving ground precisely because it represents the kind of domain — vast combinatorial search space, intuitive pattern recognition, long-range strategic planning — that has traditionally been seen as requiring human-like intelligence. If reinforcement learning can master Go from scratch, the argument goes, the same principles should apply to any domain with clear rules and a well-defined objective.

The practical implications extend to areas the paper explicitly mentions in the extended discussion of related work (Methods section on "Self-play reinforcement learning in games"): robotics, industrial control, recommendation systems, and any domain where the rules of interaction are known but optimal behavior must be discovered through experience. The paper cites work showing that algorithms first developed for zero-sum games — temporal-difference learning and Monte Carlo tree search — subsequently became foundational in these applied domains, establishing a pattern where game-playing research drives broader AI progress.

Prior Approaches and Their Limitations

To understand what makes AlphaGo Zero distinctive, we need to look carefully at what came before it. The paper positions itself against a rich history of Go-playing AI, with the immediate predecessor being its own earlier work.

The AlphaGo lineage (AlphaGo Fan and AlphaGo Lee). The published AlphaGo system (AlphaGo Fan, described in Silver et al., 2016) relied on a carefully orchestrated pipeline of human-dependent components:

A supervised learning policy network, trained to predict human expert moves from a dataset of roughly 30 million positions from the KGS Go server. This was the system's initial "intuition" — without it, the neural network had no idea what reasonable play looked like.
A reinforcement learning policy network, initialized from the supervised network and then refined through self-play. Note the critical dependency: RL could only improve on what supervised learning had already taught. Starting from random weights was not viable under this approach.
A value network, trained to evaluate board positions by predicting the winner from games played by the RL policy network against itself. This again depended on the policy network, which depended on the supervised initialization.
Fast rollout policies — lightweight, handcrafted policies based on human-designed features and Go heuristics — that simulated thousands of random game completions to evaluate positions during search. These rollouts embodied explicit human domain knowledge: which moves are worth considering, how to score territory, what patterns matter.
A tree policy within MCTS that also relied on handcrafted features to guide the search beyond what the neural networks provided.
Separate policy and value networks with distinct architectures, trained independently, requiring separate computational and memory resources.

AlphaGo Lee, the version that defeated Lee Sedol, improved on this architecture but maintained all the same fundamental dependencies. The Methods section notes that AlphaGo Lee used larger networks, more training iterations, and had the value network trained from fast self-play games by AlphaGo (an "initial step towards the tabula rasa algorithm"), but it still required human data for its policy network initialization, still used handcrafted rollouts and features, and still separated policy and value into distinct networks.

The broader Go AI landscape. Before AlphaGo, the strongest Go programs (Crazy Stone, Pachi, GnuGo — evaluated in Figure 6b) relied heavily on Monte Carlo tree search with handcrafted heuristics. The Methods section describes this tradition: MCTS programs "used substantial domain expertise: a fast rollout policy, based on handcrafted features, that evaluates positions by running simulations until the end of the game; and a tree policy, also based on handcrafted features, that selects moves within the search tree." These programs reached strong amateur levels but hit a ceiling that no amount of additional compute could break through — the handcrafted knowledge was simply too brittle and incomplete.

Self-play reinforcement learning in games (the broader context). The paper's Methods section provides a detailed genealogy of self-play RL approaches that makes clear what was novel about AlphaGo Zero. Prior systems using self-play applied it in much simpler ways:

NeuroGo and RLGO applied temporal-difference learning to Go using self-play, but represented value functions with sophisticated handcrafted features (NeuroGo used a neural network with explicit connectivity, territory, and eye-detection modules; RLGO exhaustively enumerated all $3 \times 3$ stone patterns). Both reached only weak amateur levels.
Classification-based reinforcement learning improved policies by running Monte Carlo rollouts and training a classifier to distinguish good from bad actions. A more advanced variant, CBMPI, added value function regression and achieved state-of-the-art results in Tetris. The paper notes this is a "precursor to the policy component of AlphaGo Zero's training algorithm when $\tau \to 0$ " but was "limited to simple rollouts and linear function approximation using handcrafted features."
Self-play in other games (chess, checkers, backgammon, Othello, Scrabble, poker) had achieved strong results, but every example relied on some form of human knowledge: handcrafted input features, hand-selected piece values, supervised initialization from human data, or pre-existing programs as training opponents. The paper's Methods section systematically documents these dependencies for each cited system, establishing that no prior work had achieved superhuman performance with zero domain knowledge.

What Guo et al. (2014) did and didn't do. The paper is careful to distinguish its approach from Guo et al. (2014), which also projected MCTS outputs into a neural network for Atari games. The critical difference: Guo et al.'s MCTS was fixed — there was no policy iteration. The trained network was never used to improve the search, meaning the system couldn't bootstrap its own improvement. Without this feedback loop, each component operates in isolation rather than the virtuous cycle that powers AlphaGo Zero.

Where Prior Approaches Fall Short (The Specific Gaps)

The paper identifies several concrete limitations that motivate its new design:

1. Human data as a dependency creates multiple failure modes. It's not just that human data is expensive — though it is, requiring curated datasets of expert games. The deeper problem is that human data is fundamentally limiting. The paper demonstrates this empirically in Figure 3: a network trained by supervised learning on human data achieves better move prediction accuracy on professional moves (Figure 3b) and lower value prediction error on professional games (Figure 3c) than the self-play trained network — yet the self-play network dramatically outperforms it in actual playing strength. This is the paper's most direct evidence that faithfully imitating humans is not the same as playing well. The human-trained player is optimizing for the wrong objective.

2. The rollouts bottleneck. Fast rollout policies in prior AlphaGo versions were inherently limited: they had to be computationally cheap (to run thousands of times per search) but also accurate enough to provide useful evaluations. This forced a tradeoff that no amount of engineering could fully resolve. Rollouts evaluate positions by playing random-ish games to completion, but the quality of those evaluations depends entirely on the quality of the rollout policy. A handcrafted rollout policy inevitably misses subtle tactical sequences and therefore provides noisy, biased evaluations that can mislead the search.

3. Separate networks create engineering complexity and learning inefficiency. AlphaGo Fan and Lee used separate policy and value networks with different architectures, trained on different data, optimized separately. Beyond the engineering burden (deploying and maintaining two networks in the search), this separation prevents the networks from sharing learned representations. The paper's architecture comparison (Figure 4) is crucial here: combining policy and value into a single network reduced move prediction accuracy slightly — the network is slightly worse at the specific task of predicting human moves — but improved playing strength by approximately 600 Elo. The authors attribute this to "the dual objective regularizes the network to a common representation that supports multiple use cases" — essentially, learning to evaluate positions helps the network learn better move probabilities and vice versa, even if each individual task shows slightly lower accuracy in isolation.

4. The "catastrophic forgetting" and "oscillation" problem that didn't materialize. Prior literature on multi-agent reinforcement learning and self-play had raised concerns about training instability — the possibility that self-play systems would oscillate between strategies, forget previously learned skills, or fail to converge. The paper cites specific works (Laurent et al., 2011; Foerster et al., 2017; Heinrich and Silver, 2016) that documented these challenges. This was a genuine concern: if the neural network forgets how to play against previous versions of itself, the self-play data becomes unreliable and the feedback loop collapses. Part of the paper's motivation was to demonstrate that with the right architecture and training procedure, these fears were unfounded — the learning curve in Figure 3a shows "smooth progression" without oscillation, a finding that was surprising enough to merit explicit mention.

5. No prior system had combined deep neural networks, MCTS, and reinforcement learning into a unified policy iteration loop. This is the key integrative gap. Prior work had used neural networks with MCTS (AlphaGo Fan/Lee, Guo et al.), or reinforcement learning with MCTS (older Go programs), or self-play with neural networks (NeuroGo, TD-Gammon), but no system had closed the loop: using the neural network to guide MCTS, using MCTS to generate improved training targets for the neural network, and iterating this process from tabula rasa. The paper specifically frames its contribution in terms of approximate policy iteration (Methods, "Reinforcement learning"), casting MCTS as both the policy improvement operator (generating better moves than the raw network) and the policy evaluation operator (the game outcomes provide value estimates). This unification is the conceptual innovation that makes the rest work.

How This Paper Positions Itself Relative to Existing Work

The paper is explicit about its novelty claims. The primary contribution is not any single component — residual networks, MCTS, and policy iteration were all established techniques — but rather the demonstration that their integration, properly executed, eliminates the need for human data entirely while achieving superior performance.

The paper's positioning can be understood along several axes:

Against the prior AlphaGo versions: AlphaGo Zero is cleaner, simpler, stronger, and human-free. The paper enumerates the differences precisely: no supervised learning, no rollouts, single network instead of two, simpler MCTS, raw board input only. Each of these simplifications is framed as removing a dependency on human knowledge.

Against the broader RL literature: The paper argues that pure reinforcement learning had not been shown to work in "the most challenging domains in terms of human intellect" — domains requiring "precise and sophisticated lookahead in vast search spaces." Atari games and 3D environments, while impressive, don't require the same kind of deep search that Go demands. This is why MCTS integration is essential and why the paper frames its method explicitly as reinforcement learning with search in the loop.

Against the "human data is necessary" assumption: This is perhaps the paper's most provocative positioning. The supervised learning baseline in Figure 3 shows that human data gives a head start (better initial performance, better move prediction) but that self-play rapidly overtakes it and keeps improving while the human-trained system plateaus. The paper interprets this as evidence that human data imposes a ceiling. This is not just an empirical observation — it's a philosophical claim about the nature of learning: that imitating experts may be fundamentally less effective than discovering principles from experience, even in domains where human expertise is extremely sophisticated.

Against concerns about self-play stability: As noted above, the paper explicitly addresses the fear that self-play RL would be unstable or forgetful. The smooth learning curves in Figures 3a and 6a are presented as evidence that the concerns, while legitimate for prior methods, do not apply to this approach. The paper is making the case that its specific design choices — the policy iteration framing, the use of MCTS as an improvement operator, the temperature and noise mechanisms in self-play — collectively solve the stability problem.

Against the "domain knowledge is inevitable" position: The paper takes pains to enumerate exactly what domain knowledge AlphaGo Zero does use (Methods, "Domain knowledge"), and it's minimal: perfect knowledge of the rules, Tromp-Taylor scoring, grid-structured input representation, and rotation/reflection invariance. Everything else — joseki, fuseki, tesuji, life-and-death, ko fights, influence, territory — is discovered from scratch. This enumeration is important because it draws a clear boundary around the claim: the paper is not claiming to have solved general game playing from pixels (the input is still a structured 19×19 board), but it is claiming to have eliminated Go-specific knowledge beyond the rules.

The Unstated but Obvious Motivation: Proving a Point About AI

Reading between the lines, there's a clear rhetorical motivation behind this paper that goes beyond the technical contributions. The 100–0 victory against AlphaGo Lee after 36 hours of training, on a single machine with 4 TPUs versus AlphaGo Lee's distributed system with 48 TPUs, is not just a benchmark result — it's a deliberately stark demonstration that the human-data-free approach is not merely viable but dramatically superior. The paper could have reported a narrow victory or competitive performance; the 100–0 scoreline (Extended Data Figure 1 shows the games) is designed to leave no doubt.

Similarly, the revelation that AlphaGo Zero discovered not just known human joseki patterns but new variations that it preferred over traditional ones (Figure 5b and Extended Data Figure 3) serves a rhetorical purpose: it proves that the system went beyond human knowledge, not just to it. The fact that "shicho (ladder capture sequences) — one of the first elements of Go knowledge learned by humans — were only understood by AlphaGo Zero much later in training" is highlighted to emphasize that the system's learning trajectory was genuinely different from the human one, not just a faster replay of human discovery.

This is a paper that wants to change how the AI community thinks about the role of human data in building intelligent systems. The technical achievement enables the conceptual argument, but the conceptual argument is the real payload.

3. Technical Approach

3.1 Reader Orientation

AlphaGo Zero is best understood as a self-reinforcing loop between a neural network and a tree search engine: the neural network provides the "intuition" that guides where the search should look, and the search produces higher-quality data that trains the neural network to have even better intuition. The problem it solves is learning superhuman Go strategy without any human examples or prior knowledge beyond the rules, and the shape of the solution is an approximate policy iteration algorithm where Monte Carlo tree search acts as both the policy improvement operator (generating moves better than the raw network could produce alone) and the policy evaluation operator (the final game outcomes provide value estimates), with the neural network being trained to compress the improved search outputs back into a single forward pass — so that at the next iteration, the search starts from a stronger prior and can reach even further.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five interconnected components that form a closed learning loop:

A deep residual neural network $f_\theta$ — takes a raw board position $s$ (a $19 \times 19 \times 17$ stack of binary feature planes encoding the stones' history and the current player) and produces two outputs: a probability distribution $\mathbf{p}$ over all legal moves (including pass) and a scalar value $v \in [-1, 1]$ estimating the current player's chance of winning. This single network combines what were previously separate policy and value networks.
A Monte Carlo tree search (MCTS) algorithm — takes the neural network's move probabilities and value estimates and performs many simulated traversals of the game tree, using the network's outputs to guide which branches to explore and how to evaluate newly encountered positions. After a fixed number of simulations (typically 1,600), the MCTS returns an improved move probability vector $\boldsymbol{\pi}$ that is stronger than the raw network output.
A self-play game generator — plays full games of Go where each move for both sides is selected by running the MCTS guided by the current best neural network. The terminal outcome $z \in \{-1, +1\}$ (win/loss from the current player's perspective) provides a ground-truth value label for every position in the game.
A training pipeline — samples positions uniformly from the most recent 500,000 self-play games and updates the neural network parameters $\theta$ by gradient descent to make the network's move probabilities $\mathbf{p}$ match the MCTS search probabilities $\boldsymbol{\pi}$ (via cross-entropy loss) and the network's value prediction $v$ match the actual game outcome $z$ (via mean-squared error loss), plus L2 regularization.
An evaluator — periodically pits the newly trained network against the current best network in 400-game matches (using 1,600 MCTS simulations per move). If the new network wins more than 55% of the games, it replaces the current best and becomes the new data generator for self-play. This ensures that the quality of self-play data monotonically improves.

The flow is: self-play generates data $(s, \boldsymbol{\pi}, z)$ from the current best player → the neural network trains on this data to match $\mathbf{p}$ to $\boldsymbol{\pi}$ and $v$ to $z$ → the evaluator checks if the new network is stronger → if so, it becomes the new data generator → the neural network now provides a better prior for MCTS → MCTS produces even better moves → the cycle repeats, with the neural network and the search engine mutually bootstrapping each other toward superhuman play.

3.3 Roadmap for the Deep Dive

First, the neural network architecture and its dual outputs, since it is the central learnable component that everything else depends on.
Second, the Monte Carlo tree search algorithm in detail — how it uses the neural network, the mechanics of selection/expansion/backup, and how it produces the improved move probabilities $\boldsymbol{\pi}$ . This is the mechanism that converts "raw network intuition" into "strong actual play."
Third, the loss function and training procedure, since these define the learning signal that connects search outputs back to network parameters.
Fourth, the self-play mechanism and the evaluator's role, which together determine the data distribution the network learns from and ensure monotonic improvement.
Fifth, the design choices that eliminate dependencies on human knowledge — the input representation, the absence of rollouts, the unified architecture — and why each choice matters.
Sixth, the overall policy iteration interpretation, which provides the conceptual framework for understanding why the loop works and why it converges to superhuman play rather than oscillating or collapsing.

3.4 Detailed, Sentence-Based Technical Breakdown

This is an algorithm paper whose core idea is that Monte Carlo tree search can serve as both a policy improvement operator and a policy evaluation operator within an approximate policy iteration loop, and that when you close this loop — using the neural network to prime the search, then using the search's improved outputs to retrain the neural network, then using the stronger neural network to prime the next round of search — the system bootstraps from random play to superhuman performance without any human data or domain-specific engineering.

The Neural Network Architecture: $f_\theta(s) \rightarrow (\mathbf{p}, v)$

The network $f_\theta$ is a deep convolutional neural network with a residual tower architecture based on the ResNet design from He et al. (2016). It takes a raw board representation $s$ and produces two fundamentally different outputs from a single shared representation — a design choice that the paper shows adds approximately 600 Elo of playing strength compared to separate networks (Figure 4).

Input representation. The input $s$ is a $19 \times 19 \times 17$ image stack consisting of 17 binary feature planes. Specifically:

8 feature planes $X_t$ encode the current player's stones, with $X_t^i = 1$ if intersection $i$ contains a stone of the player's color at time-step $t$ , and 0 otherwise (including if the intersection is empty, contains an opponent stone, or if $t < 0$ meaning that time-step hasn't occurred yet).
8 feature planes $Y_t$ encode the opponent's stones with the same convention.
1 feature plane $C$ encodes the color to play: a constant value of 1 if Black is to play or 0 if White is to play.

The full input stack at time-step $t$ concatenates the current position and the seven most recent board states: $s_t = [X_t, Y_t, X_{t-1}, Y_{t-1}, ..., X_{t-7}, Y_{t-7}, C]$ . The history planes $X_{t-1}, Y_{t-1}$ through $X_{t-7}, Y_{t-7}$ are necessary because Go is not fully observable from the current stones alone — the ko rule forbids repeating a previous board position, so the system must know what positions have occurred recently. The color feature $C$ is necessary because the komi (the bonus points given to White as compensation for moving second) is not observable from the board state alone.

Why this input representation matters: this is the only domain knowledge AlphaGo Zero receives beyond the game rules. The paper explicitly enumerates its domain knowledge in the Methods section: knowledge of the rules, Tromp-Taylor scoring, grid-structured input, and rotation/reflection invariance. Everything else — joseki patterns, life-and-death concepts, influence, territory, tactical sequences — must be discovered from experience. The raw board representation is the system's only window into Go; it has no handcrafted features for connectivity, eye shape, territory potential, or any other human-identified Go concept.

The residual tower. The input $s_t$ first passes through a single convolutional block consisting of:

A convolution with 256 filters of kernel size $3 \times 3$ and stride 1
Batch normalization
A rectifier (ReLU) nonlinearity

The output of this initial block feeds into a stack of either 19 or 20 residual blocks (for the 3-day, 20-block version) or 39 or 40 residual blocks (for the 40-day, 40-block version). Each residual block applies the following sequence to its input:

A convolution with 256 filters of kernel size $3 \times 3$ and stride 1
Batch normalization
A rectifier (ReLU) nonlinearity
A second convolution with 256 filters of kernel size $3 \times 3$ and stride 1
Batch normalization
A skip connection that adds the original input of the block to the output of step 5
A rectifier (ReLU) nonlinearity

Why residual blocks: the skip connection (step 6) allows gradients to flow directly through the network during backpropagation without degradation, enabling much deeper architectures. Without residual connections, very deep networks suffer from vanishing gradients — the training signal becomes progressively weaker as it propagates backward through many layers, making early layers learn extremely slowly or not at all. The identity mapping provided by the skip connection means each block only needs to learn a residual correction to its input rather than learning the full transformation from scratch, which is empirically much easier to optimize. This architectural choice is what allows the network to scale from 12 layers (in AlphaGo Lee) to 39 or 79 parameterized layers (in AlphaGo Zero's 20-block and 40-block versions) while maintaining stable and efficient training.

The dual heads. After the residual tower, the shared representation splits into two separate output heads:

Policy head — produces move probabilities $\mathbf{p}$ :

A convolution with 2 filters of kernel size $1 \times 1$ and stride 1
Batch normalization
A rectifier (ReLU) nonlinearity
A fully connected linear layer that outputs a vector of size $19^2 + 1 = 362$ , corresponding to logit probabilities for all 361 board intersections plus the pass move

Value head — produces a single scalar $v$ :

A convolution with 1 filter of kernel size $1 \times 1$ and stride 1
Batch normalization
A rectifier (ReLU) nonlinearity
A fully connected linear layer to a hidden layer of size 256
A rectifier (ReLU) nonlinearity
A fully connected linear layer to a scalar
A $\tanh$ nonlinearity outputting a scalar in the range $[-1, 1]$

The total network depth, counting only parameterized layers: the 20-block network has 39 layers in the residual tower (1 initial convolution + 19 × 2 convolutions per block) + 2 layers for the policy head + 3 layers for the value head = 44 parameterized layers. The 40-block network has 79 layers in the residual tower (1 + 39 × 2) + 2 + 3 = 84 parameterized layers.

Why $\tanh$ for the value head: the value $v$ is interpreted as the estimated probability of the current player winning, but since the final game outcomes $z$ are in $\{-1, +1\}$ , the $\tanh$ activation provides a natural mapping to the same range. The network output is scaled such that $v = +1$ means "completely confident win for current player" and $v = -1$ means "completely confident loss." Values in between represent estimated winning probabilities: $v = 0.5$ means approximately 75% win probability, $v = 0$ means 50%, etc.

Why a single network rather than separate policy and value networks (Figure 4). AlphaGo Lee and AlphaGo Fan used separate networks with independent architectures for policy and value — the policy network had one structure and was trained on one objective, while the value network had a different structure and was trained on a different objective. AlphaGo Zero combines both into a single network with shared convolutional layers feeding two heads. The empirical comparison in Figure 4 shows this dual architecture (labeled "dual–res") achieves approximately 600 Elo higher playing strength than the separate architecture ("sep–res"), even though it achieves slightly lower move prediction accuracy on professional moves. The authors attribute this to representation sharing: learning to evaluate positions and learning to select moves are complementary tasks that benefit from a shared intermediate representation. The value objective regularizes the policy features and vice versa, even if either individual task would achieve slightly better accuracy with dedicated parameters. Additionally, the single network is computationally more efficient during search — only one forward pass is needed to obtain both move probabilities and a position evaluation, rather than two separate passes through different networks.

Monte Carlo Tree Search (MCTS) in AlphaGo Zero

The MCTS is the algorithmic engine that converts the neural network's raw predictions into strong actual play. At each position $s$ , the MCTS performs many simulated traversals of the game tree, using the neural network to guide which branches to explore and to evaluate newly discovered positions. After a fixed budget of simulations, the search returns an improved move probability distribution $\boldsymbol{\pi}$ that is substantially stronger than the raw network output $\mathbf{p}$ .

Why MCTS is necessary. The neural network alone — taking the position and directly outputting a move — plays at approximately 3,055 Elo (Figure 6b, "Raw network"). With 1,600 MCTS simulations per move, the same network reaches over 4,500 Elo during training and over 5,100 Elo when fully trained. This enormous gap (~2,000 Elo) demonstrates that the search provides capabilities that the feedforward network fundamentally cannot match: the ability to explore specific tactical sequences, evaluate counterfactual branches, and "think ahead" about the consequences of moves. The neural network provides good priors about which moves are worth considering and rough evaluations of intermediate positions, but the search is what converts these priors into precise, tactical, lookahead-based decisions.

Data structures in the search tree. Each node $s$ in the search tree (representing a board position) stores a set of edges $(s, a)$ for all legal actions $a \in \mathcal{A}(s)$ . Each edge stores four statistics:

$N(s, a)$ — the visit count: how many times this edge has been traversed during the current search
$W(s, a)$ — the total action value: the sum of all evaluations $V(s')$ that have been backed up through this edge
$Q(s, a)$ — the mean action value: computed as $Q(s, a) = W(s, a) / N(s, a)$ , representing the average outcome of simulations that passed through this edge
$P(s, a)$ — the prior probability: the neural network's estimated probability of selecting this move from position $s$ , which comes from the policy head output $p_a$

The search proceeds by repeatedly executing three phases — select, expand and evaluate, backup — and then selecting a move to play.

Phase 1: Select. Each simulation begins at the root node $s_0$ (the current board position) and iteratively selects moves to traverse deeper into the tree until reaching a leaf node $s_L$ at time-step $L$ . A leaf node is defined as any position that has been reached in the tree but does not yet have its children expanded and evaluated.

At each intermediate step $t < L$ , the selection is determined by maximizing an upper confidence bound over all legal actions:

$a_t = \arg\max_a \left(Q(s_t, a) + U(s_t, a)\right)$

where $Q(s_t, a)$ is the current mean action value (exploitation term — favoring moves that have led to good outcomes in previous simulations) and $U(s_t, a)$ is an exploration bonus (favoring moves that have been tried less often relative to their prior probability).

The exploration bonus $U(s_t, a)$ follows the PUCT algorithm (Predictor + UCT, from Rosin, 2011):

$U(s, a) = c_{\text{puct}} \cdot P(s, a) \cdot \frac{\sqrt{\sum_b N(s, b)}}{1 + N(s, a)}$

where $c_{\text{puct}}$ is a constant controlling the overall level of exploration (determined by Gaussian process optimization), $P(s, a)$ is the prior probability from the neural network, $\sum_b N(s, b)$ is the total number of visits to the parent node, and $N(s, a)$ is the number of visits to this specific edge.

What this selection rule achieves, operationally, and why it has this form. The PUCT formula balances two competing goals. The $Q(s, a)$ term alone would greedily select the move with the highest average outcome so far — pure exploitation, which would cause the search to get stuck on the first promising-looking move and never discover better alternatives. The $U(s, a)$ term alone would favor moves that are under-explored — pure exploration, which would waste simulations on obviously bad moves.

The specific form of $U(s, a)$ creates several desirable properties:

Prior-guided exploration: the $P(s, a)$ factor ensures that exploration is proportional to the neural network's judgment of move quality. Moves the network considers highly probable receive a large exploration bonus; moves the network considers nearly impossible receive almost no bonus. This is the key mechanism by which the neural network's "intuition" shapes the search — without the $P(s, a)$ term, the search would waste enormous computation exploring obviously terrible moves.
Diminishing returns on repeated visits: the $1 + N(s, a)$ denominator ensures that as a move is visited more times, its exploration bonus decreases. This forces the search to eventually broaden its attention to other promising moves rather than continuing to explore the same branch indefinitely.
Proportional to total parent visits: the $\sqrt{\sum_b N(s, b)}$ numerator increases the overall exploration budget as more total simulations are spent at the parent node. Early in the search at a node (when total visits are low), the exploration bonuses are small and the search is more exploitative; later, as more simulations accumulate, the exploration bonuses grow to ensure that even moves with modest priors eventually receive attention.
The $c_{\text{puct}}$ constant: this provides a global knob controlling the exploration-exploitation tradeoff. Higher $c_{\text{puct}}$ means the search is more willing to explore moves with low visit counts; lower $c_{\text{puct}}$ means it sticks more closely to moves that have already proven effective. The value was tuned by Bayesian optimization to maximize self-play performance.

Phase 2: Expand and evaluate. When the selection phase reaches a leaf node $s_L$ , that position needs to be evaluated so the search can learn something about it. In AlphaGo Zero, leaf evaluation is done entirely by the neural network — there are no rollouts (unlike AlphaGo Fan and Lee, which played thousands of random-ish games from the leaf to estimate its value).

The leaf node $s_L$ is first transformed by a dihedral reflection or rotation selected uniformly at random from the 8 symmetries of the Go board (4 rotations × 2 reflections). The transformed position $d_i(s_L)$ is evaluated by the neural network in a mini-batch (size 8) with other leaf nodes being evaluated simultaneously:

$(d_i(\mathbf{p}), v) = f_\theta(d_i(s_L))$

The network produces prior probabilities $\mathbf{p}$ (a vector over all legal moves at $s_L$ ) and a scalar value $v \in [-1, 1]$ estimating the winning probability from $s_L$ . The leaf node is then expanded: for each legal action $a$ from $s_L$ , a new edge $(s_L, a)$ is added to the tree and initialized with:

$N(s_L, a) = 0$ (no visits yet)
$W(s_L, a) = 0$ (no accumulated value)
$Q(s_L, a) = 0$ (mean value is undefined before any visits, but initialized to 0)
$P(s_L, a) = p_a$ (the prior probability from the network's policy head)

Why random dihedral transformations: the neural network's evaluation is not perfectly symmetric with respect to board rotations and reflections, even though the game of Go itself is (aside from komi, which is handled by the color feature $C$ ). By randomly transforming each position before network evaluation, the search effectively averages over all symmetries, making the prior probabilities and value estimates more robust. This is a form of test-time data augmentation that exploits the known symmetry of the domain.

Why no rollouts: AlphaGo Fan and Lee used fast, lightweight "rollout policies" to simulate games from leaf nodes to completion, providing a second evaluation signal alongside the value network. AlphaGo Zero eliminates these entirely. The neural network's evaluation is used alone. This is possible because the network has been trained to provide accurate value estimates, and eliminating rollouts simplifies the search algorithm significantly — no need to design, implement, and tune a separate rollout policy, no need to balance rollout evaluations against network evaluations, and no distribution mismatch between rollout play and real play. The tradeoff is that the neural network must be very accurate at position evaluation, which the training loop ensures through the self-play value targets.

Phase 3: Backup. After the leaf node is evaluated, the value $v$ is propagated backward through all edges that were traversed during the selection phase of this simulation. For each step $t \leq L$ along the simulation path, the edge $(s_t, a_t)$ that was selected at that step is updated:

$N(s_t, a_t) = N(s_t, a_t) + 1$

$W(s_t, a_t) = W(s_t, a_t) + v$

$Q(s_t, a_t) = \frac{W(s_t, a_t)}{N(s_t, a_t)}$

The visit count is incremented, the total value is incremented by the leaf evaluation, and the mean action value is recomputed as the running average of all evaluations that have passed through this edge. Note that $v$ is the value from the perspective of the player at the leaf node $s_L$ , but it is backed up unchanged — the backup does not negate values for alternating players because the value $v$ is always expressed from the perspective of the current player at the time of evaluation. In a two-player zero-sum game, the value from Black's perspective at move 10 is the negative of the value from White's perspective at move 11, but the MCTS representation handles this implicitly: the stored $Q(s, a)$ values are always from the perspective of the player whose turn it is at position $s$ . When we select moves at a node, we're using $Q$ values that are all from that node's player's perspective, so the $\arg\max$ operation is coherent.

Parallel simulation with virtual loss. Multiple simulations run in parallel on separate search threads. To prevent all threads from exploring the same promising branch (which would waste the parallelism), the search uses virtual loss: when a thread selects an edge during the selection phase, it temporarily assumes that edge will lead to a loss, making it less attractive to other threads. Specifically, before the leaf is actually evaluated and the real backup occurs, the thread increments $N(s, a)$ and decrements $W(s, a)$ by a small amount ( $W(s, a)$ is reduced as if a loss were backed up). This encourages other threads to explore different branches, increasing the diversity of the overall search. When the real backup occurs, the virtual loss is replaced by the actual evaluation. The net effect is that $N$ simulations distributed across parallel threads explore a wider portion of the tree than they would if all threads converged on the same path.

Phase 4: Play — selecting the move to actually make. After all simulations are complete (for a standard evaluation, this is typically 1,600 simulations, taking approximately 0.4 seconds per move), the search produces a final move probability distribution $\boldsymbol{\pi}$ at the root node $s_0$ :

$\pi_a = \frac{N(s_0, a)^{1/\tau}}{\sum_b N(s_0, b)^{1/\tau}}$

where $\tau$ is a temperature parameter controlling the randomness of move selection.

What this formula computes, operationally: the search counts how many times each move from the root was visited during the MCTS simulations, raises these visit counts to the power $1/\tau$ , and normalizes them to form a probability distribution. Moves that received more search attention — because they led to better outcomes ( $Q$ was high) and/or had high prior probabilities ( $P$ was high) — receive higher probabilities. Moves that the search explored less receive lower probabilities.

Why the temperature parameter $\tau$ matters:

When $\tau \to 0$ (infinitesimal temperature, used for most of the game): the exponent $1/\tau \to \infty$ , so the distribution becomes deterministic — the move with the maximum visit count receives probability 1 and all others receive 0. This gives the strongest possible play, assuming the search has correctly identified the best move.
When $\tau = 1$ (used for the first 30 moves of each self-play game): the exponent is 1, so the distribution is simply proportional to the raw visit counts. This encourages exploration during the opening, ensuring that diverse positions are encountered in self-play data. If the opening were always played deterministically, the training data would be limited to a narrow set of positions and the network would never learn to handle alternative openings.
The paper uses $\tau = 1$ for the first 30 moves and $\tau \to 0$ for the remainder of the game. This is explicitly designed to balance exploration (early) with exploitation (later).

Tree reuse across moves. After selecting a move $a$ at the root, the search tree is not discarded entirely. The child node $s_1$ corresponding to the played action becomes the new root; the entire subtree below $s_1$ is retained along with all its statistics (visit counts, action values, priors). The rest of the tree is discarded. This means that at the next move, the search doesn't start from scratch — it already has partial search results for the subtree below the move just played, which provides a warm start for the next round of MCTS.

Comparison to MCTS in prior AlphaGo versions. The Methods section enumerates the specific differences from AlphaGo Fan/Lee's MCTS:

No rollouts: AlphaGo Zero uses only the neural network for leaf evaluation. AlphaGo Fan/Lee supplemented the value network with thousands of fast random rollouts using a handcrafted policy based on human Go heuristics.
Single network instead of separate policy and value networks: only one forward pass is needed at each leaf.
Always expand leaf nodes: AlphaGo Fan/Lee used dynamic expansion where a node might be visited multiple times before being expanded. AlphaGo Zero always expands a leaf node as soon as it's reached.
Synchronous evaluation: each search thread waits for the neural network evaluation to complete rather than performing evaluation and backup asynchronously. This is simpler and avoids race conditions.
No separate tree policy: AlphaGo Fan/Lee used handcrafted features in the tree policy that selected moves within the search tree. AlphaGo Zero's selection depends only on $Q$ , $U$ , and the neural network's prior $P$ .
Transposition table (only in the 40-block, 40-day version): if different move sequences lead to the same board position, they are merged into a single node with shared statistics. This is especially relevant for Go due to move-order transpositions being common.

The Loss Function and Training Procedure

The neural network is trained to predict two targets from each self-play position: the MCTS search probabilities $\boldsymbol{\pi}$ (for the policy head) and the game outcome $z$ (for the value head). The training objective combines these into a single loss:

$l = (z - v)^2 - \boldsymbol{\pi}^\top \log \mathbf{p} + c \|\theta\|^2$

where $z \in \{-1, +1\}$ is the game outcome from the perspective of the current player (the opponent of the player who eventually won the game — since $z$ is defined as $\pm r_T$ where $r_T$ is the final reward, the sign flips appropriately at each step via the perspective mechanism), $v \in [-1, 1]$ is the network's predicted value, $\boldsymbol{\pi}$ is the vector of MCTS search probabilities (a probability distribution over moves), $\mathbf{p}$ is the network's output move probability vector, and $c \|\theta\|^2$ is L2 weight regularization with $c = 10^{-4}$ .

Breaking down the loss term by term:

Term 1 — Mean-squared error on value: $(z - v)^2$ . This is a standard regression loss that penalizes the network when its predicted winning probability $v$ deviates from the actual game outcome $z$ . Since $z$ is $+1$ for a win and $-1$ for a loss, and $v$ is in the same range due to the $\tanh$ activation, the squared error ranges from 0 (perfect prediction) to 4 (worst possible: predicting $+1$ when the outcome is $-1$ ).

What this computes in operational terms: for each training position, the network outputs a scalar $v$ estimating how likely the current player is to win. After the game concludes, we know the actual outcome $z$ . The squared difference measures how wrong the estimate was. Minimizing this term trains the network to become an accurate position evaluator — to recognize, from the board position alone, which side is winning and by approximately how much.

Why MSE rather than cross-entropy for the value: the value is a continuous prediction (estimated win probability mapped to $[-1, 1]$ ) rather than a classification, so mean-squared error is the natural choice. The $\tanh$ output activation bounds the prediction to the same range as the target, preventing extreme predictions from dominating the loss.

Term 2 — Cross-entropy on policy: $-\boldsymbol{\pi}^\top \log \mathbf{p}$ . In expanded form: $-\sum_a \pi_a \log p_a$ , where the sum is over all legal moves. $\pi_a$ is the MCTS search probability for move $a$ (a scalar between 0 and 1, with the vector summing to 1), and $p_a$ is the network's predicted probability for move $a$ . The cross-entropy measures how well the network's move distribution matches the search's improved distribution.

What this computes operationally: the MCTS produces a much stronger move distribution $\boldsymbol{\pi}$ than the raw network output $\mathbf{p}$ . The cross-entropy term penalizes the network when $\mathbf{p}$ diverges from $\boldsymbol{\pi}$ . Minimizing this term trains the network to internalize the search's improved judgment — to directly output probabilities that resemble what the MCTS would produce after performing 1,600 simulations, but in a single forward pass. This is the "policy improvement projection" step: compressing the computationally expensive search output into the fast feedforward network.

Why cross-entropy: cross-entropy is the standard loss for training a classifier to match a target probability distribution. It has the property that the gradient with respect to $p_a$ is $-\pi_a / p_a$ , which means the network receives a very strong gradient when the search probability $\pi_a$ is high but the network's probability $p_a$ is low — it urgently needs to increase probability on moves the search recommends. Conversely, when $\pi_a$ is near 0, the gradient is also near 0, meaning the network is not penalized for assigning small probabilities to moves the search ignores. This asymmetric gradient signal is more effective for policy distillation than MSE, which would give equal weight to matching both high and low probabilities.

Term 3 — L2 regularization: $c \|\theta\|^2 = 10^{-4} \sum_i \theta_i^2$ . This penalizes large weight values, encouraging the network to find simpler solutions. L2 regularization is standard practice to prevent overfitting, but it takes on additional importance in the self-play setting: without regularization, the network could potentially memorize specific positions from recent games rather than learning generalizable patterns. The coefficient $c = 10^{-4}$ is relatively small, providing a gentle pressure toward simplicity without dominating the task-specific losses.

Why the two components are weighted equally: the paper states that "the cross-entropy and MSE losses are weighted equally (this is reasonable because rewards are unit scaled, $r \in \{-1, +1\}$ )." The implicit argument is that since both targets ( $\boldsymbol{\pi}$ and $z$ ) are bounded — $\pi_a$ is in $[0, 1]$ and the cross-entropy for a uniform distribution over 362 moves is approximately $\log(362) \approx 5.9$ , while $(z - v)^2$ is in $[0, 4]$ — the two loss terms naturally operate on similar scales. No explicit weighting coefficient is needed because the problem design (unit rewards, softmax over a fixed number of moves) already provides reasonable scaling.

Training data and optimization. The training data consists of $(s, \boldsymbol{\pi}, z)$ tuples sampled uniformly at random from all positions of the most recent 500,000 games of self-play. This windowed sampling is crucial: it ensures the training data reflects the current strength of the system and does not include outdated positions from when the system was weaker, which would provide incorrect value targets (positions that were evaluated as losing might actually be winning if played better) and suboptimal policy targets (the MCTS move recommendations from a weak network are less reliable).

The optimization uses stochastic gradient descent with momentum (momentum parameter = 0.9) on the Google Cloud using TensorFlow, with 64 GPU workers and 19 CPU parameter servers. Each worker processes mini-batches of size 32, giving an effective total mini-batch size of $64 \times 32 = 2,048$ positions per update. The learning rate is annealed according to a schedule: starting at $10^{-3}$ , then dropping to $10^{-4}$ after 400,000 steps, and to $10^{-5}$ after 600,000 steps (for the 20-block network; the 40-block network uses a proportionally longer schedule, detailed in Extended Data Table 3). The optimization produces a new checkpoint every 1,000 training steps.

For the 20-block, 3-day run: parameters were updated from 700,000 mini-batches of 2,048 positions each, totaling approximately 1.4 billion training positions. For the 40-block, 40-day run: 3.1 million mini-batches of 2,048 positions, totaling approximately 6.3 billion training positions.

Self-Play and Evaluation: The Policy Iteration Loop

The training loop is orchestrated by three asynchronous components: self-play generation, optimization, and evaluation.

Self-play generation. The current best player $\alpha_{\theta^*}$ (the one that wins the evaluator's comparison, described below) plays 25,000 games of self-play. In each game, at each position, MCTS is run with 1,600 simulations per move (approximately 0.4 seconds per search). The temperature $\tau$ is set to 1 for the first 30 moves and $\tau \to 0$ thereafter, as described above.

Dirichlet noise for additional exploration. At the root node $s_0$ of each search (i.e., the actual board position where a move will be played), Dirichlet noise is added to the neural network's prior probabilities before the search begins:

$P(s_0, a) = (1 - \varepsilon) \cdot p_a + \varepsilon \cdot \eta_a$

where $\varepsilon = 0.25$ , $\eta \sim \text{Dir}(0.03)$ , and $p_a$ is the raw prior from the network. The Dirichlet distribution with concentration parameter 0.03 produces noise vectors that are concentrated on a small number of moves (most entries are near zero, a few entries have significant probability mass). Mixing 25% of this noise with 75% of the network prior means that:

The neural network's judgment still dominates (75% weight), so the search is not random.
But every move receives at least some probability mass from the noise, and a few moves receive significant noise probability, ensuring the search tries alternatives that the network might initially dismiss.
The search can still overrule bad moves: even if the noise gives high probability to a terrible move, the $Q$ values accumulated during the search will penalize it, and the visit counts will reflect that it's actually bad. The noise only affects the prior; the posterior (reflected in visit counts after many simulations) will still be dominated by genuinely good moves.
The concentration parameter 0.03 is small, meaning the Dirichlet distribution is highly "peaky" — it tends to concentrate most of the noise budget on 2–3 moves rather than spreading it uniformly. This is appropriate because in Go, even for exploration, you don't want to waste simulations on obviously dead stones or filling your own eyes; you want to occasionally try alternative plausible moves.

Resignation mechanism. To save computation, clearly lost games are resigned. The resignation threshold $v_{\text{resign}}$ is automatically adjusted so that false positives (games that could have been won if not resigned) remain below 5%. To measure the false positive rate, resignation is disabled in 10% of self-play games, and those games are played to completion. By comparing the actual outcomes of these full games against what would have been resigned, the system tracks how often resignation would have cost a win, and adjusts the threshold to keep this rate acceptably low. The exact mechanism: if the root value $v$ and the best child value $\max_a Q(s_0, a)$ both fall below $v_{\text{resign}}$ , the system resigns.

Game termination. A game ends under three conditions: both players pass on consecutive moves, the search value drops below the resignation threshold, or the game exceeds a maximum length of $19 \times 19 \times 2 = 722$ moves (an upper bound: each intersection can change hands at most twice before the position must repeat or stabilize). The terminal position is scored to give a final reward $r_T \in \{-1, +1\}$ from the perspective of the player who made the last move. For scoring, Tromp-Taylor rules are used during self-play since they are well-defined even when territorial boundaries are not fully resolved (unlike Chinese, Japanese, or Korean rules, which assume players will complete the boundary-formation phase before passing). However, all tournament and evaluation games are scored using Chinese rules with a komi of 7.5 points, matching the conditions used in the Lee Sedol match.

Data labeling. For each time-step $t$ in the game, the self-play data tuple $(s_t, \boldsymbol{\pi}_t, z_t)$ is stored, where $z_t = \pm r_T$ is the game winner from the perspective of the current player at step $t$ . Since $r_T$ is the reward for the terminal player (the player who made the last move), the sign alternates: if the terminal player won, then $z_t = +1$ for all positions where the same player was to move, and $z_t = -1$ for all positions where the opponent was to move. This perspective-relative labeling ensures that the value network learns to predict "probability of winning for whichever player is about to move."

The evaluator. The evaluator is the quality-control mechanism that determines whether a newly trained network checkpoint is good enough to become the new self-play data generator. Every 1,000 training steps, a new checkpoint $\theta_i$ is produced. The evaluator compares $\alpha_{\theta_i}$ (MCTS search using $f_{\theta_i}$ ) against the current best player $\alpha_{\theta^*}$ in 400 games. Both players use 1,600 MCTS simulations per move with infinitesimal temperature $\tau \to 0$ (deterministic best-move selection) to ensure the strongest possible play from each network.

If $\alpha_{\theta_i}$ wins more than 55% of the games, it replaces $\alpha_{\theta^*}$ as the new best player. The 55% threshold (rather than 50% + 1 game) serves as a statistical significance filter: with 400 games, a 55% win rate corresponds to 220 wins versus 180 losses, which is far enough from 50% to be unlikely to occur by chance alone. If the threshold were exactly 50%, the system might frequently switch to new checkpoints that are not actually stronger but merely got lucky in a small number of games — and if the new checkpoint is actually weaker and is used for self-play generation, the quality of training data degrades and the virtuous cycle can reverse into a death spiral.

Why the evaluator is critical for stability. The evaluator enforces monotonic improvement in the quality of self-play data. Without it, the system could potentially oscillate: a slightly worse network generates lower-quality games, the next network trains on those games and becomes worse still, and the system collapses. The evaluator breaks this potential feedback loop by only allowing the self-play data generator to be updated when there is strong evidence of genuine improvement. This is the mechanism that prevents the "oscillations or catastrophic forgetting" that prior literature had warned about — the learning curve in Figure 3a is smooth precisely because the evaluator ensures that the target distribution for training (the search outputs and game outcomes from the self-play player) is always improving or holding steady, never degrading.

The overall policy iteration interpretation. The Methods section explicitly frames the algorithm as approximate policy iteration:

Policy improvement: starting from the neural network policy $\mathbf{p}$ , MCTS produces an improved policy $\boldsymbol{\pi}$ through explicit lookahead search. The improvement is substantial — the search policy plays at a much higher Elo than the raw network policy — because MCTS can explore tactical sequences and evaluate counterfactual branches that the feedforward network cannot compute in a single forward pass.
Policy evaluation: the outcome $z$ of a self-play game, where both players use the search-improved policy $\boldsymbol{\pi}$ to select moves, provides an unbiased estimate of the value of each position under the improved policy. This is "evaluation" in the RL sense: we're estimating the expected return (win/loss) when following the improved policy from each state.
Projection: the neural network is trained to compress both the improved policy $\boldsymbol{\pi}$ (via cross-entropy) and the improved value estimates $z$ (via MSE) back into the function approximator $f_\theta$ . This projection step is lossy — the network cannot perfectly represent the search outputs — but it captures the essential patterns that make the search effective.
Iteration: the projected network becomes the new prior for the next round of MCTS-based improvement. Because the prior is now stronger, MCTS can search more effectively (better $P$ values guide exploration, better $v$ values provide more accurate leaf evaluations), producing an even stronger improved policy. This positive feedback loop is what enables the system to bootstrap from random play to superhuman performance.

Eliminating Human Knowledge: Design Choices That Enable Tabula Rasa Learning

The paper's central claim is that AlphaGo Zero learns superhuman Go "without human data, guidance or domain knowledge beyond game rules." To substantiate this claim, the Methods section enumerates exactly what domain knowledge the system does use, so that the boundary of the claim is precise:

1. Perfect knowledge of the game rules. The system knows the rules of Go: which moves are legal (stones cannot be placed on occupied intersections, suicide is forbidden, the ko rule prevents immediate repetition of a previous board position), how captures work (connected groups of stones with no liberties are removed), when the game ends (both players pass consecutively), and how to score. This knowledge is used during MCTS to simulate the positions that would result from a sequence of moves (the tree search needs to know what board state results from playing a move at a particular intersection) and to determine when terminal states are reached. This domain knowledge is the minimum required to learn the game at all — if the system didn't know the rules, it couldn't generate valid self-play games.

2. Tromp-Taylor scoring. During MCTS simulations and self-play training, a specific scoring rule set (Tromp-Taylor) is used because it is well-defined even when the game terminates before territorial boundaries are fully resolved. Human scoring rules (Chinese, Japanese, Korean) all assume that players will complete a "boundary-determination phase" where they explicitly mark which stones are dead and which territory is claimed, but this phase has its own conventions that are not codified in the basic rules. Tromp-Taylor rules provide an unambiguous mathematical definition of scoring that works for any terminal position, making them suitable for automated self-play. Final tournament and evaluation games use Chinese rules (with a komi of 7.5 points), which matches the conditions of the human matches.

3. Grid-structured input representation. The network's input is structured as a $19 \times 19$ image — that is, the architecture assumes the Go board is a two-dimensional grid and uses convolutional layers with local receptive fields. This encodes the spatial structure of Go (adjacent intersections are related, distant intersections are mostly independent) but doesn't encode any Go-specific concepts like "groups," "liberties," "eyes," or "territory." The spatial prior from convolutional architectures is a very weak form of domain knowledge compared to the handcrafted features used in prior systems, which explicitly encoded connectivity graphs, eye-detection modules, and territory-mapping heuristics.

4. Rotation and reflection invariance. The rules of Go are invariant under the dihedral group $D_4$ (rotations by 0°, 90°, 180°, 270° and reflections). AlphaGo Zero exploits this symmetry in two ways: during training, the dataset is augmented by including random rotations and reflections of each position (so each training example generates up to 8 variations with the same value target and appropriately transformed policy targets); during MCTS, random rotations or reflections are applied to leaf positions before neural network evaluation. This encodes the knowledge that Go strategy should be symmetric — a good move on the left side should also be a good move on the right side (appropriately reflected) — but it's a mathematical invariance of the board geometry, not a Go-specific insight.

What is NOT included — the absences that matter. Perhaps more important than what is included is what is deliberately excluded:

No human game records: the system never sees a single human-played game. All training data comes from self-play.
No handcrafted evaluation features: no explicit encoding of territory, influence, connectivity, eye shape, life-and-death status, or any other Go concept. The network must discover these patterns from raw board positions.
No rollout policy: no handcrafted fast policy for simulating game completions during search. The neural network's value head alone evaluates leaf nodes.
No tree policy: no handcrafted heuristics for selecting moves within the search tree beyond the PUCT formula, which depends only on the neural network's prior and the accumulated search statistics.
No legal move filtering beyond the basic rules: the system considers all legal moves, including moves that fill in the player's own eyes — a standard heuristic used in all previous Go programs to prune obviously bad moves. AlphaGo Zero must learn through experience that filling your own eyes is bad, just as it learns everything else.
No initialization from human data: the network starts with random weights. The very first MCTS is guided by a completely random neural network, and the system must bootstrap from literally nothing.

This enumeration of inclusions and exclusions serves an important purpose: it draws a clear line around the "tabula rasa" claim. The system is not a blank slate in the philosophical sense — it has a specific neural network architecture and training algorithm designed by humans — but it is a blank slate with respect to Go knowledge. The architecture provides a learning capability, not Go expertise.

Hyperparameters and Their Tuning

The paper provides specific hyperparameter values, several of which were determined by systematic optimization:

MCTS hyperparameters. The constant $c_{\text{puct}}$ was "selected by Gaussian process optimization, so as to optimize self-play performance of AlphaGo Zero using a neural network trained in a preliminary run. For the larger run (40 blocks, 40 days), MCTS search parameters were re-optimized using the neural network trained in the smaller run (20 blocks, 3 days)." Bayesian optimization with Gaussian processes is a standard method for tuning hyperparameters of expensive black-box functions — it builds a probabilistic model of the relationship between hyperparameters and performance, and sequentially chooses hyperparameter settings to evaluate based on an acquisition function that balances exploration and exploitation. The specific value of $c_{\text{puct}}$ is not reported in the paper, but the optimization process is described.

Training hyperparameters. The key values are:

Momentum: 0.9
L2 regularization coefficient $c = 10^{-4}$
Total mini-batch size: 2,048 (64 GPU workers × 32 per worker)
Learning rate schedule (Extended Data Table 3): starts at $10^{-3}$ for the first 400,000 steps, drops to $10^{-4}$ for steps 400,000–600,000, and drops to $10^{-5}$ for steps beyond 600,000 (20-block network; the 40-block schedule is proportionally longer)
Training data window: most recent 500,000 games
Checkpoint frequency: every 1,000 training steps

Self-play hyperparameters.

Games per iteration: 25,000
MCTS simulations per move: 1,600 (approximately 0.4 seconds per move)
Temperature: $\tau = 1$ for first 30 moves, $\tau \to 0$ thereafter
Dirichlet noise: $\varepsilon = 0.25$ , $\eta \sim \text{Dir}(0.03)$
Resignation threshold: automatically tuned to keep false positives below 5%

Evaluation hyperparameters.

Games per evaluation match: 400
MCTS simulations per move: 1,600
Temperature: $\tau \to 0$
Threshold for promotion: > 55% win rate

Summary of Why the Design Enables Tabula Rasa Learning

AlphaGo Zero works because the policy iteration via MCTS framework provides a bootstrapping mechanism that does not require an initial policy to be "in the right ballpark." Here's why:

MCTS with random priors still explores. When the neural network is random, $P(s, a)$ is essentially uniform across all legal moves, and $v$ is essentially noise. The PUCT formula with a uniform prior reduces to essentially uniform exploration. The search will try many different moves, and the $Q$ values accumulated during the search will reflect actual outcomes of these explorations. On average, good moves will accumulate higher $Q$ values (because they lead to better positions), and bad moves will accumulate lower $Q$ values. So even with a completely random network, MCTS produces a move distribution $\boldsymbol{\pi}$ that is better than random — it represents the results of explicit trial-and-error within the search tree.
Training on $\boldsymbol{\pi}$ captures the search's discoveries. Even though the network's raw output $\mathbf{p}$ is terrible, the search output $\boldsymbol{\pi}$ is somewhat better. Training the network to predict $\boldsymbol{\pi}$ captures the search's improved judgment in the network weights. At the next iteration, the network's prior $\mathbf{p}$ will be slightly less random and slightly more correlated with good moves.
The value signal comes from actual game outcomes. The value target $z$ is the actual result of the game. Even if the initial players are terrible, the outcomes are real: one side wins, the other loses. Training the network to predict $z$ from board positions teaches it to associate certain board configurations with winning and others with losing. Initially, these associations will be weak (because the players are bad, the correlation between position quality and outcome is noisy), but they provide a genuine learning signal.
The evaluator ensures monotonic improvement. By only promoting checkpoints that demonstrate statistically significant improvement, the evaluator prevents the system from replacing a working network with a broken one. This is what prevents the oscillation and forgetting that plagued earlier self-play systems: if a new checkpoint is worse (even slightly), it never becomes the data generator, so the quality of self-play data never degrades.
The windowed training data keeps targets relevant. By training only on the most recent 500,000 games, the network is always learning from positions generated by a relatively strong player (the current best). It doesn't waste capacity trying to predict the outcomes of games between very weak players, which would teach less useful patterns.
The virtuous cycle compounds. Each iteration: the network becomes slightly better at move priors and position evaluation → MCTS produces slightly better move recommendations → the games are played at a slightly higher level → the value targets are more informative → the network learns more sophisticated patterns → these patterns guide MCTS more effectively → the search produces even better moves. This positive feedback loop is what enables exponential improvement in the early stages and continued improvement over 40 days of training.

4. Key Insights and Innovations

Innovation 1: Reframing Monte Carlo Tree Search as a Policy Improvement Operator Inside a Closed Learning Loop

The dominant paradigm before AlphaGo Zero — exemplified by AlphaGo Fan, AlphaGo Lee, and essentially all prior work combining neural networks with search — was to treat the neural network and the search algorithm as separate tools that are built independently and then connected at inference time. You train a policy network (usually via supervised learning on human data), you train a value network (via regression on self-play outcomes), you bolt them onto an MCTS engine, and you run the search at test time. The network improves the search, but the search does not feed back to improve the network. The loop is open.

AlphaGo Zero closes this loop by re-conceptualizing MCTS as a policy improvement operator within an approximate policy iteration framework. This is not merely an engineering convenience — it is a conceptual reframing with profound implications. MCTS, given a policy prior and a leaf evaluator, produces a posterior move distribution ( $\boldsymbol{\pi}$ ) that is substantially stronger than the prior. Why? Because the search performs explicit lookahead: it explores counterfactual branches, evaluates their consequences via the value head, and accumulates statistical evidence about which moves actually work. This posterior represents a genuine policy improvement — in the formal reinforcement learning sense of a policy that achieves higher expected return. The innovation is recognizing that this improved policy can be projected back into the neural network by training the policy head to match $\boldsymbol{\pi}$ , compressing thousands of simulations' worth of tactical reasoning into a single forward pass. Then, critically, the improved network becomes the new prior for the next round of search, creating a virtuous cycle: better prior → more efficient search → stronger posterior → better training target → better prior.

This reframing explains why the system can bootstrap from random play. Even a randomly initialized network provides some signal to the search — the prior is essentially uniform, but the search's accumulated $Q$ values, built from actual simulated outcomes, will favor moves that empirically work better than others. Training the network to predict this slightly-better-than-random $\boldsymbol{\pi}$ captures the search's discoveries. At the next iteration, the prior is slightly less random, so the search can spend its simulation budget more efficiently, producing an even stronger posterior. The search creates the training signal that the network needs, and the network amplifies the search's effectiveness, with each turn of the crank producing measurable improvement.

Prior to AlphaGo Zero, systems that projected MCTS outputs into neural networks existed — Guo et al. (2014) did this for Atari games — but critically, they used a fixed MCTS that never improved. There was no iteration, no feedback, no bootstrapping. The network learned to imitate a static search procedure rather than participating in a mutual improvement process. The prior AlphaGo versions had elements of iteration (AlphaGo Lee iterated its value network training using self-play outcomes), but the policy network remained anchored to human data, preventing the full bootstrapping effect.

What makes this a fundamental contribution rather than an incremental refinement is that it identifies the missing feedback connection that enables pure reinforcement learning to work in domains requiring deep lookahead. It transforms MCTS from a test-time inference technique into a training-time improvement operator, which is a genuinely new way to think about the relationship between learning and search. The evidence is in the learning curves (Figures 3a, 6a): the Elo rating rises smoothly and continuously without plateaus or oscillations, demonstrating that the closed loop produces stable, compounding improvement — something prior self-play systems could not achieve without human data to anchor the policy.

Innovation 2: Demonstrating That Human Data Is Not Just Unnecessary but Actively Imposes a Performance Ceiling

The prevailing assumption in AI for complex domains — especially Go, which had resisted computational approaches for decades — was that human expertise was an essential bootstrap. You needed human games to teach the system what reasonable play looked like; without that initial guidance, reinforcement learning would flounder in an enormous search space, never discovering the basic patterns (joseki, life-and-death, influence) that humans had accumulated over millennia. The original AlphaGo paper (Silver et al., 2016) explicitly relied on this assumption: the policy network was initialized by supervised learning on 30 million human positions, and reinforcement learning could only refine what supervised learning had first taught. Every strong Go program before AlphaGo Zero — and indeed every strong game-playing AI in complex domains — depended on human data or handcrafted human knowledge in some form.

AlphaGo Zero's most provocative finding is not just that human data is unnecessary — it's that human data imposes a ceiling on ultimate performance. This is not a philosophical claim; it's demonstrated empirically in Figure 3. A neural network trained by supervised learning on the KGS dataset achieves higher move prediction accuracy on professional moves (Figure 3b) and lower value prediction error on professional game outcomes (Figure 3c) than the self-play trained network. By the metrics that supervised learning directly optimizes — "how well do you imitate human experts?" — the human-trained network is superior. Yet the self-play network dramatically outperforms it in actual playing strength, and continues improving long after the supervised network plateaus. The human-trained player is optimizing for the wrong objective: fidelity to human play rather than winning.

This is a fundamental conceptual shift. It reframes human data from an asset to a potential liability. Human knowledge is incomplete — there exist strategies, tactical patterns, and positional judgments that no human has discovered. A system trained to imitate humans will, by construction, never find them, because it's being rewarded for reproducing what humans already do. The explicit evidence that AlphaGo Zero transcended human knowledge comes from the joseki analysis (Figure 5 and Extended Data Figures 2–3): the system not only independently discovered established corner sequences that humans evolved over centuries, but also discovered and preferred novel variations unknown to professional play. This is not incremental improvement on human play — it's discovering genuinely new knowledge in a domain humans have studied intensively for thousands of years.

The significance of this finding extends far beyond Go. It challenges the default assumption in many AI application domains that the best path to strong performance is to collect large human expert datasets and train models to imitate them. If human data imposes a ceiling in Go — a domain where human expertise is extraordinarily sophisticated and well-documented — the same may be true in other domains. The paper's opening paragraph makes the general argument explicit: expert datasets are expensive, unreliable, sometimes unavailable, and "may impose a ceiling on the performance of systems trained in this manner." AlphaGo Zero provides the strongest empirical evidence to date for that ceiling.

The 100–0 victory against AlphaGo Lee after 36 hours of training (Extended Data Figure 1) is not just a benchmark result — it is a deliberately stark demonstration designed to leave no doubt that the tabula rasa approach is not merely competitive but dramatically superior. The 89–11 victory against AlphaGo Master (Extended Data Figure 6), itself a system that defeated top professionals 60–0, extends this demonstration to the strongest possible human-data-based baseline. The gap is not marginal; it is decisive.

Innovation 3: Unifying Policy and Value Networks Into a Single Architecture with Shared Representations

Prior to AlphaGo Zero, the standard architecture for deep reinforcement learning in games — across AlphaGo Fan, AlphaGo Lee, and the broader literature — was to train separate networks for policy (action selection) and value (position evaluation). The rationale was straightforward: these are different tasks with different output structures (a probability distribution over hundreds of moves versus a single scalar), and dedicating separate parameters to each task would avoid interference. If the features that predict good moves are different from the features that predict winning positions, why force them to share?

AlphaGo Zero's architecture comparison (Figure 4) provides a striking empirical counterargument. Comparing four architectures — dual-res (combined policy/value, residual network), sep-res (separate policy and value, residual), dual-conv (combined, convolutional), and sep-conv (separate, convolutional, the AlphaGo Lee architecture) — the dual-res architecture achieves approximately 600 Elo higher playing strength than sep-res, and another approximately 600 Elo higher than sep-conv. The combined architecture (dual-res) achieves slightly lower move prediction accuracy on professional moves than the separate architecture — meaning the network is slightly worse at the specific task of predicting what humans would play — but significantly better at actual playing strength.

What's happening here is a form of representation learning via multi-task regularization. Learning to evaluate positions and learning to select moves are complementary objectives that, when trained jointly, produce internal representations that are more robust and general than either task would produce alone. The value objective forces the network to attend to features that predict long-term outcomes — territory, influence, group safety, tactical potential — while the policy objective forces attention to features that predict immediate good moves. A representation that simultaneously supports both tasks must capture the structural properties of Go positions that are relevant for both prediction and decision-making, rather than specializing to either. The slight decrease in move prediction accuracy is actually evidence that this is working: the network is not overfitting to the specific distribution of human moves, but instead learning a more general representation that happens to be slightly less calibrated to human move frequencies.

This unification also brings practical advantages: only one network to deploy during search (halving the computational cost of leaf evaluation), only one set of weights to maintain, and a simpler training pipeline. But the paper argues the performance gain is "more importantly [due to] the dual objective regulariz[ing] the network to a common representation that supports multiple use cases" — the conceptual contribution is the recognition that joint training provides a beneficial inductive bias, not just engineering convenience.

This finding generalizes beyond Go. The idea that policy and value networks should share representations has since become standard in deep reinforcement learning (e.g., in AlphaZero's successors for chess and shogi, and in many actor-critic architectures), but AlphaGo Zero provided the first clear empirical demonstration that unification not only doesn't hurt but substantially helps — and that the mechanism is regularization, not just efficiency.

Innovation 4: Eliminating Rollouts — Proving That a Learned Evaluator Alone Suffices for Deep Search

A cornerstone of Monte Carlo tree search since its inception — in Go programs (Coulom, 2006; Gelly and Silver, 2011), in general game-playing (Browne et al., 2012), and in the original AlphaGo — was the rollout: simulating thousands of rapid, semi-random games from leaf nodes to completion, using a lightweight policy, to obtain noisy but unbiased estimates of position value. Rollouts were considered essential because learned value functions, while more accurate per evaluation, were biased — they might systematically misevaluate certain types of positions due to blind spots in their training distribution. Rollouts provided a complementary signal: noisy but asymptotically unbiased (if the rollout policy is reasonable), and therefore a crucial safeguard against value function errors. The original AlphaGo combined rollout evaluations with value network evaluations via a weighted average, and this combination was widely considered necessary for strong play.

AlphaGo Zero eliminates rollouts entirely. Leaf nodes are evaluated solely by the neural network's value head — a single forward pass producing a scalar $v \in [-1, 1]$ . This is possible because the value network, trained within the policy iteration loop on self-play outcomes, becomes sufficiently accurate that the noise reduction from rollouts is no longer worth their computational cost (they consume simulation budget that could instead be spent on deeper tree search guided by the network's priors). But the deeper insight is that the alleged necessity of rollouts was an artifact of insufficiently strong learned evaluators. When your value function is trained on human games or on outcomes from a weak policy, it has systematic blind spots that rollouts can partially correct. When your value function is trained within a bootstrapping loop where the policy and value improve together, it becomes sufficiently reliable that the bias-variance tradeoff shifts decisively in favor of the learned evaluator.

This is a significant conceptual contribution because it simplifies the MCTS algorithm substantially — no need to design, implement, or tune a separate rollout policy, no need to balance rollout evaluations against network evaluations, no distribution mismatch between rollout play and real play — and because it demonstrates that learned evaluation can fully replace simulation-based evaluation when the learning loop is properly constructed. The evidence is in the final performance: AlphaGo Zero with no rollouts substantially outperforms AlphaGo Lee and AlphaGo Master, both of which used elaborate rollout mechanisms. The rollouts were not just unnecessary; they were a crutch that, once removed, enabled a cleaner and stronger system.

This finding parallels a broader trend in AI where learned components replace engineered ones — learned value functions replacing handcrafted evaluation heuristics in chess (from Deep Blue to AlphaZero), learned policies replacing handcrafted rollout policies in Go — but AlphaGo Zero provides the cleanest demonstration because the entire system, not just one component, is learned from scratch.

5. Experimental Analysis

Evaluation Methodology

Dataset. The primary training signal comes from self-play games generated continuously during training — 4.9 million games for the 20-block, 3-day run, and 29 million games for the 40-block, 40-day run. The neural network is trained on positions sampled uniformly from the most recent 500,000 self-play games, producing a constantly refreshing dataset of (s, π, z) tuples that tracks the system's improving strength. For supervised learning comparisons and move prediction accuracy measurements, the KGS Go Server dataset (amateur games) serves as training data, while the GoKifu dataset (professional games) serves as the validation set. For Elo rating computation, evaluation games between different players are used, with the Elo scale anchored to previously published values from Silver et al. (2016) to provide human-relevant calibration.
Base model(s). The paper studies two instances of the AlphaGo Zero architecture. The smaller instance (typically the focus of training dynamics analysis) uses a neural network with 20 residual blocks (39 parameterized layers in the residual tower plus output heads), trained for approximately 3 days. The larger instance, used for the final performance evaluation, uses 40 residual blocks (79 parameterized layers in the residual tower), trained for approximately 40 days. Both start from completely random weights — there is no pretraining, no initialization from human data, no transfer from a weaker version. The choice of two scales serves to demonstrate both rapid learning dynamics (the 20-block system reaches superhuman level in ~36 hours) and asymptotic performance limits (the 40-block system substantially surpasses all prior versions). For the FLOPs-matched architecture comparison (Figure 4), four network variants are studied: dual-res (combined policy/value, residual), sep-res (separate policy and value, residual), dual-conv (combined, 12-layer convolutional), and sep-conv (separate, 12-layer convolutional, matching AlphaGo Lee's architecture).
Metrics. Performance is measured on the Elo rating scale, using the standard logistic model where a 200-point gap corresponds to approximately 75% probability of winning. Elo ratings are computed via Bayesian logistic regression using the BayesElo program (Coulom, 2008), with the constant c_elo = 1/400. Two additional metrics assess learned knowledge quality: move prediction accuracy (the percentage of positions where the neural network assigns its highest probability to the human professional move, measured on the GoKifu dataset) and mean squared error of game outcome prediction (the MSE between the actual outcome z ∈ {-1, +1} and the network's predicted value v, scaled by a factor of 1/4 to the range 0–1, also on the GoKifu dataset). These metrics capture different aspects of learning: Elo measures playing strength, prediction accuracy measures how well the network imitates human experts, and value MSE measures how accurately the network evaluates positions.
Baselines. The paper evaluates against a hierarchy of prior systems:
- AlphaGo Master: the previously unpublished system that defeated top human professionals 60–0 in online games in January 2017 (Huang, 2017). It uses the same neural network architecture, reinforcement learning algorithm, and MCTS algorithm as AlphaGo Zero — but was initialized by supervised learning from human data and uses the same handcrafted features and rollouts as AlphaGo Lee. This is the strongest human-data-based baseline.
- AlphaGo Lee: the version that defeated Lee Sedol 4–1 in March 2016. Uses separate policy and value networks (12 convolutional layers of 256 planes each), trained with supervised learning initialization followed by reinforcement learning, with handcrafted rollouts and features. Distributed over many machines using 48 TPUs.
- AlphaGo Fan: the previously published version (Silver et al., 2016) that defeated European champion Fan Hui in October 2015. Distributed over many machines using 176 GPUs.
- Raw network: the AlphaGo Zero neural network without any MCTS search — it simply selects the move with maximum probability p_a in a single forward pass. This isolates the contribution of the network alone versus the network-plus-search combination.
- Prior Go programs: Crazy Stone, Pachi, and GnuGo, representing the state of the art in pre-neural-network Go engines. Their Elo ratings are anchored to the values from Silver et al. (2016).
- Supervised learning baseline (Figures 3, Extended Data Tables 1–2): a network with identical architecture to AlphaGo Zero, trained by supervised learning on the KGS dataset to predict human expert moves (setting π_a = 1 for the human move and weighting the value MSE component by 0.01). This isolates the contribution of the learning algorithm (self-play RL vs. human imitation) while holding architecture constant.
Generation budget / compute accounting. The primary unit of test-time compute is MCTS simulations per move. All evaluation games (for Elo computation) use 1,600 simulations per move (approximately 0.4 seconds of thinking time), with the exception of the 5-second-per-move tournament used for the final Elo comparison (Figure 6b), where each program runs with whatever simulations it can complete in 5 seconds. Self-play generation also uses 1,600 simulations per move. For the final head-to-head matches against AlphaGo Lee and AlphaGo Master, 2-hour time controls with 3 byoyomi periods of 60 seconds per move are used, matching the conditions of the Lee Sedol match in Seoul. Computational resources are explicitly tracked: AlphaGo Zero and AlphaGo Master each use a single machine with 4 TPUs; AlphaGo Fan and AlphaGo Lee are distributed over 176 GPUs and 48 TPUs respectively — meaning AlphaGo Zero achieves its results with substantially less hardware. Training resources: 64 GPU workers and 19 CPU parameter servers on Google Cloud, with a total mini-batch size of 2,048 positions.
Cross-validation / statistical protocol. The evaluator (described in Methods) serves as the primary quality-control and selection mechanism. Every 1,000 training steps, a new neural network checkpoint is produced. The evaluator pits this checkpoint (α_{θ_i}, using MCTS with 1,600 simulations and τ → 0) against the current best player (α_{θ*}) in 400 games. If the new checkpoint wins more than 55% of the games, it replaces the current best and becomes the new self-play data generator. The 55% threshold (rather than 50%) provides a statistical significance filter on a finite sample of 400 games. Elo ratings for training progress (Figures 3a, 6a) are computed from evaluation games between different iterations of α_{θ_i} during self-play training, with further evaluations against baseline players with anchored Elo ratings. The final tournament Elo ratings (Figure 6b) incorporate the results of matches against human professionals (Fan Hui and Lee Sedol) to ground the scale to human references and mitigate self-play bias in Elo computation (whereby systems that only play each other can develop inflated ratings).

Main Quantitative Results

Training Dynamics and Comparison Against Human-Data-Based Systems (Figure 3, Extended Data Figure 1)

Headline result: AlphaGo Zero surpasses AlphaGo Lee after 36 hours of self-play training, winning 100–0 in a 2-hour time control match, and substantially exceeds the performance of a supervised learning baseline despite the baseline achieving better move prediction accuracy on professional moves.

Figure 3a shows the Elo rating of AlphaGo Zero (20-block) as a function of training time, alongside a player trained by supervised learning from human KGS data (using the identical neural network architecture). The self-play RL player starts from essentially random play (negative Elo) and improves smoothly throughout the 72-hour training run, passing successively through amateur levels and into superhuman territory. The supervised learning baseline starts at a higher initial Elo (it begins with useful move predictions rather than random ones) but plateaus quickly and is overtaken by the self-play player within the first 24 hours of training. After 36 hours, AlphaGo Zero's self-play player surpasses AlphaGo Lee (which was trained over several months). The learning curve shows no oscillations, no catastrophic forgetting, and no plateaus — the improvement is continuous.

Figure 3b shows move prediction accuracy on professional moves (GoKifu dataset) for both the self-play and supervised networks, evaluated at each training iteration. The supervised network achieves higher accuracy throughout — it is better at predicting what human professionals would play. The self-play network's accuracy actually decreases slightly in the later stages of training, even as its playing strength continues to rise. Figure 3c shows the mean squared error on predicting professional game outcomes. The supervised network achieves lower (better) MSE initially, but the self-play network overtakes it and achieves substantially lower error by the end of training — it becomes better at evaluating who is winning, even in professional games, despite never having seen a human game.

The head-to-head match result (described in the main text and shown in Extended Data Figure 1): after 72 hours of training, AlphaGo Zero (20-block) was evaluated against "the exact version of AlphaGo Lee that defeated Lee Sedol, under the same 2 h time controls and match conditions that were used in the man-machine match in Seoul." AlphaGo Zero used a single machine with 4 TPUs; AlphaGo Lee was distributed over many machines using 48 TPUs. AlphaGo Zero won 100 games to 0. Extended Data Figure 1 shows the first 100 moves of the first 20 games; the full games are provided in the Supplementary Information.

Interpretation of Figure 3: The decoupling of move prediction accuracy from playing strength is one of the paper's most important empirical findings. The network that better imitates human moves is not the better Go player. This demonstrates that the objective "predict human moves" is fundamentally different from the objective "win games," and that optimizing for the former can actually impede the latter. The fact that the self-play network's move prediction accuracy on professional moves decreases late in training — while its Elo continues to climb — suggests it is discovering strategies that humans do not employ, which makes its move predictions diverge from the human distribution. The value prediction error (Figure 3c) tells a complementary story: the self-play network becomes better at evaluating positions, even in professional games, because understanding "who is winning" is more fundamental and transferable than understanding "what move would a human play here."

Architecture Comparison: Dual vs. Separate, Residual vs. Convolutional (Figure 4)

Headline result: The combined policy-and-value network with residual architecture (dual-res) achieves approximately 1,200 Elo higher playing strength than the separate policy and value networks with convolutional architecture (sep-conv) used in AlphaGo Lee, with approximately 600 Elo attributable to the residual architecture and approximately 600 Elo attributable to the unified dual-head design.

Figure 4 reports a controlled comparison of four neural network architectures, all trained on the same fixed dataset: the final 2 million self-play games generated by a previous AlphaGo Zero run. Each trained network is then combined with AlphaGo Zero's MCTS search to produce a player, and Elo ratings are computed from evaluation games between these players using 5 seconds of thinking time per move.

The four architectures are:

dual-res: combined policy and value heads sharing a 20-block residual tower (AlphaGo Zero's architecture)
sep-res: separate 20-block residual towers for policy and value (same total capacity, but no shared representation)
dual-conv: combined policy and value heads sharing a 12-layer non-residual convolutional tower
sep-conv: separate 12-layer convolutional towers for policy and value (AlphaGo Lee's architecture)

Elo results (Figure 4a):

dual-res achieves the highest Elo, approximately 4,500
sep-res is lower by approximately 600 Elo (≈3,900)
dual-conv is lower than sep-res by approximately another 600 Elo (≈3,300)
sep-conv is the weakest, close to 3,300 but slightly below dual-conv

The ~600 Elo gap between dual-res and sep-res isolates the effect of combining policy and value into a single network. The further ~600 Elo gap between sep-res/dual-conv and sep-conv isolates the effect of using residual connections. The total gap of ~1,200 Elo from sep-conv (AlphaGo Lee architecture) to dual-res (AlphaGo Zero architecture) is enormous — representing going from an approximately 75% win rate to somewhere around a 99.9% win rate against the same opponent.

Move prediction accuracy (Figure 4b): The dual-res network achieves slightly lower move prediction accuracy on professional moves (approximately 46.5% at its best) compared to sep-res (approximately 47.5%). In other words, combining the heads slightly reduces the network's ability to predict what humans would play. The residual architectures (dual-res, sep-res) substantially outperform the convolutional architectures (dual-conv, sep-conv) on move prediction — approximately 47% vs. 45% — confirming that residual connections improve representational capacity.

Value prediction error (Figure 4c): The dual-res network achieves the lowest MSE on professional game outcome prediction (approximately 0.165 at its best), followed by sep-res (approximately 0.170), then dual-conv and sep-conv (both around 0.185–0.190). The dual architecture reduces value error compared to separate networks, and the residual architecture reduces it further.

Why this is important: This ablation separates the contributions of architecture and learning algorithm. The self-play RL training procedure (Section 3) is the same for all networks — they all minimize equation (1) on the same fixed dataset. The differences in playing strength therefore isolate the effect of network architecture on the search's ability to use the network's outputs effectively. The finding that combining policy and value into a single network improves playing strength by 600 Elo, despite slightly reducing move prediction accuracy, is the key evidence for the paper's claim that shared representations provide beneficial regularization. The network learns features that simultaneously support policy and value prediction, and these features prove more useful for guiding search than features specialized to either task alone.

Knowledge Discovery During Training (Figure 5, Extended Data Figures 2–5)

Headline result: AlphaGo Zero independently discovers standard human Go knowledge (joseki, fuseki, life-and-death, etc.) during self-play training, and also discovers novel joseki variations unknown in human play, with the timeline of discovery showing that different concepts emerge at different stages of training — and notably, shicho (ladder capture sequences), one of the first concepts learned by human beginners, is only understood by AlphaGo Zero much later in training.

Figure 5a shows five standard human joseki (corner sequences common in professional play) that AlphaGo Zero independently discovered during self-play training, along with timestamps indicating the first training hour at which each sequence appeared (taking account of rotation and reflection). Extended Data Figure 2 shows the frequency of occurrence of each sequence over the full training run — the sequences appear, sometimes grow in popularity, and sometimes decline as the system moves on to other strategies.

Figure 5b shows five joseki that were most frequently played by AlphaGo Zero during at least one iteration of self-play training. This includes the 3–3 invasion (a standard human professional joseki, favored at 47 hours) and new variations that AlphaGo Zero subsequently discovered and preferred — joseki that are "previously unknown" in human play. Extended Data Figure 3 tracks the frequency of these sequences over training, showing how preferences shift: the 3–3 invasion rises to prominence, then a new variation appears and partially displaces it. The system does not simply find and fixate on human patterns; it discovers them, uses them, and then moves beyond them to novel variations.

Figure 5c shows the first 80 moves of three self-play games played at different stages of training, using 1,600 simulations (approximately 0.4 seconds) per search:

At 3 hours: "the game focuses greedily on capturing stones, much like a human beginner." The play is unsophisticated and tactical rather than strategic.
At 19 hours: "the game exhibits the fundamentals of life-and-death, influence and territory." The system has progressed from pure tactics to strategic concepts.
At 70 hours: "the game is remarkably balanced, involving multiple battles and a complicated ko fight, eventually resolving into a half-point win for white." The play is sophisticated, involving advanced concepts like ko fights and precise endgame calculation.

Extended Data Figures 4 and 5 show full tournament-length games (2-hour time controls) played at regular intervals throughout the 3-day (20-block) and 40-day (40-block) training runs, providing a visual record of the system's progression from random play to superhuman sophistication. The paper's main text summarizes the learning trajectory: "AlphaGo Zero rapidly progressed from entirely random moves towards a sophisticated understanding of Go concepts, including fuseki (opening), tesuji (tactics), life-and-death, ko (repeated board situations), yose (endgame), capturing races, sente (initiative), shape, influence and territory, all discovered from first principles."

The shicho (ladder) finding: The paper explicitly highlights that "shicho ('ladder' capture sequences that may span the whole board) — one of the first elements of Go knowledge learned by humans — were only understood by AlphaGo Zero much later in training." This is a non-obvious finding that demonstrates AlphaGo Zero's learning trajectory is genuinely different from the human one. Human beginners learn ladders early because they are a simple, deterministic tactical pattern that humans explicitly teach. AlphaGo Zero apparently discovers other concepts first (perhaps because they provide more immediate reward signal in self-play) and only later learns to handle the long-range tactical sequences that ladders require.

What this analysis demonstrates: The knowledge discovery timeline is not just a qualitative curiosity — it is evidence that the system is building genuine understanding from first principles, not merely memorizing patterns from human data (it has none) or exploiting superficial statistical correlations. The fact that it independently discovers concepts that took humans millennia to codify, and then goes beyond them to find novel strategies, provides compelling evidence that the learning process is extracting real structural knowledge about Go rather than overfitting to a particular data distribution.

Final Performance: AlphaGo Zero vs. Prior Versions (Figure 6, Extended Data Figure 6)

Headline result: The fully trained 40-block AlphaGo Zero achieves an Elo rating of 5,185 — substantially surpassing AlphaGo Master (4,858), AlphaGo Lee (3,739), and AlphaGo Fan (3,144) — and wins a 100-game head-to-head match against AlphaGo Master by 89–11 under 2-hour time controls.

Figure 6a shows the learning curve for the 40-block AlphaGo Zero over 40 days of training, using 0.4 seconds per search for Elo computation. The improvement is smooth and asymptotic, with the system passing AlphaGo Lee's level early in training and continuing to improve steadily up to approximately 5,200 Elo, substantially beyond AlphaGo Master (shown as a horizontal reference line at approximately 4,858).

Figure 6b shows the final tournament results, with all programs allowed 5 seconds of thinking time per move:

AlphaGo Zero (40-block): 5,185 Elo (single machine, 4 TPUs)
AlphaGo Master: 4,858 Elo (single machine, 4 TPUs) — a gap of 327 Elo, corresponding to approximately 87% expected win rate for AlphaGo Zero
Raw network (AlphaGo Zero without MCTS): 3,055 Elo — this is the neural network alone, selecting the move with maximum probability without any search. The gap of 2,130 Elo between the raw network and AlphaGo Zero demonstrates the enormous contribution of MCTS search.
AlphaGo Lee: 3,739 Elo (distributed over many machines, 48 TPUs)
AlphaGo Fan: 3,144 Elo (distributed over many machines, 176 GPUs)
Crazy Stone, Pachi, GnuGo: positioned lower on the scale, anchored to previously published values from Silver et al. (2016)

The head-to-head match result (Extended Data Figure 6): AlphaGo Zero (40-block) defeated AlphaGo Master by 89 games to 11 in a 100-game match with 2-hour time controls. Extended Data Figure 6 shows the first 100 moves of the first 20 games; full games are in the Supplementary Information.

Scale calibration note: The Elo ratings are anchored to include the results of the human matches (AlphaGo Fan vs. Fan Hui, AlphaGo Lee vs. Lee Sedol), which prevents self-play inflation and grounds the scale in human-relevant terms. Without this anchoring, Elo ratings from a closed pool of self-play evaluations can drift upward or compress, making absolute comparisons misleading.

The raw network result is particularly important: at 3,055 Elo, the raw neural network without any search is already at a strong amateur or low professional level — above AlphaGo Fan (3,144? no, below — 3,055 vs 3,144). Actually, 3,055 is below AlphaGo Fan's 3,144, meaning even without search the network plays at approximately the level of the strongest pre-AlphaGo programs (in the range of Crazy Stone, Pachi, GnuGo — their exact ratings aren't given but they anchor below AlphaGo Fan). The search adds over 2,000 Elo on top of this, demonstrating that the neural network provides good "intuition" but the lookahead search is what converts that intuition into superhuman precision.

Head-to-Head Match Details: 100–0 vs. AlphaGo Lee (Extended Data Figure 1)

Extended Data Figure 1 shows the first 100 moves of the first 20 games of the 100-game match between AlphaGo Zero (20-block, 3 days) and AlphaGo Lee, played under 2-hour time controls matching the Lee Sedol match conditions. AlphaGo Zero won all 100 games. The displayed game records show diverse opening patterns, sophisticated middle-game fighting, and what Go experts would recognize as high-level strategic play. The 100–0 scoreline — a complete sweep against a system that defeated one of the strongest human players in history — is presented as decisive evidence that the tabula rasa approach is not merely competitive but dramatically superior.

Ablation Studies and Robustness Checks

Residual vs. convolutional architecture (Figure 4, dual-res vs. sep-res vs. dual-conv vs. sep-conv): The residual architecture provides approximately 600 Elo of playing strength improvement over the convolutional architecture, holding the policy/value combination constant. This is visible in the Elo gap between dual-res and dual-conv, and between sep-res and sep-conv. The residual architecture also substantially improves move prediction accuracy (Figures 4b: approximately 47% vs. 45% for the policy/value combined variants) and reduces value prediction error (Figures 4c: approximately 0.165–0.170 vs. 0.185–0.190). This confirms that the ResNet architecture's ability to train deeper networks is critical to AlphaGo Zero's performance — the 12-layer convolutional network simply cannot represent Go knowledge as effectively as the 39/79-layer residual network.

Combined vs. separate policy and value networks (Figure 4): The dual architecture provides approximately 600 Elo of playing strength improvement over separate networks, despite reducing move prediction accuracy by approximately 1 percentage point (Figures 4b: ~46.5% vs. ~47.5% for residual variants). The value prediction error is lower for the dual architecture (Figures 4c: ~0.165 vs. ~0.170). This demonstrates that the benefit of shared representations is not simply due to parameter efficiency (the separate architecture has twice as many parameters) — it's a genuine regularization effect where learning to evaluate positions improves the features used for move selection and vice versa.

Supervised learning vs. reinforcement learning (Figure 3): Holding the neural network architecture constant (both use the 20-block residual design), the reinforcement learning player overtakes the supervised learning player within 24 hours of training and continues to improve for the full 72 hours, while the supervised player plateaus. The supervised player achieves better move prediction accuracy on professional moves (Figures 3b) and initially better value prediction (Figures 3c), but worse actual playing strength. This ablation isolates the effect of the training objective: predicting human moves vs. learning to win through self-play. The result directly supports the paper's claim that human data imposes a performance ceiling.

MCTS search vs. raw network (Figure 6b): The raw AlphaGo Zero neural network (without MCTS, selecting the maximum-probability move) achieves 3,055 Elo, compared to 5,185 Elo with 1,600 MCTS simulations and ~4,500 Elo during training evaluations (Figures 3a, 6a). This 2,130 Elo gap demonstrates that the search is essential for superhuman performance and is not merely a small refinement on top of an already-superhuman network.

Training duration and network scale (Figures 3 vs. 6): The 20-block network trains for 3 days (4.9 million self-play games, 700K mini-batches) and reaches roughly 4,500 Elo. The 40-block network trains for 40 days (29 million self-play games, 3.1M mini-batches) and reaches 5,185 Elo. The learning curve in Figure 6a continues to rise slowly at 40 days, suggesting further training might yield additional gains, though the rate of improvement has substantially slowed. The gap between 4,500 and 5,185 represents a significant jump — noting that a 200 Elo gap corresponds to ~75% win probability, this ~685 Elo improvement corresponds to going from ~75% to ~98%+ expected win rate against the same opponent.

Evaluation against different AlphaGo versions (Figure 6b): The systematic Elo ranking across AlphaGo Zero, AlphaGo Master, AlphaGo Lee, AlphaGo Fan, and the raw network establishes a complete performance hierarchy. The gaps are large and consistent: ~327 Elo to AlphaGo Master, ~1,446 Elo to AlphaGo Lee, ~2,041 Elo to AlphaGo Fan. These gaps are measured under uniform 5-second-per-move conditions, enabling direct comparison of the systems' strength independent of hardware differences (since AlphaGo Zero and AlphaGo Master use single machines while AlphaGo Lee and Fan used distributed systems).

Time control and hardware fairness: AlphaGo Zero (20-block) defeated AlphaGo Lee 100–0 using a single machine with 4 TPUs, while AlphaGo Lee was distributed over many machines with 48 TPUs, under identical 2-hour time controls. The 100–0 result is therefore despite a substantial hardware disadvantage. Similarly, AlphaGo Zero (40-block) defeated AlphaGo Master 89–11 with both using single machines with 4 TPUs — hardware parity. This demonstrates that the performance advantage is algorithmic, not due to raw computational superiority.

Rotation and reflection augmentation (described in Methods but not separately ablated): The MCTS uses random dihedral transformations of leaf positions before neural network evaluation, and training data is augmented with all 8 symmetries. This exploits the known invariance of Go under rotation and reflection, and is listed as one of the four items of domain knowledge the system uses. The paper does not report an ablation without this augmentation, but the practice is well-established in image-based deep learning and prior Go work.

Dirichlet noise for exploration (described in Methods): The root node prior is blended with Dirichlet noise (ε = 0.25, η ~ Dir(0.03)) to ensure exploration. The specific parameters were presumably tuned (the MCTS hyperparameters were optimized via Gaussian process optimization), but no ablation on the noise level or distribution is reported. The temperature schedule (τ = 1 for first 30 moves, τ → 0 thereafter) similarly lacks an explicit ablation but is a standard exploration-exploitation tradeoff design.

Resignation threshold tuning: The resignation threshold v_resign is automatically adjusted to keep false positives below 5%, measured by disabling resignation in 10% of self-play games and checking whether resigned games could have been won. This is a self-tuning mechanism rather than a fixed hyperparameter, and the 5% threshold is a design choice without reported sensitivity analysis.

Training data window (500K most recent games): The network trains on positions from the most recent 500,000 self-play games. This window size is a hyperparameter that controls the tradeoff between data diversity (larger window = more positions, including older weaker play) and data relevance (smaller window = only recent, strong play). No ablation on window size is reported.

Missing ablation: value of the evaluator threshold. The 55% threshold for promoting a new checkpoint is a key stability mechanism, but no ablation on this threshold (e.g., 50%, 60%) is reported. It would be informative to know whether a lower threshold causes instability and whether a higher threshold unnecessarily slows progress.

Missing ablation: number of MCTS simulations. All evaluations use 1,600 simulations per move. An ablation sweeping simulation counts (e.g., 100, 400, 1,600, 6,400) would reveal how performance scales with search depth at different training stages, and whether more simulations would yield further gains or plateau. The paper implicitly provides one data point: the raw network (0 simulations) vs. 1,600 simulations (Figures 6b), but no intermediate values.

Critical Assessment

Claim 1: "AlphaGo Zero achieved superhuman performance, winning 100–0 against the previously published, champion-defeating AlphaGo" (from the abstract).

This claim is fully supported. The 100–0 result against AlphaGo Lee (Extended Data Figure 1) is decisive and clearly documented. The match used identical time controls and conditions to the Lee Sedol match, establishing that AlphaGo Zero's victory was not due to computational advantages (AlphaGo Zero used fewer TPUs on a single machine vs. AlphaGo Lee's distributed system). The 89–11 result against the even stronger AlphaGo Master (Extended Data Figure 6) under 2-hour time controls with hardware parity further reinforces the claim. The final Elo ratings (Figure 6b) provide a complete performance hierarchy that is internally consistent and anchored to human references.

Genuine strength: The 100–0 sweep against AlphaGo Lee on a single machine with 4 TPUs (vs. AlphaGo Lee's 48 TPUs distributed) is a remarkably clean result. It eliminates any concern that the new system merely had more compute — it had less, and still won every game.

Nuance: The claim is about winning against "the previously published, champion-defeating AlphaGo" — i.e., AlphaGo Lee, which defeated Lee Sedol. This is precisely what was tested. The 89–11 against AlphaGo Master is an even stronger result but is not part of this specific wording.

Claim 2: "Starting tabula rasa" — learning from random play with "no human data, guidance or domain knowledge beyond game rules."

This claim requires careful examination because "no domain knowledge" is a strong statement that the paper itself qualifies. The Methods section explicitly enumerates four forms of domain knowledge AlphaGo Zero uses: perfect knowledge of the game rules, Tromp-Taylor scoring, the 19×19 grid-structured input, and rotation/reflection invariance. The question is whether these constitute "domain knowledge beyond game rules."

The game rules (legal moves, capture mechanics, ko, game termination conditions) are the minimum necessary to simulate the game at all — without them, there is no way to generate valid self-play trajectories or to know the outcome of a completed game. This is genuinely unavoidable for any system that learns by playing the game.

The grid-structured input is a form of architectural prior — convolutional networks assume spatial locality — but it's a very weak one. It encodes the fact that the Go board is a 2D grid where nearby intersections are related, but it does not encode any Go-specific concepts (groups, liberties, eyes, territory, etc.). This is qualitatively different from the handcrafted features used in prior Go programs (explicit group connectivity, eye-detection modules).

The rotation/reflection invariance exploits a mathematical symmetry of the board geometry. It is also a fairly weak prior — it says that the board has no distinguished orientation, which is true of the rules — and is standard practice in image-based deep learning.

Tromp-Taylor scoring is arguably the most domain-specific knowledge beyond basic rules. It is a specific mathematical formalization of Go scoring that handles edge cases (unresolved territorial boundaries) differently from human scoring systems. However, some scoring rule is necessary to determine the winner of a completed game, and Tromp-Taylor was chosen for its mathematical well-definedness rather than for Go-specific reasons.

Overall, the "no domain knowledge beyond game rules" claim is broadly justified as stated, with the caveat that the architectural priors (convolution, symmetry) encode weak geometric knowledge. The paper is transparent about what knowledge is used and is careful to enumerate it. Whether a skeptic would accept "convolutional network with symmetry augmentation" as "no domain knowledge beyond rules" depends on philosophical priors, but the paper's disclosure is complete and the priors are minimal compared to prior work.

Claim 3: "A pure reinforcement learning approach requires just a few more hours to train, and achieves much better asymptotic performance, compared to training on human expert data" (from the Conclusion).

This claim is supported with strong quantitative evidence. Figure 3a shows the RL player overtaking the supervised baseline within 24 hours and continuing to improve while the supervised baseline plateaus. The "few more hours" is empirically accurate: the supervised baseline starts stronger but plateaus, while RL keeps improving. The "much better asymptotic performance" is visible in the Elo gap at 72 hours (the RL player is substantially higher) and in the 100–0 result against AlphaGo Lee (which was trained with human data).

Nuance in the comparison: The supervised learning baseline uses the identical neural network architecture as AlphaGo Zero and trains on the KGS dataset (amateur games). However, there is a subtle confound: the supervised network was trained to match the specific loss in equation (1), but with the value component weighted by 0.01 (to prevent overfitting, as described in Methods). This means the supervised network receives less pressure to learn accurate value predictions compared to the RL network, which weights policy and value losses equally. It's possible that a supervised network trained with equal weighting (and better regularization against overfitting) might perform better, though the paper's earlier work (Silver et al., 2016) suggests value overfitting is a genuine problem for supervised training on human data.

Additionally, the supervised baseline is initialized randomly and trained only on KGS data — it does not use the full AlphaGo Lee pipeline (supervised pretraining + RL fine-tuning + rollouts + separate value network). The supervised baseline in Figure 3 is therefore not the strongest possible human-data-based system; AlphaGo Lee represents that. The 100–0 over AlphaGo Lee addresses this concern.

Claim 4: The architecture unification (dual-res) adds ~600 Elo over separate networks, and residual networks add another ~600 Elo (Figure 4).

This claim is supported by the controlled architecture comparison in Figure 4, where all networks are trained on identical data. The Elo gaps are clear and substantial. However, the experiment has a limitation: all networks were trained on data generated by a previous run of AlphaGo Zero (the 20-block, 3-day version). The data is therefore from a system that already benefits from the dual-res architecture. If the training data were generated by each architecture's own self-play (as in the full training loop), the relative performance might differ — architectures that initially learn faster might generate better training data sooner, amplifying their advantage through the virtuous cycle. The fixed-dataset comparison isolates architectural effects from training dynamics, which is scientifically clean but may underestimate or overestimate the advantage in the full bootstrapping setting.

Claim 5: The system discovers human Go knowledge and novel strategies (Figure 5, Extended Data Figures 2–5).

This claim is qualitatively supported by the visual evidence and frequency analysis. The joseki timelines (Figures 5a–b, Extended Data Figures 2–3) show clear patterns of discovery and preference shifts. The game records (Figures 5c, Extended Data Figures 4–5) show progression from random play to sophisticated strategic understanding. The claim that the system discovered "novel strategies that provide new insights into the oldest of games" (from the Conclusion) is more subjective — it depends on expert Go judgment that is not fully presented in the paper (e.g., were the "new" joseki variations genuinely unknown to professionals, or merely uncommon?). The paper does not provide external validation (e.g., professional Go player commentary) of the novelty claim. However, for the paper's main argument — that self-play RL can discover sophisticated domain knowledge from scratch — the evidence is sufficient.

The shicho finding — that ladders are learned late — is particularly striking but is presented as a single observation without systematic analysis of why this order occurs. It would have been informative to see a more systematic mapping of when different Go concepts emerge and what drives the ordering.

Overall experimental strengths:

Clean baselines. The paper compares against every relevant prior version of AlphaGo, as well as a supervised learning baseline with identical architecture, and traditional Go programs.
Hardware transparency. Computational resources are explicitly reported for each system, and the results show that AlphaGo Zero's advantage is not due to having more compute.
Consistent evaluation conditions. Time controls, hardware, and simulation budgets are specified and (where possible) matched across comparisons.
Multiple metrics. Elo, move prediction accuracy, value prediction error, and qualitative game analysis provide complementary views of the system's capabilities.
Training dynamics analysis. The learning curves (Figures 3a, 6a) demonstrate stability and continued improvement, addressing prior concerns about self-play oscillation.
The evaluator mechanism provides built-in statistical rigor. The 400-game matches with 55% threshold for promotion are a form of continual validation that prevents overfitting to stale data.

Overall experimental weaknesses and missing analyses:

No sensitivity analysis on key hyperparameters. The choice of 1,600 MCTS simulations, the 55% evaluator threshold, the training data window of 500K games, the Dirichlet noise parameters, and the temperature schedule are all presented as fixed values without ablations showing how sensitive results are to these choices. This is partly understandable given the computational expense of full training runs, but it limits understanding of which design choices are critical vs. incidental.
Single domain. All results are for Go. The paper argues (persuasively) that Go is an ideal testbed because of its complexity and the prior failure of pure RL approaches, but the claims about "tabula rasa learning of superhuman proficiency in challenging domains" are demonstrated only for this one domain. The subsequent AlphaZero work (Silver et al., 2018) addressed this by extending to chess and shogi, but that evidence is not in this paper.
The 20-block → 40-block scaling is only partially explored. The paper presents two training runs (3 days vs. 40 days) but doesn't systematically explore the scaling relationship between network size, training time, and performance. Would a 60-block network trained for 80 days continue to improve? Is there a point of diminishing returns? The learning curve in Figure 6a appears to still be rising slowly at 40 days, but the asymptotic behavior is unclear.
No analysis of computational efficiency vs. prior systems in FLOPs terms. The paper reports hardware (TPU counts, single vs. distributed) but doesn't provide a FLOPs-matched comparison between AlphaGo Zero's training and AlphaGo Lee's training. It would be informative to know whether AlphaGo Zero achieves superior performance with more, less, or comparable total computation. The single-machine vs. distributed comparison is suggestive (AlphaGo Zero uses a fraction of the hardware) but incomplete.
Self-play Elo inflation concern. The paper anchors Elo ratings to human matches to mitigate self-play bias, but the Elo ratings during training (Figures 3a, 6a) are computed from a closed pool of AlphaGo Zero iterations playing each other. These internal ratings may not be directly comparable to the anchored tournament ratings in Figure 6b. The paper does not discuss the magnitude of this potential discrepancy, though the inclusion of anchored baseline evaluations provides some calibration.
The 100–0 result may not fully reflect AlphaGo Lee's capabilities under different conditions. AlphaGo Lee was designed for and evaluated under specific match conditions. The paper matches those conditions, so this is a fair comparison. However, the 100–0 scoreline — a complete sweep — suggests that AlphaGo Zero is so much stronger that the match may not have been a competitive contest, raising the question of whether an even stronger human-data-based system (perhaps one incorporating the architectural improvements from AlphaGo Zero while retaining human data initialization) would close the gap. The AlphaGo Master comparison partly addresses this, but AlphaGo Master already incorporates the new architecture and algorithm, making it not a pure test of "human data vs. no human data."
Move prediction accuracy on professional moves decreases late in RL training (Figure 3b). The paper interprets this as evidence that the system is discovering non-human strategies. An alternative interpretation is that the network is overfitting to its self-play distribution and losing generalization to human-style positions. The fact that playing strength continues to improve argues against simple overfitting, but without evaluating against a broader set of human players or positions, it's possible the system is becoming narrowly superhuman against itself while developing blind spots that a different style of play could exploit. The 89–11 victory against AlphaGo Master (which was trained with human data and uses rollouts) provides some evidence against this concern, but the possibility of self-play specialization is inherent to the approach and not fully addressed.

Missing experiments that would have strengthened the paper:

Ablation on the policy iteration loop: What happens if you train without the evaluator (always using the latest checkpoint)? This would directly test the claim that the evaluator prevents oscillation.
Ablation on MCTS simulation budget: How does playing strength scale with simulation count at different training stages? This would reveal how much the network's improving prior reduces the need for search.
Comparison against the strongest possible human-data baseline: Train a network with the dual-res architecture, initialized by supervised learning and then fine-tuned with RL, using rollouts — i.e., AlphaGo Master's approach but starting from the improved architecture. This would test whether human data is harmful or merely unnecessary when combined with the architectural improvements.
Out-of-distribution evaluation: Test AlphaGo Zero against human players or human-trained systems on positions or opening strategies that are rare in AlphaGo Zero's self-play distribution, to test for blind spots from self-play specialization.
Transfer learning: After training on 19×19 Go, test the same network on 9×9 or 13×13 Go to assess whether the learned knowledge generalizes across board sizes, which would be evidence for abstract understanding rather than pattern matching at a specific scale.

6. Limitations and Trade-offs

The Tabula Rasa Claim Depends on a Specific Definition of "No Domain Knowledge" That Excludes Architectural Priors

The assumption or constraint. The paper's central claim is that AlphaGo Zero learns "without human data, guidance or domain knowledge beyond game rules" and "tabula rasa." The Methods section ("Domain knowledge") carefully enumerates four forms of domain knowledge the system does use: perfect knowledge of the game rules, Tromp-Taylor scoring, grid-structured input representation as a 19×19 image, and rotation/reflection invariance. The paper argues these are minimal — the rules are necessary to simulate the game at all, Tromp-Taylor scoring is chosen for mathematical well-definedness rather than Go-specific reasons, and the convolutional architecture and symmetry augmentation are weak geometric priors rather than Go expertise.

However, this framing elides a deeper question: how much work are the architectural priors actually doing? Convolutional neural networks with local receptive fields encode a strong assumption about the nature of the problem — that nearby inputs are related and distant inputs are largely independent, and that the same local patterns are meaningful regardless of where they appear (translation invariance). For Go, this is an excellent prior: adjacent intersections genuinely interact through stone connectivity, captures, and territory formation, while the strategic significance of a pattern rarely depends on its absolute board coordinates (corner play is different from center play, but the board is symmetric). The residual architecture — 20 or 40 blocks deep — encodes the assumption that very deep hierarchical feature extraction is necessary, which was validated by the ~600 Elo improvement over the 12-layer convolutional architecture (Figure 4). The paper frames these as generic deep learning choices, but they encode substantial structural knowledge about the problem domain: Go is a spatially structured, translation-invariant, hierarchically compositional domain. A different domain lacking these properties — one where long-range dependencies dominate, or where absolute position is critical, or where shallow pattern matching suffices — would not benefit from the same architectural priors, and the "tabula rasa" claim would not transfer.

The consequence. The claim that AlphaGo Zero uses "no domain knowledge beyond game rules" is technically true as stated but potentially misleading about the generality of the result. The architectural choices — convolutional residual networks with dihedral symmetry augmentation — were made by human engineers who understood the structure of Go (grid-based board, local interactions, rotation/reflection symmetry) and selected an architecture matched to that structure. A truly domain-agnostic learning system would need to discover the spatial structure and symmetries of the problem from raw experience, which is a substantially harder problem. The paper does not test whether the same architecture would work for domains with different structural properties, or whether the learning algorithm would succeed with a more generic architecture (e.g., a fully connected network, or a transformer without positional encodings).

This matters for the paper's broader philosophical claim — that the results demonstrate a path toward general AI that learns "tabula rasa" in any domain. The system succeeds in Go partly because Go's structure is well-matched to convolutional architectures (a fact that was known and exploited by human designers), not because the learning algorithm is universally domain-agnostic. For a domain with fundamentally different structure — natural language understanding, causal reasoning, multi-agent negotiation — the same "minimal domain knowledge" approach would require different architectural priors, and identifying the right priors is itself a form of domain knowledge injection.

What evidence exists in the paper. The paper does not ablate the architectural choices systematically to determine how much each prior contributes. There is no experiment with a fully connected network, a transformer architecture, or a network without symmetry augmentation. The architecture comparison in Figure 4 compares residual vs. convolutional and dual vs. separate heads, but all architectures are convolutional with grid-structured input and symmetry augmentation. The paper does provide one indirect piece of evidence about the importance of architecture: the ~1,200 Elo gap between sep-conv (AlphaGo Lee's architecture, 12 layers convolutional) and dual-res (AlphaGo Zero's architecture, 20 blocks residual) demonstrates that architectural choices dramatically affect performance. But this doesn't tell us whether the architectural priors are merely helpful (accelerating learning that would eventually happen anyway) or essential (without them, tabula rasa learning would fail entirely).

Mitigation status. The paper is transparent about the four items of domain knowledge it uses, and the enumeration in the Methods section is explicit and fair. The authors do not claim to have eliminated all priors — they claim to have eliminated Go-specific knowledge beyond game rules, which is a more defensible and precise claim. The paper acknowledges in the Methods that the input features are "structured as a 19×19 image; that is, the neural network architecture is matched to the grid-structure of the board" and that "the rules of Go are invariant under rotation and reflection; this knowledge has been used." This is honest disclosure, but it does not resolve the underlying tension between the "tabula rasa" framing and the reality of architecture engineering. The paper does not suggest future work on architecture-agnostic learning or on automatically discovering problem structure.

Hardware and Computation Requirements Are Substantial, and the Training Cost Is Not Characterized Relative to Baselines

The assumption or constraint. The paper reports hardware usage qualitatively: AlphaGo Zero uses "a single machine with 4 TPUs" for playing, compared to AlphaGo Lee's "distributed over many machines using 48 TPUs" and AlphaGo Fan's "distributed over many machines using 176 GPUs." Training uses "64 GPU workers and 19 CPU parameter servers" on Google Cloud. The 20-block version generates 4.9 million self-play games over 3 days and processes 700,000 mini-batches of 2,048 positions each (approximately 1.4 billion training positions). The 40-block version generates 29 million self-play games over 40 days and processes 3.1 million mini-batches (approximately 6.3 billion training positions).

These numbers are reported but never converted into a standardized compute metric (e.g., total FLOPs, TPU-hours, or GPU-days). The paper does not provide a FLOPs-matched comparison between AlphaGo Zero's total training compute and that of AlphaGo Lee, AlphaGo Master, or the supervised learning baseline. The single-machine vs. distributed-hardware comparison during play suggests efficiency, but playing is only a small fraction of total compute — the vast majority is spent on self-play generation (25,000 games per iteration, each involving 1,600 MCTS simulations per move with neural network evaluations at every leaf node, across hundreds of moves per game) and neural network training (distributed synchronous SGD across 64 GPU workers). Without quantifying this total cost, it is impossible to determine whether AlphaGo Zero achieves superior performance because of a better algorithm or because it simply consumed more total computation than prior systems.

The consequence. A practitioner deciding whether to adopt this approach needs to know: how much does it cost? If AlphaGo Zero required 10× or 100× more total FLOPs than AlphaGo Lee's training pipeline (combining supervised pretraining on 30 million human positions, RL fine-tuning, and the training of separate policy and value networks), then the claimed advantages of "no human data" come with a substantial computational premium that may not be worthwhile for domains where human data is available. Conversely, if AlphaGo Zero is actually more compute-efficient than the human-data-based pipeline, that's an important finding that the paper should quantify. The absence of standardized compute accounting makes the paper's efficiency claims (e.g., "a pure reinforcement learning approach requires just a few more hours to train") difficult to evaluate — a "few more hours" on what hardware configuration, compared to what baseline training regime?

This is particularly important because the self-play RL approach involves a fundamental computational overhead: every position in the training data is generated by running an expensive MCTS (1,600 neural network evaluations per move, plus tree operations), whereas supervised learning uses pre-existing human game records that require no search. The self-play generation cost — 4.9 million or 29 million games, each involving hundreds of MCTS searches — likely dominates the total compute budget. If human game records are available (as they are for Go, chess, and many other domains), the supervised pretraining approach amortizes the cost of data generation across many training runs (the KGS dataset can be downloaded once and reused). The self-play approach pays this cost anew for every training run. The paper provides no analysis of whether the superior asymptotic performance justifies this recurring cost.

What evidence exists in the paper. The paper reports wall-clock training time (3 days for 20-block, 40 days for 40-block) and hardware configuration (64 GPU workers, 19 CPU parameter servers), but not total FLOPs or cost. There is no FLOPs-matched comparison against prior systems. The single-machine vs. distributed-hardware comparison applies only to playing (inference), not to training. The paper does not report the training hardware or duration for AlphaGo Lee, AlphaGo Master, or the supervised learning baseline, making any efficiency comparison impossible from the data provided.

Mitigation status. The paper does not address this limitation. The focus is on demonstrating the possibility and performance of tabula rasa learning, not on characterizing its computational efficiency relative to alternatives. The authors likely viewed the compute cost as secondary to the conceptual contribution (proving that pure RL can achieve superhuman performance). However, for practitioners and for the broader claim that this approach is preferable to human-data-based methods, the missing efficiency analysis is a significant gap. The paper does not suggest future work on reducing the computational cost of self-play generation or on amortizing search costs across training iterations.

The Approach Has No Demonstrated Path for Scaling to Domains Without a Perfect Simulator and Well-Defined Reward Signal

The assumption or constraint. AlphaGo Zero's learning loop depends critically on two things that Go provides and that many real-world domains do not: a perfect, fast simulator of the environment (the game rules, which allow the system to instantly determine the state resulting from any move, and to generate unlimited self-play games without external interaction) and a clean, unambiguous reward signal (the game outcome: win = +1, loss = −1, known with certainty at the terminal state). These two properties are what enable the entire training pipeline: the MCTS can simulate millions of lookahead trajectories because the environment model is perfect and instantaneous; the value targets are noise-free because the game rules determine a unique winner; and self-play can proceed indefinitely without any external feedback because the rules fully specify legal play and scoring.

The paper does not claim to work without these properties. It explicitly lists "perfect knowledge of the game rules" as the first item of domain knowledge (Methods, "Domain knowledge"). But the paper also frames its contribution in sweeping terms: "A long-standing goal of artificial intelligence is an algorithm that learns, tabula rasa, superhuman proficiency in challenging domains" (abstract). The gap between Go (perfect simulator, clean reward) and most "challenging domains" of practical interest is substantial and unaddressed.

The consequence. In any domain where the environment dynamics are unknown, stochastic, or expensive to simulate, the MCTS-based policy improvement loop breaks down. You cannot simulate lookahead trajectories through a model you don't have. In any domain where the reward signal is sparse, noisy, delayed, or multi-dimensional, the value training signal degrades. If wins and losses are not clearly defined — consider dialogue systems, robotic manipulation, scientific discovery, or strategic planning — the binary win/loss framework does not apply, and the entire policy evaluation mechanism (regressing v toward z) has no obvious analogue.

Domain examples where the approach would fail without substantial modification:

Robotics: no perfect simulator of physical dynamics (sim-to-real gap), rewards may be continuous and multi-objective (energy efficiency + task completion + safety), and self-play against oneself is not meaningful for most manipulation tasks.
Autonomous driving: environment dynamics are complex and other agents are not playing a zero-sum game. Self-play would require a realistic multi-agent traffic simulator that doesn't exist.
Medical treatment planning: the "simulator" is a human patient; the "reward" (treatment outcome) is observed weeks or months later and is confounded by countless unobserved variables. You cannot run 29 million self-play episodes.
Dialogue and negotiation: no well-defined reward (what is a "win" in a conversation?), no perfect simulator of human responses, and self-play against oneself may not produce realistic interaction dynamics.
Scientific discovery: the "game" is against nature, not an opponent; the reward is whether a hypothesis is true, which requires physical experiments, not simulation.

The paper's approach transfers cleanly to other two-player zero-sum perfect-information games (chess, shogi, as the subsequent AlphaZero work demonstrated), but the claim that it demonstrates a path to "challenging domains" generally depends on whether you view "challenging two-player perfect-information board games" as representative of the broader class of challenging AI problems. The paper implicitly argues they are — citing the historical transfer of game-playing AI techniques to robotics, industrial control, and recommendation systems (Methods, "Self-play reinforcement learning in games") — but the transfer in those cases required substantial adaptation that the paper does not discuss or prototype.

What evidence exists in the paper. The paper provides no experiments outside of Go, no analysis of what happens when the simulator has errors or the rewards are noisy, and no discussion of how the approach would be adapted to domains lacking these properties. The Methods section's review of prior self-play RL in other games (chess, checkers, backgammon, Scrabble, poker) indirectly supports the claim that self-play can work across game domains, but all of these share the perfect-simulator property. The paper's citation of applications in robotics, industrial control, and recommendation systems (references 60–65) is a gesture toward broader applicability but does not demonstrate that the specific AlphaGo Zero algorithm would transfer.

Mitigation status. The paper does not address this limitation. It does not suggest future work on model-based RL with learned (imperfect) environment models, on handling noisy or delayed rewards, or on extending the approach beyond two-player zero-sum games. The limitation is implicit in the choice of domain (Go), and the paper is careful to restrict its explicit claims to Go performance. However, the broader rhetoric — "a long-standing goal of artificial intelligence is an algorithm that learns, tabula rasa, superhuman proficiency in challenging domains" — invites readers to extrapolate beyond the demonstrated domain, and the paper provides no caution about the simulator/reward assumptions that make the extrapolation non-trivial.

The 20-Block Results Show Rapid Initial Progress, But the Paper Provides No Systematic Scaling Analysis to Predict Performance Under Different Resource Constraints

The assumption or constraint. The paper presents two training runs: a 20-block network trained for 3 days (4.9 million self-play games) and a 40-block network trained for 40 days (29 million self-play games). The learning curves (Figures 3a, 6a) show smooth improvement in both cases, with the 40-block network continuing to improve slowly at day 40. These two data points establish that the approach scales with network size and training duration, but they do not constitute a scaling law — there is no systematic sweep of model sizes, training durations, or compute budgets to characterize the relationship between resources and performance.

The consequence. A practitioner cannot answer basic capacity-planning questions from the paper's data: How much performance would a 10-block network achieve after 10 days? How much training time would be needed to reach a target Elo of 4,000 with a 20-block network? Is there a point of diminishing returns where additional training yields negligible improvement? Does the benefit of doubling network depth depend on training duration (i.e., do larger networks need longer training to realize their advantage, or are they immediately better)? Without answers to these questions, it is impossible to determine whether the 40-block, 40-day run was compute-optimal or whether a different allocation (e.g., 30 blocks for 60 days, or 60 blocks for 20 days) would have yielded better performance for the same total compute.

This is particularly relevant for practitioners considering using the approach in other domains, where the right network size and training budget are unknown and running multiple full-scale experiments to determine them would be prohibitively expensive. In the absence of scaling laws, the approach requires expensive trial-and-error to tune to a new domain.

The paper also provides no analysis of how performance scales with inference-time compute (MCTS simulation budget). All evaluations use 1,600 simulations per move, except the 5-second-per-move tournament (Figure 6b) where simulations are whatever fits in 5 seconds. The 2,130 Elo gap between the raw network (0 MCTS simulations) and AlphaGo Zero (~1,600 simulations, Figures 6b) demonstrates that search dramatically improves performance, but we don't know the shape of this curve: does performance improve linearly with the log of simulation count? Does it plateau? Could 6,400 simulations yield further substantial gains? This missing analysis limits understanding of the fundamental tradeoff between network quality and search depth.

What evidence exists in the paper. The paper provides exactly two data points for the network-size/training-duration relationship: 20 blocks / 3 days → ~4,500 Elo, and 40 blocks / 40 days → 5,185 Elo. These numbers are not directly comparable because the Elo ratings come from different evaluation pools. The 40-block curve in Figure 6a shows the learning trajectory but does not compare against smaller networks at matched training durations. The paper mentions in the Methods that MCTS search parameters were "selected by Gaussian process optimization" for both the 20-block and 40-block runs, and that for the larger run "MCTS search parameters were re-optimized using the neural network trained in the smaller run." This suggests that hyperparameters are not transferable across scales, which further complicates any attempt to extrapolate from the reported results.

Mitigation status. The paper does not address this limitation. It does not claim to provide scaling laws, and the two-run presentation is adequate for the paper's primary purpose (demonstrating that tabula rasa RL works and can achieve superhuman performance). However, the absence of scaling analysis is a significant gap for anyone seeking to apply the method to a new domain or to understand the resource requirements for achieving a target performance level. The paper does not suggest future work on characterizing scaling relationships.

Self-Play Evaluation Introduces Systematic Measurement Biases That Are Only Partially Mitigated by Human Anchoring

The assumption or constraint. The Elo ratings reported for AlphaGo Zero's training progress (Figures 3a, 6a) and for comparison between AlphaGo Zero and its predecessors (Figure 6b) are computed from games within a relatively closed pool of players. During training, the Elo of each iteration α_{θ_i} is computed from evaluation games against other iterations and against baseline players with anchored Elo ratings. The final tournament (Figure 6b) includes AlphaGo Zero, AlphaGo Master, AlphaGo Lee, AlphaGo Fan, Crazy Stone, Pachi, GnuGo, and the raw network. The paper states that "the Elo ratings of AlphaGo Fan, Crazy Stone, Pachi and GnuGo were anchored to the tournament values from previous work, and correspond to the players reported in that work. The results of the matches of AlphaGo Fan against Fan Hui and AlphaGo Lee against Lee Sedol were also included to ground the scale to human references, as otherwise the Elo ratings of AlphaGo are unrealistically high due to self-play bias."

This anchoring is acknowledged as necessary because self-play evaluation pools produce inflated Elo ratings. When all players in a pool share similar weaknesses or blind spots (because they are iterations of the same training run, or variants of the same architecture), the Elo system cannot detect these shared limitations — it only measures relative differences within the pool. A system that is excellent against itself may have systematic vulnerabilities that a differently-trained opponent (e.g., a human with a contrasting style) could exploit, but these vulnerabilities never manifest in the Elo computation because no such opponent is in the pool.

The consequence. The absolute Elo values reported — 5,185 for AlphaGo Zero, 4,858 for AlphaGo Master, 3,739 for AlphaGo Lee — may not be directly comparable to Elo ratings from human tournament play or from a more diverse pool of AI systems. While the human-match anchoring (Fan Hui, Lee Sedol) provides some calibration, the Elo scale is not linear in an absolute sense — a 200-point gap means a 75% win probability within the pool in which the Elo was computed, but this relationship may not hold when comparing across pools with different playing styles or capability profiles.

This is particularly concerning for the claim that AlphaGo Zero's raw network (no search) achieves 3,055 Elo. This network was evaluated within the same pool as AlphaGo Zero's search-based players, which share its architecture and training distribution. A 3,055 Elo within this pool does not necessarily mean the raw network would achieve a 3,055-level performance against a diverse set of human players or differently-architected AIs — its apparent strength may be partly an artifact of being evaluated against systems that share its representational biases.

More broadly, the entire training process is a closed loop: the system plays against itself, learns from self-play outcomes, and is evaluated against itself (and its predecessors). While the evaluator mechanism (promoting only checkpoints that win >55% against the current best) ensures monotonic improvement within the self-play distribution, it does not guarantee improvement against out-of-distribution opponents or strategies. The paper provides evidence of generalization (strong performance against AlphaGo Master, which was trained with human data and uses rollouts), but the 89–11 result is still between systems that ultimately share the same core algorithm and architecture. There is no evaluation against a system with a fundamentally different approach to Go (e.g., a pure MCTS program with no neural network, or a symbolic AI system using explicit pattern libraries) that would test whether AlphaGo Zero has developed genuine general Go understanding or merely highly effective strategies against neural-network-based opponents.

What evidence exists in the paper. The paper is transparent about the self-play bias issue: it explicitly states that "the results of the matches of AlphaGo Fan against Fan Hui and AlphaGo Lee against Lee Sedol were also included to ground the scale to human references, as otherwise the Elo ratings of AlphaGo are unrealistically high due to self-play bias." This acknowledgment demonstrates awareness of the problem. The inclusion of diverse baselines (Crazy Stone, Pachi, GnuGo, which use different underlying algorithms) partly mitigates the concern by expanding the evaluation pool. The 100–0 victory against AlphaGo Lee and the 89–11 victory against AlphaGo Master are head-to-head results that do not suffer from Elo pool inflation — they are direct measurements of relative strength.

However, the evidence for out-of-distribution generalization is limited to these match results. There is no systematic evaluation against a diverse set of playing styles (e.g., different openings, different strategic philosophies) to test for blind spots. There is no analysis of whether AlphaGo Zero's self-play training distribution covers the space of possible Go strategies broadly enough to ensure robustness, or whether the system might have undiscovered vulnerabilities that a different style of play would expose.

Mitigation status. The paper partially mitigates this limitation through human-match anchoring and inclusion of non-neural-network baselines, but the mitigation is incomplete. The core issue — that self-play training and evaluation occur in a closed distribution that may miss important regions of strategy space — is inherent to the approach and not fully addressed. The paper does not discuss the possibility of self-play specialization leading to fragile strategies, nor does it suggest methods for detecting or preventing such fragility (e.g., training against a diverse league of opponents rather than only the current best self, or adversarial testing to discover blind spots). The subsequent AlphaStar work (Vinyals et al., 2019) on StarCraft II addressed this issue explicitly through a league-based training approach, suggesting the authors were aware of the limitation even if it wasn't addressed in this paper.

The Knowledge Discovery Analysis Is Qualitative and Potentially Overstates the System's Independence from Human Concepts

The assumption or constraint. Section 5 and Figure 5 present AlphaGo Zero's learning trajectory as a process of independent discovery: the system "discovered a remarkable level of Go knowledge during its self-play training process," including "fundamental elements of human Go knowledge" and "non-standard strategies beyond the scope of traditional Go knowledge." The evidence consists of: (1) timelines showing when specific joseki (corner sequences) first appeared in self-play games (Figure 5a), (2) frequency analysis of preferred joseki at different training stages (Figure 5b, Extended Data Figures 2–3), and (3) example game records at different training stages annotated with qualitative descriptions of the concepts being demonstrated (Figure 5c, Extended Data Figures 4–5).

This analysis is presented as objective measurement of knowledge acquisition, but it depends on a significant unstated assumption: that the human-identified patterns and concepts are the right vocabulary for describing what AlphaGo Zero has learned. The analysis identifies when the system plays moves that correspond to human-named joseki, or when its play exhibits characteristics interpretable as "life-and-death," "influence," "territory," etc. But these are human categories projected onto the system's behavior. It is possible — and indeed likely, given the paper's own finding that the self-play network's move predictions diverge from human moves (Figure 3b) — that AlphaGo Zero has developed internal representations and strategies that do not cleanly map onto human Go concepts. The joseki analysis captures only the subset of AlphaGo Zero's strategic repertoire that happens to be recognizable through a human lens.

The consequence. The qualitative knowledge discovery analysis may understate how alien AlphaGo Zero's understanding actually is, creating a misleading impression that the system "reinvented human Go" when it may have discovered something substantially different that only partially overlaps with human knowledge. The claim that the system discovered "novel strategies that provide new insights into the oldest of games" (Conclusion) is based on identifying joseki variations that don't appear in standard human references — but the analysis doesn't systematically characterize how much of AlphaGo Zero's play is unrecognizable in human terms, or whether the "new insights" are genuinely novel strategies or just variations that humans have explored but not documented.

Conversely, the analysis may overstate the independence of discovery by implying that AlphaGo Zero independently arrived at concepts that humans spent millennia developing. An alternative interpretation — not addressed in the paper — is that the structure of Go strongly constrains what good play looks like, such that any sufficiently strong optimization process (whether human culture or neural network training) will converge to similar patterns. If corner joseki are largely determined by the geometry of the corner and the rules of capture, then "discovering" them is less a feat of creative intelligence and more a consequence of optimizing a well-defined objective in a constrained space. The paper's narrative of independent discovery would be strengthened by showing that AlphaGo Zero found different but equally effective approaches to corners, rather than converging to the same joseki that humans use.

What evidence exists in the paper. The move prediction accuracy metric (Figure 3b) provides indirect evidence that AlphaGo Zero's play diverges from human patterns: the self-play network's accuracy on predicting professional moves decreases late in training, even as its playing strength increases. This is the paper's clearest quantitative evidence for genuine strategic divergence. However, the knowledge discovery analysis (Figure 5) primarily highlights convergence — the fact that AlphaGo Zero found human joseki — rather than quantifying the extent of divergence. The "new joseki variations" in Figure 5b are presented as specific examples, but we don't know how many other novel patterns exist in AlphaGo Zero's play, how different they are from human play, or whether they represent marginal variations or qualitatively different strategic principles. The paper provides no systematic comparison of AlphaGo Zero's move distribution against the human move distribution to characterize the overlap.

Mitigation status. The paper does not address this limitation. The knowledge discovery analysis is presented as corroborating evidence for the claim of genuine learning, and the authors likely viewed the convergence to human patterns as impressive and surprising (hence worthy of highlighting) rather than as a potential confound. The possibility that the apparent "discovery" of human concepts reflects the constrained nature of Go strategy rather than independent reinvention is not discussed. The paper does not suggest future work on characterizing the overlap and divergence between learned and human strategies, or on developing methods to interpret neural network representations without projecting human categories onto them.

7. Implications and Future Directions

How This Work Changes the Landscape

AlphaGo Zero fundamentally shifts the conversation about how to build expert-level AI systems. Before this paper, the dominant narrative — which the original AlphaGo itself helped establish — was that human expertise provides an essential bootstrap: you collect large datasets of human behavior, train models to imitate that behavior, and then optionally refine through reinforcement learning. This was the recipe not just for Go but for machine translation, speech recognition, image captioning, and most other domains where human performance was the target. Human data was the initial scaffolding that made learning possible.

AlphaGo Zero demolishes the necessity of that scaffolding. The paper's most disruptive finding is not that reinforcement learning can work from scratch — TD-Gammon proved that in 1994 — but that in an extraordinarily complex domain requiring sophisticated lookahead, the human-data-free approach substantially outperforms the human-data-based approach, and that human data not only fails to help but actually imposes a ceiling. Figure 3 makes this concrete: the human-trained network is better at predicting human moves (Figure 3b) and initially better at evaluating positions (Figure 3c), yet the self-play network overtakes it within 24 hours and keeps improving while the human-trained network plateaus. The 100–0 result against AlphaGo Lee — which was trained with human data over months, distributed across 48 TPUs, and had already defeated one of the strongest human players in history — is the empirical stake in the ground.

This is a genuine paradigm shift in how to think about the role of human knowledge in AI. Before AlphaGo Zero, the default assumption was that human data was valuable when available — you would always want to include it if you could, because it provides a warm start and constrains the search space. After AlphaGo Zero, the calculus inverts: human data becomes a potential liability, because it anchors the system to the human distribution and prevents the discovery of strategies that humans haven't found. The paper's finding that AlphaGo Zero discovered and preferred joseki variations unknown to professional play (Figure 5b, Extended Data Figure 3) is the positive evidence: the system went beyond human knowledge, not just to it. The finding that shicho (ladder capture sequences) — "one of the first elements of Go knowledge learned by humans — were only understood by AlphaGo Zero much later in training" is the negative evidence: the system's learning trajectory was genuinely different from the human one, discovering concepts in a different order based on what provided useful reinforcement signal rather than what humans explicitly teach.

The paper also resolves a standing tension in the reinforcement learning literature about the stability of self-play. Prior work (Laurent et al., 2011; Foerster et al., 2017; Heinrich and Silver, 2016) had documented oscillations, catastrophic forgetting, and failure to converge in multi-agent self-play settings. The concern was that without an external anchor (like human data or a fixed opponent distribution), self-play systems would chase their own tails: a new strategy beats the old one, the system adapts to the new strategy, a counter-strategy emerges, and the cycle never stabilizes. AlphaGo Zero's smooth learning curves (Figures 3a, 6a) demonstrate that these concerns, while legitimate for prior methods, are not fundamental to self-play — they are artifacts of specific algorithmic choices. The paper's architecture (the evaluator enforcing monotonic improvement, the windowed training data tracking the current best player, the policy-iteration framing where MCTS serves as an improvement operator rather than just a search procedure) collectively solves the stability problem. This is methodologically significant because it means self-play RL can be reliable and predictable, not fragile and oscillatory — which matters enormously for anyone considering deploying it in a new domain.

The paper also redefines the relationship between search and learning. Before AlphaGo Zero, MCTS was primarily a test-time inference technique: you trained a policy and value network, then used them to guide search at evaluation time. The search made the network stronger but didn't feed back to improve the network. AlphaGo Zero closes that loop, treating MCTS as a training-time policy improvement operator. This reframing — search as a component of the learning algorithm rather than an add-on to a trained model — has conceptual implications far beyond Go. It suggests that any domain where search is possible (where you can simulate the consequences of actions, even approximately) can benefit from integrating search into the training loop, using the search to generate improved targets and then compressing those improvements back into the model. This is the idea that AlphaZero subsequently applied to chess and shogi, and that later work is exploring in domains like program synthesis, theorem proving, and drug discovery — anywhere you can define a forward model and evaluate outcomes.

Research directions that become more attractive after this paper include: pure reinforcement learning for domains where human data exists but may be limiting (chess, poker, real-time strategy games, robotic control in simulation), integration of search-based improvement operators into RL training loops for any domain with a forward model, and the study of what specific properties of human data cause the ceiling effect — is it that humans systematically underexplore certain strategies, or that human move distributions are poorly calibrated for the actual value function, or something else?

Research directions that become less attractive include: efforts to build superhuman game-playing AI primarily through larger supervised datasets of human play (the ceiling is real), and efforts to engineer increasingly sophisticated rollout policies or handcrafted evaluation features (AlphaGo Zero proves they're unnecessary if the learned evaluator is good enough).

Follow-Up Research This Work Enables

1. Systematic characterization of the "human data ceiling" — does human data hurt, or merely fail to help when RL is given enough compute? The paper shows that self-play RL outperforms supervised learning on human data (Figure 3a), and that the self-play network achieves worse human move prediction accuracy but better playing strength (Figure 3b). But the comparison is between two extremes: pure supervised learning on KGS data vs. pure RL from self-play. The most informative experiment would be an interpolation: initialize the network with supervised learning on human data, then switch to the self-play RL pipeline (identical to AlphaGo Zero's training loop but with a warm start). Does the human initialization accelerate early learning and then get "unlearned" as the system discovers better strategies, or does it create a persistent bias that limits asymptotic performance? If the former, human data is a useful accelerator; if the latter, the ceiling effect is robust and human data should be avoided even when available. The paper's AlphaGo Master baseline (which uses human data initialization plus the new architecture and algorithm) and its 89–11 loss to AlphaGo Zero is partial evidence for the "persistent bias" interpretation, but a controlled experiment with identical architecture and training budget would be more definitive. A strong follow-up would train multiple AlphaGo Zero instances with varying amounts of human-data pretraining (0%, 10%, 100% of the KGS dataset) and compare both learning curves and asymptotic Elo at matched total compute.

2. Scaling laws for self-play RL with search — how does performance scale with network size, training duration, and MCTS simulation budget? The paper provides exactly two data points: 20 blocks / 3 days / ~4,500 Elo and 40 blocks / 40 days / 5,185 Elo. This is insufficient to predict what a 60-block network trained for 100 days would achieve, or whether a 10-block network trained for 1 day would already be superhuman (making the approach practical for smaller teams). A systematic scaling study would sweep network depth (e.g., 5, 10, 20, 40, 80 residual blocks), training duration (1, 3, 10, 40, 100 days), and inference-time simulation budget (100, 400, 1,600, 6,400, 25,600 simulations per move) to characterize the relationships. Key questions: Does the benefit of increased network depth depend on training duration (do larger networks need longer training to surpass smaller ones)? Does the optimal MCTS simulation budget grow with network quality, or does a better prior reduce the need for search? Is there a point of diminishing returns where additional training compute yields negligible Elo improvement? The paper's finding that the evaluator automatically selects the best checkpoint enables this experiment: you can run many configurations in parallel and let the evaluator determine when each has peaked. This would produce the equivalent of Chinchilla scaling laws (Hoffmann et al., 2022) but for self-play RL with search — essential knowledge for anyone applying the method to a new domain.

3. Where do the 2,130 Elo from MCTS search actually come from? The raw AlphaGo Zero network (no search) achieves 3,055 Elo; with ~1,600 MCTS simulations, the same network reaches ~5,185 Elo (Figure 6b). This enormous gap demonstrates that search provides something the feedforward network fundamentally lacks — but what, exactly? Is it primarily about deep tactical calculation (reading out capture sequences that span 10+ moves), about precise endgame evaluation (comparing subtle territorial boundaries), about exploring counterfactual branches that the policy head undervalues, or about something else? A detailed analysis would measure the contribution of search as a function of game phase (opening, middle game, endgame), position type (tactical capturing races vs. strategic territorial decisions), and move number. You could compare the raw network's move selection against the search's move selection on a large corpus of positions, categorizing disagreements by whether they involve tactical sequences beyond some depth horizon, endgame counting, or strategic judgment. If search primarily improves tactical calculation, that suggests a path to stronger raw networks by training on more tactical positions. If search primarily improves endgame precision, that suggests training on more endgame-heavy self-play. Understanding why search helps is essential for designing architectures that can internalize more of the search's capability — potentially reducing or eliminating the need for expensive lookahead at inference time.

4. Can the architecture's implicit priors be weakened further without breaking the learning loop? The paper enumerates four forms of domain knowledge (Methods, "Domain knowledge") and argues they are minimal. But three of them — grid-structured convolutional input, rotation/reflection symmetry augmentation, and residual architecture depth — encode structural assumptions about Go (spatial locality, translation invariance, hierarchical composition) that might be doing substantial work. A stress-test would replace the convolutional residual network with a more generic architecture — a transformer with learned positional encodings, or a graph neural network operating on the board connectivity graph — that makes weaker assumptions about the problem structure, and test whether the same self-play RL pipeline still achieves superhuman performance. If it does, the "no domain knowledge" claim is substantially strengthened and the approach becomes more credible as a general recipe for arbitrary domains. If it doesn't — if the convolutional prior is essential to bootstrapping from random play — then the tabula rasa claim has an important boundary condition: you need an architecture that is at least approximately matched to the problem structure, which requires human engineering judgment (a form of domain knowledge). A parallel stress-test would remove the symmetry augmentation and test whether the system can learn rotation/reflection invariance from data given enough training, or whether the explicit augmentation is necessary. The paper already randomly transforms positions during MCTS evaluation, so removing it from the MCTS while keeping the same network architecture would isolate whether the augmentation is providing a critical exploration benefit or merely accelerating learning.

5. Does self-play training produce fragile strategies that fail against out-of-distribution opponents? The paper acknowledges self-play bias in Elo computation and mitigates it by anchoring to human matches, but a deeper question remains: does AlphaGo Zero's closed self-play training distribution create systematic blind spots that a differently-styled opponent could exploit? The system is evaluated against AlphaGo Master (89–11) and AlphaGo Lee (100–0), which share similar algorithmic foundations despite using human data. A more revealing evaluation would pit AlphaGo Zero against a diverse league of Go-playing agents with fundamentally different architectures and training procedures: pure MCTS with no neural network (using handcrafted heuristics), symbolic AI systems with explicit pattern libraries, adversarially trained opponents designed to exploit known weaknesses of neural network policies, and human professionals with contrasting playing styles. If AlphaGo Zero maintains superhuman performance against all of these, its strategies are genuinely robust. If it performs worse against specific styles — particularly styles that are rare in self-play because AlphaGo Zero itself doesn't generate them — then the self-play approach has a coverage problem that league-based training (as subsequently developed in AlphaStar for StarCraft II) would address. The paper's 100–0 sweep against AlphaGo Lee is suggestive of robustness (AlphaGo Lee has a different architecture and was trained differently), but a single opponent is insufficient to characterize the distribution of vulnerabilities. This experiment would directly test the paper's implicit claim that the self-play distribution is broad enough to cover the space of effective Go strategies.

6. Does policy iteration via MCTS work with learned (imperfect) world models, or does it require a perfect simulator? The paper's approach depends on perfect knowledge of the game rules for MCTS lookahead. In most real-world domains, perfect simulators don't exist — you have at best an approximate learned model of environment dynamics. A critical next step is to test whether the virtuous cycle (network → MCTS → improved policy → better network → better MCTS) survives when the MCTS is operating on a learned world model with some error. You could take a domain with a perfect simulator (like Go) and artificially degrade the simulator: introduce stochastic transitions, add noise to state representations, or replace it entirely with a neural network dynamics model trained on observed transitions. At what level of model error does the policy improvement loop break down? Does the network learn to compensate for model inaccuracies (developing strategies that are robust to the specific errors in the learned model), or does model error compound across the lookahead horizon and produce worthless training targets? This experiment is essential for understanding whether the AlphaGo Zero approach transfers to robotics, autonomous driving, or any domain where the environment must be learned rather than given. The paper's related work section cites applications in robotics and industrial control (references 60–65), but none of those use the search-as-improvement-operator framework — they use simpler forms of RL. Bridging this gap requires understanding how model error interacts with the policy iteration loop.

Practical Applications and Downstream Use Cases

1. Accelerated game balancing and design for complex strategy games. Game developers spend enormous effort balancing complex games — tuning unit strengths, resource costs, and map designs to ensure diverse viable strategies and prevent dominant "solved" metagames. This process typically relies on extensive human playtesting, which is slow, expensive, and limited in coverage (human testers can only play so many games and tend to converge on known strategies). An AlphaGo Zero-style system, trained tabula rasa on a new game's rules, could serve as an automated playtesting engine: after training, the system's discovered strategies (analogous to the joseki analysis in Figure 5) reveal which game elements are over- or under-powered, whether the strategic diversity collapses to a single dominant approach, and whether there exist degenerate strategies that human testers haven't found. The 3-day training time for the 20-block version (achieving superhuman performance) suggests such a system could provide actionable feedback within a week of starting from a new rule set, substantially faster than human playtesting cycles. The key insight from the paper is that the system discovers strategies from first principles rather than imitating human play, meaning it can find exploits and imbalances that humans might never consider — exactly what game designers need to know before release.

2. Data generation for supervised training in domains where RL is impractical but human data is scarce. The paper demonstrates that self-play RL can produce training data (positions, search-informed move recommendations, and outcome labels) of extremely high quality — far beyond what human experts can provide. For domains where running the full RL loop is impractical (no simulator, expensive or irreversible actions) but supervised fine-tuning on expert demonstrations is feasible, the self-play pipeline could serve as a data generation engine running in a separate simulated environment. For example, in robotic manipulation, you could define a simplified simulation of the task (with approximate physics), train an AlphaGo Zero-style system in simulation to superhuman performance, and then use the generated state-action-outcome tuples to train a policy via behavioral cloning or offline RL that transfers to the real robot. The paper provides evidence that this data is higher quality than human demonstrations: the self-play network's move recommendations lead to stronger play than human moves (Figure 3a vs. 3b). The 25,000 self-play games per iteration, each containing hundreds of positions with search-informed policy targets and outcome labels, represent a data generation pipeline that produces millions of high-quality training examples per day — sufficient to train large models even in domains where human data is too scarce or expensive to collect at scale. The key number is the training data window of 500,000 recent games, corresponding to hundreds of millions of labeled positions available for distillation at any time.

3. Automated scientific discovery in domains with well-defined rules and fast simulators. The paper's finding that AlphaGo Zero independently discovered known human Go knowledge (joseki, fuseki, life-and-death) and novel strategies (Figure 5, Extended Data Figures 2–3) from first principles suggests a template for automated discovery in any domain with formal rules and a fast simulator. Protein folding: the rules are physical (energy minimization), the simulator is molecular dynamics (computationally expensive but tractable for small proteins), and the "moves" are amino acid sequence modifications or folding pathway choices. Drug design: the rules are chemical binding affinities, the simulator is docking software, and the "moves" are molecular modifications. Theorem proving: the rules are logical inference, the simulator is a proof checker, and the "moves" are inference steps. In each case, an AlphaGo Zero-style system could be trained tabula rasa to discover strategies (folding motifs, molecular scaffolds, proof techniques) that human experts haven't found, and the pattern of discovering known human knowledge en route to novel discoveries would provide a validation signal: when the system rediscovers established results, you gain confidence that its novel discoveries are meaningful. The critical requirement is a simulator that is sufficiently fast and accurate — the paper's 4.9 million self-play games for the 20-block version corresponds to approximately 5 billion MCTS simulations (4.9M games × ~200 moves/game × ~1,600 simulations/move, i.e., roughly 1.5 trillion position evaluations), so the simulator must support billions of queries at interactive speeds. Domains where simulators already exist at that scale (computational chemistry, formal verification, circuit design) are immediate candidates.

4. Efficient inference-time compute scaling through learned priors. The paper reveals an enormous performance gap between the raw neural network (3,055 Elo) and the MCTS-augmented version (5,185 Elo) in Figure 6b. From an inference-efficiency perspective, this is both a problem and an opportunity. The problem: you need 1,600 neural network evaluations plus tree operations to make each move, making inference computationally expensive. The opportunity: the raw network alone is already at 3,055 Elo — superhuman or near-superhuman relative to most human players and well above all pre-AlphaGo programs (Crazy Stone, Pachi, GnuGo, which are all lower than AlphaGo Fan's 3,144). This means you can deploy the system across a spectrum of compute budgets: use the raw network (1 forward pass per move) for applications where fast, good-enough play is sufficient; use a few dozen MCTS simulations for intermediate quality; and scale to 1,600+ simulations only when maximum strength is required. The smooth improvement during training (Figure 3a) suggests that more training produces a better prior, which in turn means the raw network captures more of the search's capability — a direction the paper doesn't explore but that is directly actionable: train longer, distill the search into the network, and see how close the raw network can get to the search-augmented version. If the gap can be substantially narrowed (e.g., a 40-block network trained for 400 days might produce a raw network at 4,500 Elo), then superhuman performance becomes available at dramatically lower inference cost for applications like real-time game commentary, tutoring systems that need to evaluate student moves instantly, or mobile Go apps that can't run MCTS on-device.

When to Prefer This Method

The paper articulates a clear tradeoff between pure self-play reinforcement learning (AlphaGo Zero) and systems that incorporate human data (supervised initialization, handcrafted features, rollouts). The decision rule is grounded in the empirical evidence from Figures 3–6:

Prefer tabula rasa self-play RL (AlphaGo Zero) when:

The domain has a perfect, fast simulator (game rules, physics engine, formal system) — without it, the MCTS-based policy improvement loop cannot function, and you fall back to model-free RL approaches that the paper doesn't evaluate.
You have sufficient compute to run the self-play pipeline from scratch — the 20-block version required 3 days on 64 GPU workers plus 19 CPU parameter servers, and the 40-block version required 40 days. If this budget is available, the approach produces superior asymptotic performance.
The goal is to exceed human performance, not match it — the paper shows that human data accelerates early learning but limits asymptotic performance (Figure 3a), and that the self-play network discovers strategies beyond human knowledge (Figure 5b). If surpassing the best human experts is the objective, human data is a ceiling, not a floor.
Domain knowledge beyond the rules is scarce, unreliable, or expensive to obtain — the paper demonstrates that even for Go (where human knowledge is abundant and high-quality), the human-data-free approach wins. For domains where human expertise is thin or biased, the advantage would likely be larger.

Prefer human-data-based approaches (supervised pretraining + RL, with or without rollouts) when:

Compute budget for self-play generation is limited — the supervised baseline in Figure 3a starts stronger and would be the better player at a fixed, small total FLOPs budget (e.g., if you can only afford 12 hours of training, the supervised network wins).
The domain lacks a perfect simulator — if the environment must be learned or queried from the real world, the MCTS lookahead becomes unreliable and the AlphaGo Zero training loop breaks. Human demonstrations provide training signal without requiring lookahead.
Evaluation is against a fixed human distribution and generalization to novel strategies is not required — the supervised network achieves better move prediction accuracy on professional moves (Figure 3b). If the application is analyzing human games or predicting human behavior (rather than achieving maximal playing strength), imitating humans is the right objective.
Latency or hardware constraints prevent running MCTS at inference time — while both approaches can use the raw network without search, the supervised network's raw predictions are better calibrated to human moves (though not necessarily better for winning). If the system must make decisions in a single forward pass, the supervised initialization provides stronger priors faster.

The paper's explicit finding that the residual architecture adds ~600 Elo and the dual-head architecture adds another ~600 Elo (Figure 4) applies to both approaches: whichever training paradigm you choose, you should use the AlphaGo Zero architecture (dual-res) rather than the AlphaGo Lee architecture (sep-conv), regardless of whether you initialize from human data or from scratch. The architecture improvements are orthogonal to the training data question.