Benchmarking Distributional Alignment of Large Language Models

🎯 Pitch

Language models can describe human opinion distributions far more accurately than they can simulate them, with a knowledge-to-simulation gap that balloons to over 40% error for some models. This disconnect, coupled with the finding that standard log-probability methods consistently underestimate alignment, challenges current practices in using LLMs as human simulacra.

1. Executive Summary

This paper studies how language models can match the opinion distribution of specific demographic groups—a task termed distributional alignment—by constructing a benchmark that systematically varies three understudied dimensions: the distribution expression method (e.g., model log-probabilities vs. emitting a 30-token sequence vs. directly verbalizing the distribution in JSON), the steering method (e.g., persona-based prompting vs. few-shot prompting with ground-truth distribution examples), and the question domain (expanding beyond political values to include a new NYT Book Opinions dataset on subjective reading preferences). Across GPT-4, GPT-3.5, Anthropic Opus and Haiku, and Llama-3 70B evaluated on OpinionQA, GlobalOpinionQA, and NYT Book Opinions, the paper finds that LMs can more accurately describe opinion distributions verbally than simulate them via log-probabilities or sequential sampling—a phenomenon the authors term the knowledge-to-simulation gap (reaching up to 43.6% error increase for Anthropic Opus)—and that model log-probabilities systematically underestimate distributional alignment, performing worse than even a uniform baseline. The paper also establishes that distributional alignment and steering remain substantially more challenging for non-political subjective values, with all models and humans showing reduced steerability on NYT Book Opinions compared to directly value-laden survey questions, and that few-shot steering with prior distributional data consistently improves over persona-only steering, though the best LMs with optimal expression methods only perform comparably to—rather than substantially outperforming—a human baseline already known to be poor at estimating others' opinions.

2. Context and Motivation

The Core Problem: Do Language Models Actually Represent Human Opinion Diversity?

The fundamental question this paper tackles is whether language models can faithfully simulate the distribution of opinions within a specific demographic group. This is distinct from asking whether a model can output a single "correct" answer or align with majority preferences. A model that always answers "Important" to a question about whether belief in God is necessary for morality might appear accurate if evaluated only against the majority response, but it completely fails to represent the 45% of people who hold the minority view. The paper formalizes this as the distributional alignment problem: given a survey question and a target demographic group, can an LM produce outputs whose distribution matches the actual distribution of human responses from that group?

This matters because LMs are increasingly deployed as simulacra for human populations across a range of socially consequential applications. The paper enumerates several domains where this practice is already underway:

Agent-based simulations (Park et al., 2023a): Researchers build virtual societies populated by LLM-driven agents to study emergent social behavior, economic dynamics, or policy effects. If these agents misrepresent the distribution of opinions in the populations they purport to model, the simulation results are systematically biased.
Piloting survey design (Hwang et al., 2023; Zhou et al., 2024; Aher et al., 2023; Argyle et al., 2023): Social scientists use LLMs to pretest survey instruments—generating synthetic responses to evaluate question clarity, estimate expected answer distributions, or validate experimental designs before costly human data collection. A model that produces artificially concentrated or stereotypical distributions renders this pretesting exercise misleading.
Synthetic data generation for social science: When LLMs are used to augment or replace human survey respondents (Bisbee et al., 2024), the validity of downstream statistical analyses depends entirely on the fidelity of the model's opinion distributions.

The paper draws an explicit parallel to the Perception Gap literature from political science (Yudkin et al., 2019; Levendusky and Malhotra, 2015), which documents that humans are systematically poor at estimating the opinion distributions of out-group members—Democrats misestimate the prevalence of Republican views and vice versa. If LLMs inherit similar biases from their training data (which largely consists of human-generated text exhibiting these very misperceptions), then deploying them as opinion simulators could amplify rather than mitigate existing failures of cross-group understanding.

Conflicting Prior Evidence and Unexplained Heterogeneity

The paper is motivated by a genuine contradiction in the existing literature regarding whether LMs can perform distributional alignment at all. On one side, proponents argue that the extensive training corpora of LLMs embed sufficient knowledge about diverse human perspectives to enable faithful simulation:

"some argue that the extensive training corpus of LLMs enables them to faithfully simulate demographic groups" (Grossmann et al., 2023)

On the other side, critics provide empirical evidence that such simulations are inaccurate and reflect flattened, stereotypical representations:

"others show such simulations are inaccurate and stereotypical" (Liu et al., 2024; Wang et al., 2024a)

The paper's central diagnostic insight is that these conflicting findings arise not because either side is wrong, but because the measurement of distributional alignment is highly sensitive to experimental design choices that prior work has not systematically controlled. The authors identify three sources of variation that have been underexplored in the literature, each of which can dramatically alter conclusions about an LM's distributional alignment capabilities:

1. Distribution Expression Method. Most prior work on opinion alignment (Santurkar et al., 2023; Durmus et al., 2024) evaluates LMs using model log-probabilities—comparing the probability mass the model assigns to each multiple-choice answer token against the ground-truth human distribution. However, the paper provides a compelling demonstration that this method can be fundamentally misleading. In the biased coin flip experiment (Section 3.1, Figure 2), the model is explicitly told the ground-truth distribution (e.g., P(H) = p, P(T) = 1 - p) and asked to simulate coin flips. Despite possessing this information, the model's log-probabilities for the tokens 'H' and 'T' are highly uncalibrated—they concentrate probability mass in ways that deviate sharply from the known distribution. Meanwhile, instructing the model to emit a sequence of 30 coin flips (producing something like "H T T H T...") or to directly verbalize the distribution in text ("H: 25%, T: 75%") yields much better calibration. This reveals that the failure observed in log-probability evaluations may not reflect a lack of knowledge but rather a problem with how that knowledge is measured—a finding with direct implications for how prior results should be interpreted.

2. Steering Method. Prior work has predominantly studied persona steering—prompting the model to "pretend to be a Democrat" or "simulate an answer from a group of Republicans" (Santurkar et al., 2023; Simmons, 2023; Perez et al., 2023; Cheng et al., 2023a). However, this approach is known to produce stereotyping, polarization amplification, and echo chamber effects (Perez et al., 2023; Cheng et al., 2023a; Wang et al., 2024a). An alternative that has received far less systematic evaluation is few-shot steering: providing the model with ground-truth distributional data from similar questions as in-context examples before asking for a distribution estimate. This setting reflects scenarios where practitioners have access to existing survey data (e.g., prior waves of a longitudinal study) and want to use it to improve simulation quality. The paper argues that without comparing these methods directly, it is impossible to know whether observed alignment failures are due to fundamental model limitations or suboptimal prompting strategies.

3. Dataset Domain. The literature has overwhelmingly focused on political opinions and cultural values (Santurkar et al., 2023; Durmus et al., 2024; Naous et al., 2024; Wang et al., 2024b). These are domains where opinions are strongly correlated with demographic identity markers, highly polarized, and extensively discussed in the public sphere—all factors that make them relatively easy for an LLM to learn from training data. The paper questions whether findings from these domains generalize to non-political yet subjective preferences, such as interest in reading particular books, where demographic correlations are weaker, opinions are less publicly signaled, and the mapping from identity to preference is more nuanced. Without evaluation on such domains, claims about LLMs' general ability to simulate human opinion distributions rest on a narrow evidential base.

Where Existing Approaches Fall Short

The paper identifies specific limitations in prior approaches across these three axes:

The default evaluation metric (log-probabilities) is systematically misleading. As demonstrated by the biased coin experiment and the main results (Table 1a), models evaluated via log-probabilities score worse than a uniform distribution baseline (0.363 total variation for uniform vs. 0.455–0.550 for raw log-probabilities across GPT-3.5, GPT-4, and Llama-3 70B). This means that a naive baseline that assigns equal probability to every answer choice—representing zero knowledge about the target group—would be judged as better aligned than some of the most capable LLMs. This is not because the models lack knowledge; it is because log-probabilities from RLHF-trained models exhibit concentrated probability mass on one or two answer choices rather than the dispersed distributions seen in human populations (Durmus et al., 2024). The paper explicitly connects this to the well-known observation that RLHF leads to poorly calibrated conditional probabilities (Kadavath et al., 2022). The implication is that an entire line of research—work on OpinionQA and its successors—may have substantially underestimated LM capabilities by using an evaluation method that penalizes the specific failure mode introduced by RLHF training.

Persona steering alone is insufficient. Existing persona-based approaches ask models to role-play demographic groups without providing any concrete information about how those groups actually respond to similar questions. The paper finds that this leads to stereotypical outputs (Figure 5): persona-steered models simulating Democrats systematically overestimate reading interest (simulating a 13% "very unlikely to read" rate vs. the human 33%), essentially presuming that Democrats are more intellectually engaged than the data supports. This corroborates broader concerns about persona steering flattening identity groups and amplifying stereotypes (Cheng et al., 2023b; Liu et al., 2024), but the paper goes further by quantifying the improvement when prior distributional data is available as few-shot examples, showing that the gap between persona and reality can be substantially reduced.

No publicly available dataset for non-political subjective preferences. While Santurkar et al. (2023) created OpinionQA from PEW surveys and Durmus et al. (2024) extended this to cross-national perspectives with GlobalOpinionQA, the field lacked a dataset measuring distributional alignment on preferences that are subjective but not directly tied to political identity or cultural values. This means existing evaluations cannot distinguish whether LLMs are genuinely good at simulating any human opinion distribution, or whether their performance on political surveys reflects the fact that political opinions are heavily represented in training data and strongly signaled by demographic markers.

No unified framework for comparing design choices. Individual papers have explored variations in prompting (Hu and Levy, 2023; Liu et al., 2023), uncertainty quantification (Tian et al., 2023; Mondal et al., 2024), and calibration (Guo et al., 2017; Kadavath et al., 2022) in other contexts, but none have systematically studied how these variables interact in the distributional alignment setting. The field lacks a benchmark that jointly varies distribution expression method, steering method, and question domain to understand which combinations work, which fail, and why.

How This Paper Positions Itself

The paper explicitly positions itself not as proposing a new method for improving distributional alignment, but as providing the first benchmark that systematically characterizes the sensitivity of distributional alignment measurements to experimental design choices. The contribution is primarily diagnostic and taxonomic: by varying the three key dimensions (distribution expression method, steering method, dataset) within a controlled evaluation framework and comparing against both model baselines and human annotators performing the same task, the paper aims to establish what we can and cannot conclude about LLM distributional alignment from existing methods.

The paper draws a clear line to the broader pluralistic alignment agenda (Sorensen et al., 2024; Feng et al., 2024; Kirk et al., 2024; Chen et al., 2024), which argues that models should represent the full diversity of human preferences rather than collapsing to an average. Sorensen et al. (2024) explicitly acknowledge that "there is limited knowledge of explicit alignment procedures to increase distributional calibration," and this paper directly addresses that gap by providing a systematic comparison of procedures. Rather than advocating for a single best approach, the paper aims to equip practitioners with an understanding of the tradeoffs: when verbalization is necessary vs. when sequential sampling suffices, when persona steering can be augmented with few-shot examples, and whether findings from political domains are likely to transfer to non-political preference settings.

Finally, the paper positions itself relative to the human baseline. By recruiting crowd workers to perform the same distributional estimation task (estimate the opinion distribution of a demographic group given a survey question), the paper establishes that LLM performance should be judged against a known-weak standard: humans are documented as poor predictors of out-group opinions (Yudkin et al., 2019), so merely matching human performance is not a sufficient criterion for deployment. The fact that the best models with optimal expression methods only achieve parity with this weak baseline (Table 2) suggests that distributional alignment remains fundamentally unsolved, not simply undermeasured.

3. Technical Approach

3.1 Reader Orientation

This paper constructs a benchmarking framework, not a new model or training method. The core idea is to systematically measure how well language models can match the opinion distribution of specific demographic groups—a task the authors call distributional alignment—by exposing and controlling three previously entangled variables: how the model expresses its distribution (log-probabilities, sequences, or verbalized text), how the model is steered toward a target group (persona prompting or few-shot examples), and what domain of questions is being asked (political values, cross-national cultural attitudes, or non-political subjective preferences like book interest). The framework produces a leaderboard ranking models and humans on the same task, and the key diagnostic contribution is identifying that LMs possess distributional knowledge they cannot simulate—they can accurately describe what a group believes when asked to verbalize percentages directly, but produce miscalibrated or stereotypical distributions when forced to generate individual samples, whether through log-probability sampling or sequential token emission.

3.2 Big-Picture Architecture (Diagram in Words)

The benchmark has five major components:

Datasets (Y) — the ground-truth human opinion distributions that serve as evaluation targets. Three datasets are used: OpinionQA (US political and social values from PEW surveys, 100 questions, 6 demographic groups), GlobalOpinionQA (cross-national cultural values from World Values Survey and PEW Global Attitudes, 100 questions, 19 countries filtered to top contentious pairs), and NYT Book Opinions (a newly collected dataset of 235 books with 4-point Likert-scale reading interest ratings from 346 annotators across political and gender groups).
Steering Methods (S) — the mechanisms that shift an LM's output distribution toward a target demographic group. Two methods are evaluated: persona steering (the LM is instructed to role-play as a member of the target group, e.g., "Please simulate an answer from a group of Democrats") and few-shot steering (the LM receives five examples of ground-truth distributions from the target group on similar questions before making its estimate). A no-steering baseline is also included.
Distribution Expression Methods (O) — the three ways an LM can represent an opinion distribution. These are: model log-probabilities (the probability mass the model assigns to each answer-choice token), sequence-of-tokens (the model emits 30 individual answer choices as a string, e.g., "ABBBAABDDBACBDB"), and verbalize-distributional-knowledge (the model directly outputs the distribution as formatted text, e.g., {A: 25%, B: 20%, C: 45%, D: 10%}).
Evaluation Metric — the average total variation distance between the model's predicted distribution and the ground-truth human distribution, computed per-group, per-question, and averaged across groups and questions. A lower number indicates better distributional alignment.
Human Baseline — crowd workers recruited on Prolific perform the same distributional estimation task (given a question and a target group, estimate the percentage of that group selecting each answer choice) under no-steering, persona-steering, and few-shot-steering conditions. This provides a direct point of comparison: can LLMs outperform humans at guessing what groups believe?

Information flows as follows: a question from one of the three datasets enters the system → a steering method (none, persona, or few-shot) prepends context identifying the target demographic group → the steered prompt is fed to an LLM → the LLM expresses an opinion distribution through one of the three expression methods → the predicted distribution is compared to the ground-truth human distribution via total variation distance → results are averaged across questions, groups, steering methods, and expression methods to produce the leaderboard.

3.3 Roadmap for the Deep Dive

First, the distributional alignment objective and the total variation metric, since this defines what "good alignment" means and why alternatives like KL divergence are unsuitable for the survey setting.
Second, the three distribution expression methods, since the paper's central finding—that log-probabilities systematically underestimate alignment—depends on understanding what each method measures and how the biased coin flip experiment reveals the calibration gap.
Third, the two steering methods and their prompts, since steering is what distinguishes alignment to a specific group from the model's default behavior, and the comparison between persona and few-shot reveals how much distributional knowledge is missing from demographic labels alone.
Fourth, the three datasets with emphasis on the construction of NYT Book Opinions, since the domain of questions determines whether alignment findings from political surveys generalize to non-political subjective preferences.
Fifth, the human baseline annotation protocol, since it establishes the performance floor against which LLMs are judged and reveals that models are only competitive with a known-weak human standard.
Sixth, the knowledge-to-simulation gap metric and temperature scaling procedure, since these are the paper's novel diagnostic tools for separating what models know from what they can sample.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a benchmarking and diagnostic paper whose core idea is that distributional alignment measurements are highly sensitive to experimental design choices—specifically, the distribution expression method, steering method, and question domain—and that systematically varying these dimensions reveals that LMs possess distributional knowledge they cannot faithfully simulate, that widely-used log-probability evaluations are misleading, and that non-political subjective preferences remain substantially harder to align than political values.

The Distributional Alignment Objective and Total Variation Metric

The paper formalizes distributional alignment as follows. Let $q \in \mathcal{Q}$ be a survey question to which respondents from group $g \in \mathcal{G}$ have an opinion distribution $y_{g,q}$ across multiple-choice answers. The goal is to understand how a language model can represent a group $g$ through a steering method $\mathcal{S}$ (e.g., persona or few-shot prompting), which shifts the LM's opinion distribution toward that of the target group. The model expresses its predicted distribution $\hat{y}_{g,q}$ using a distribution expression method $\mathcal{O}$ (e.g., model log-probabilities, sequence emission, or verbalization).

The paper defines distributional alignment as the average total variation distance between the ground-truth and predicted distributions, computed across all groups and questions:

$A(Y, \hat{Y}_{\mathcal{S}, \mathcal{O}}) = \frac{1}{|\mathcal{G}|} \sum_{g \in \mathcal{G}} \frac{1}{|\mathcal{Q}|} \sum_{q \in \mathcal{Q}} \frac{1}{2}||y_{g,q} - \hat{y}_{g,q}||_1$

where $Y = \{y_{g,q} \mid 1 \leq g \leq G, 1 \leq q \leq Q\}$ is the set of ground-truth human opinion distributions, $\hat{Y}_{\mathcal{S}, \mathcal{O}} = \{\hat{y}_{g,q} \mid 1 \leq g \leq G, 1 \leq q \leq Q\}$ is the set of model-predicted distributions under steering method $\mathcal{S}$ and expression method $\mathcal{O}$ , and $\frac{1}{2}||\cdot||_1$ is the total variation distance between two probability vectors.

What it computes: for each demographic group $g$ and each question $q$ , the absolute difference between the ground-truth probability and the model-predicted probability is summed across all answer choices, divided by two (to normalize to the range $[0, 1]$ ), then averaged over all questions for that group, and finally averaged over all groups. A score of 0 means perfect distributional alignment (the model's predicted distribution exactly matches the human distribution for every question and every group). A score of 0.5 means the distributions are maximally different on average.

Why this form: total variation is chosen over Kullback-Leibler (KL) divergence for a specific practical reason grounded in the nature of survey data. When no human respondents select a particular answer choice, the ground-truth probability for that choice is exactly zero. KL divergence $D_{KL}(P||Q) = \sum_i P(i) \log(P(i)/Q(i))$ becomes infinite when $Q(i) = 0$ while $P(i) > 0$ , which occurs naturally when a model assigns non-zero probability to an answer choice that received zero human responses. Consider the example the paper provides in Appendix A.12: if the ground truth is $\{A: 0.6, B: 0.35, C: 0.05\}$ and the model predicts $\{A: 0.6, B: 0.40, C: 0\}$ , then the KL divergence term for answer choice C involves $0.05 \cdot \log(0.05 / 0)$ , which goes to infinity. Total variation handles this gracefully—the contribution from C is simply $|0.05 - 0| = 0.05$ —and the final distance is $0.05$ , reflecting that these distributions are nearly identical. This robustness to zero-probability events is critical because both the sequence emission method (limited to 30 tokens) and the verbalization method (which can output zero for some choices) can produce distributions with zero entries that would break KL-based comparisons.

Additionally, total variation has a direct interpretation: a total variation distance of $d$ means that the maximum difference in probability assigned to any event (any subset of answer choices) is at most $d$ . This provides an intuitive bound on how much the model's distribution can differ from the human distribution when used for any downstream decision that depends on these probabilities.

The paper does not treat this as a matrix optimization over all $g$ and $q$ simultaneously because each question $q$ can have a different number of answer choices. The dimensionality of $y_{g,q}$ depends on $q$ , making the structure a jagged array rather than a matrix. The averaging is therefore performed over the available questions for each group-question pair, with each question weighted equally regardless of its number of answer choices.

The Biased Coin Flip Experiment: Demonstrating the Expression Method Problem

Before evaluating on real survey data, the paper conducts a controlled toy experiment that isolates the calibration properties of different distribution expression methods. The setup is deliberately simple: an LLM is instructed to simulate flips of a biased coin with known probability $P(\text{H}) = p$ and $P(\text{T}) = 1 - p$ . The instruction explicitly states the ground-truth distribution in the prompt, so the model has complete knowledge of what distribution it should produce. The question is whether the model's expressed distribution—under three different expression methods—matches the known ground truth.

Model log-probabilities (Figure 2, left panel): the model's next-token probability assigned to the token 'H' (and its complement for 'T') is extracted directly from the logit distribution after the softmax layer. The paper finds these probabilities are highly uncalibrated: they deviate systematically from the true $p$ across the full range of $p \in [0, 1]$ . The log-probability curve is not the identity line—it is compressed, meaning the model assigns probabilities that are less extreme than the true values (a form of underconfidence at the extremes) or, in some regions, exhibits non-monotonic behavior. The paper notes that temperature scaling—a post-hoc calibration technique from Guo et al. (2017) that divides logits by a learned temperature parameter $\tau$ before the softmax—substantially improves the calibration of log-probabilities (shown in green in Figure 2). Temperature scaling finds the optimal $\tau$ that minimizes total variation between the model's probabilities and the ground-truth distribution, using the formula:

$\min_{\tau} \frac{1}{2} \left\lVert y_{g,q} - \hat{y}^{1/\tau}_{g,q,\text{norm}} \right\rVert_1$

where $y_{g,q}$ is the ground-truth distribution, $\hat{y}^{1/\tau}_{g,q,\text{norm}}$ is the temperature-scaled and renormalized model distribution, and the optimization searches over $\tau$ to find the single scalar that minimizes total variation. The paper acknowledges that this gives temperature scaling access to ground-truth labels (it is an oracle calibration method in this setting), so it represents an upper bound on what log-probability calibration can achieve.

Sequence of tokens (Figure 2, middle panel): the model is instructed to emit 30 tokens, each either 'H' or 'T', representing individual coin flip outcomes. The predicted probability is the empirical frequency of heads in this 30-token sequence. The paper finds that this method produces distributions that are much better calibrated than raw log-probabilities—the empirical frequencies closely track the true $p$ across the full range, without requiring temperature scaling.

Verbalize distributional knowledge (Figure 2, right panel): the model is instructed to directly state the probability distribution in text form (e.g., "H: 25%, T: 75%"). The paper finds this method is also well-calibrated, with the verbalized percentages closely matching the known ground truth.

What this experiment demonstrates: the model possesses the correct distributional knowledge (it was explicitly given the probabilities in the prompt). The failure of log-probabilities to express this knowledge is therefore not a knowledge deficit—it is a measurement artifact specific to how token-level probabilities are computed and calibrated in modern LLMs, particularly those trained with RLHF. The implication is that evaluations using log-probabilities conflate two separate capabilities: (1) whether the model knows the distribution and (2) whether the model's token-level probabilities are calibrated. The sequence and verbalization methods disentangle these by expressing the distribution through generated tokens rather than internal probabilities, and the results show that the knowledge is present even when log-probabilities suggest otherwise.

The paper explicitly connects this to the RLHF calibration problem: models fine-tuned with reinforcement learning from human feedback exhibit concentrated probability mass on preferred outputs (Kadavath et al., 2022), which directly causes the miscalibration observed in the log-probability method. The sequence and verbalization methods bypass this by forcing the model to translate its knowledge into explicit token outputs, which are subject to different (and apparently more favorable) calibration dynamics.

Distribution Expression Method 1: Model Log-Probabilities

Definition: for each multiple-choice question, the model's raw next-token log-probability assigned to each answer choice identifier (e.g., the token 'A', 'B', 'C', 'D') is extracted from the model's output distribution after the final softmax layer, with sampling temperature set to 1.0. These probabilities are normalized across the available answer choices to form a valid probability vector. This is the canonical method used in OpinionQA (Santurkar et al., 2023), GlobalOpinionQA (Durmus et al., 2024), and most follow-up work on LLM opinion alignment.

What it measures: the model's internal uncertainty about which answer choice would be produced if the model were to sample a single token. It captures the model's implicit representation of answer plausibility, as encoded in the logit values after the full transformer forward pass and RLHF alignment.

Temperature-scaled variant: the paper also evaluates temperature-scaled log-probabilities as a post-hoc correction. For each dataset and steering method, a temperature parameter $\tau$ is selected that minimizes total variation between the scaled model distribution and the ground-truth distribution (Equation 3 in Appendix A.2). The scaled probabilities are computed as $\hat{y}^{1/\tau}_{g,q,\text{norm}}$ , where the logits are divided by $\tau$ before the softmax and the resulting probabilities are renormalized. Crucially, this means that $\tau$ is chosen with access to the ground-truth labels—it is an oracle calibration technique that establishes an upper bound on what log-probabilities can achieve. The paper finds that temperature scaling substantially improves GPT-3.5 and GPT-4 but provides minimal benefit for Llama-3-70B (see Figure 6 in Appendix A.2 and Expected Calibration Error values in Table 3: 0.11 vs. 0.13 for Llama-3-70B, compared to 0.07 vs. 0.28 for GPT-4 and 0.06 vs. 0.20 for GPT-3.5).

Observed failure mode: raw log-probabilities exhibit highly concentrated probability mass on one or two answer choices. Durmus et al. (2024) previously observed this effect; the paper confirms it quantitatively in the main leaderboard (Table 1a), where raw log-probabilities score between 0.455 and 0.550 total variation—worse than the uniform baseline of 0.363, which assigns equal probability to every answer choice and represents zero knowledge about any demographic group. The paper attributes this to RLHF training, which optimizes models to produce specific preferred outputs and in doing so "sharpens" the token-level probability distribution, reducing the entropy that would be needed to match the dispersed distributions typical of human survey responses.

Why this method matters despite its failures: it is the dominant evaluation paradigm in the existing literature, so its failure has implications for how entire lines of prior work should be interpreted. If log-probability evaluations systematically underestimate distributional alignment, then negative findings in prior work may reflect measurement limitations rather than genuine model incapabilities. The paper's contribution is not to discard log-probabilities but to demonstrate that they answer a different question than other expression methods—they measure the model's calibrated sampling distribution, while verbalization measures the model's explicit distributional knowledge—and that failing to distinguish these leads to incorrect conclusions about model capabilities.

Distribution Expression Method 2: Sequence of Tokens

Definition: the model is instructed to emit a sequence of exactly 30 samples from the distribution, where each token in the sequence is an answer choice identifier (e.g., 'A', 'B', 'C', 'D'). The instruction format is to produce a string like "ABBBAABDDBACBDB" of length 30. The predicted distribution is the empirical frequency of each answer choice in this sequence—for example, if the sequence contains 12 A's, 6 B's, 8 C's, and 4 D's, the predicted probabilities are A: 0.40, B: 0.20, C: 0.267, D: 0.133.

What it measures: the model's ability to act as a sampler from its own distributional knowledge. Unlike log-probabilities, which capture static token-level probabilities, the sequence method requires the model to autoregressively generate individual draws that collectively reflect a target distribution. This is directly relevant to practitioners who want to generate synthetic survey responses for simulation purposes—each token in the sequence can be treated as an individual simulated respondent.

The discretization error problem: because the sequence length is finite (30 tokens), the empirical distribution estimated from the sequence is an approximation of the continuous distribution the model intends to express. Even if the model perfectly intends to sample from the ground-truth distribution, the finite sample introduces estimation error. The paper reports the discretization error as a baseline: for each question, the authors repeatedly draw 30 samples from the actual ground-truth human distribution and compute the total variation between this finite-sample empirical distribution and the true ground-truth distribution. This discretization error is approximately 0.115–0.136 depending on the dataset (Table 1a, Table 10 in Appendix A.13). This means that no model could achieve total variation below this value using the 30-token sequence method—it is the irreducible error floor imposed by the finite-sample approximation. Any deviation of a model's sequence-method performance above this floor represents genuine misalignment beyond what sampling noise would produce.

Why 30 tokens? The paper does not provide an explicit justification for the choice of 30, but this number likely reflects a tradeoff: longer sequences would reduce discretization error but increase inference cost and the probability that the model loses coherence or repeats patterns due to autoregressive degeneration. The sequence length is fixed across all experiments to ensure comparability.

Advantages over log-probabilities: the sequence method does not require API access to token-level log-probabilities (which some model providers, notably Anthropic, do not expose). It also provides a concrete set of simulated survey responses that can be directly used in downstream analyses, rather than an abstract probability distribution. Most importantly, as demonstrated in the biased coin flip experiment, it avoids the calibration pathologies of log-probabilities in RLHF-trained models—the model can generate well-calibrated samples even when its token-level probabilities are miscalibrated.

Limitations: the discretization error means the sequence method has a fundamental accuracy ceiling that the verbalization method does not share. It also requires the model to follow a specific output format reliably, which the paper notes that smaller models (e.g., Llama-3-8B, Llama-2-7B/13B) struggle with—Table 7 in Appendix A.9 shows success rates for sequence-following as low as 0% for Deepseek-coder models and 3% for Llama-2-70B, motivating the paper's restriction to larger instruction-tuned models.

Distribution Expression Method 3: Verbalize Distributional Knowledge

Definition: the model is instructed to directly output the opinion distribution as structured text, specifically in a JSON format mapping answer choices to percentages (e.g., {"A": 25, "B": 20, "C": 45, "D": 10}). No post-processing, estimation, or sampling is applied—the percentages are extracted directly from the model's text output by parsing the JSON structure.

What it measures: the model's explicit, declarative knowledge about how a demographic group's opinions are distributed. By asking the model to state percentages rather than generate samples, this method isolates knowledge of the distribution from the ability to sample from that distribution. This is the key conceptual distinction the paper introduces: a model might know that "Democrats respond to this question as A:10%, B:60%, C:30%" and be able to state this explicitly, while simultaneously being unable to produce a sequence of individual responses that reflects these proportions (due to autoregressive biases, calibration issues, or other generation-time artifacts).

Advantages over sequence and log-probabilities: the verbalization method has no inherent discretization error, since the model outputs continuous percentages directly. It is not subject to the calibration pathologies of log-probabilities, since it relies on the model's text-generation capabilities rather than its token-level probability distributions. And it provides a direct, human-readable estimate that requires no statistical post-processing. The paper consistently finds that verbalization produces the best distributional alignment across all models and datasets (Table 1a: the top 5 entries in the leaderboard are all verbalization results, with Anthropic Opus at 0.226 ± 0.006 and GPT-4 at 0.229 ± 0.006).

Limitations: verbalized distributions are not directly usable as samples for simulation purposes—a practitioner who needs 1000 synthetic survey responses cannot use a JSON blob of percentages; they would need an external random sampler to convert the percentages into individual draws. This means verbalization is not a complete solution for simulation use cases; it merely reveals that the distributional knowledge exists and could in principle be surfaced with an appropriate sampling mechanism. The paper also notes that verbalization requires the model to reliably produce structured JSON output, which, like the sequence method, can fail for smaller or less instruction-tuned models (Table 7 shows success rates for Llama-2-7B at 52% and Llama-3-8B at 40%).

The Knowledge-to-Simulation Gap

The paper formalizes the discrepancy between what a model knows (verbalized distribution) and what it can simulate (sequence emission) through a metric called the knowledge-to-simulation gap:

$\text{KSS} = \frac{A(Y, \hat{Y}_{\mathcal{S}, \text{Sequence}})}{A(Y, \hat{Y}_{\mathcal{S}, \text{Verbalize}})} - 1$

where $A(Y, \hat{Y}_{\mathcal{S}, \text{Sequence}})$ is the total variation distance when the model emits a 30-token sequence, and $A(Y, \hat{Y}_{\mathcal{S}, \text{Verbalize}})$ is the total variation distance when the model verbalizes the distribution directly.

What it computes: the percentage increase in total variation error incurred by requiring the model to emit individual samples rather than state the distribution directly. If $\text{KSS} = 0$ , the model is equally good at simulating and verbalizing—there is no gap between knowledge and sampling. If $\text{KSS} = 0.5$ , the simulation error is 50% larger than the verbalization error, meaning the model's sampling process introduces substantial additional misalignment beyond what its knowledge alone would produce.

Why this form: the ratio formulation normalizes by the verbalization performance, making the gap comparable across models with different absolute levels of distributional knowledge. A model with poor knowledge will have high error in both verbalization and simulation; the ratio isolates whether simulation specifically degrades performance relative to what the model demonstrably knows. The subtraction of 1 converts the ratio to a percentage increase, which is more interpretable than a raw ratio. An additive difference $A_{\text{Seq}} - A_{\text{Verb}}$ would not control for baseline knowledge levels and would be harder to compare across models with different overall capabilities.

Results (Table 1b): the knowledge-to-simulation gap varies substantially across models. GPT-3.5 Turbo has the smallest gap at 9.17%, meaning its sequence emission is only marginally worse than its verbalization. GPT-4 shows a 21.35% gap. Anthropic Haiku shows a 21.49% gap. Llama-3-70B shows a 34.65% gap—its simulation error is over a third larger than what its knowledge would predict. Anthropic Opus shows the largest gap at 43.63%, despite being the best model at verbalization (0.226 total variation, the top of the leaderboard). This means that Anthropic Opus possesses the most accurate distributional knowledge of any model tested, but is the least capable of translating that knowledge into simulated samples. The gap is not a function of overall model quality—it is a specific failure mode that affects some model families more than others.

What this gap implies practically: if a practitioner is using models with a high knowledge-to-simulation gap (particularly Llama-3 and Anthropic models), they should consider a two-stage pipeline: first, prompt the model to verbalize the distribution; second, use an external random number generator or a separate sampling step to draw individual responses from that distribution. This separates the knowledge elicitation problem from the simulation problem and avoids the penalty that the model's own autoregressive sampling imposes.

The gap for log-probabilities: while Equation 2 defines the gap relative to the sequence method, the paper notes that the gap between verbalization and log-probabilities is even larger. In Table 1a, the best verbalization result (Anthropic Opus: 0.226) and the best raw log-probability result (GPT-3.5: 0.455) differ by a factor of approximately 2×—the log-probability error is 101% larger than the verbalization error. For GPT-4, the log-probability result (0.550) is 140% larger than its verbalization result (0.229). This underscores the main methodological warning of the paper: using log-probabilities to measure distributional alignment systematically and dramatically underestimates what models know.

Steering Method 1: Persona Steering

Definition: following the approach of Santurkar et al. (2023) and Kambhatla et al. (2022), the model is instructed to role-play as a member of the target demographic group by prepending a persona description to the prompt. The exact instruction format is: "Please simulate an answer from a group of [Demographic Group]." where the demographic group is one of the target categories (e.g., "Democrats," "Republicans," "Men," "Women," "Black," "White" for OpinionQA; the corresponding groups for the other datasets).

What it measures: the model's ability to shift its opinion distribution toward a specific group based solely on the semantic content of the demographic label. This tests whether the model's training data has encoded associations between demographic identities and opinion distributions that can be activated through a simple identity prompt.

Observed limitations: the paper identifies three specific failure modes of persona steering:

Stereotyping in non-political domains (Figure 5): when persona-steered to simulate Democrats evaluating book interest (NYT Book Opinions), the models systematically overestimate reading engagement. The simulated Democrat has a 13% chance of responding "very unlikely to read," while the actual human Democrat annotators had a 33% chance—a 20-percentage-point gap. Conversely, the simulated Democrat has a 25% chance of responding "very likely to read," compared to the human 12%. The models are essentially assuming that Democrats are more intellectually curious than the data supports, reflecting a stereotypical association between political identity and cultural engagement that the training data may encode.
Flattened polarization: the persona-steered distributions tend toward more extreme, less nuanced representations. The paper finds this corroborates prior work by Cheng et al. (2023b) and Liu et al. (2024) showing that persona steering produces caricatured rather than representative distributions.
Insufficient information content in demographic labels: the demographic label alone does not provide enough information for the model to accurately recover the target distribution, as evidenced by the consistent and significant improvement when few-shot examples are added (discussed below). This suggests that the model's implicit associations between demographic groups and opinions are not sufficiently granular to reproduce real survey distributions.

Why this method is still studied: it represents the most common approach in the literature and the simplest deployment scenario—a practitioner needs no prior survey data, only a target demographic label. Understanding its failure modes is therefore of direct practical importance, even if the method itself is found wanting.

Steering Method 2: Few-Shot Steering

Definition: the model is provided with five in-context examples of ground-truth opinion distributions from the target demographic group on questions similar to the target question, before being asked to estimate the distribution for the target question. The five examples are selected through a two-stage process designed to maximize informativeness while avoiding trivial copying:

Similarity computation: for each question in the dataset, the 10 most similar other questions are identified using cosine similarity between text embeddings produced by SimCSE (Gao et al., 2021). This ensures the examples are topically coherent—if the target question is about driverless vehicles, the examples will be about related technology or policy topics.
Distinctiveness filtering: from these top-10 most similar questions, the 5 questions with the most distinct output distributions (i.e., where the ground-truth opinion distribution differs most from the target question's distribution) are selected. This filtering step prevents the model from simply copying the distribution from a near-duplicate question. The paper notes that OpinionQA contains many variants of the same question, and providing identical or near-identical distributions as examples would inflate apparent performance without testing genuine generalization.

What it measures: the model's ability to leverage explicit distributional data to improve its estimates, representing scenarios where practitioners have access to prior survey results (e.g., previous waves of a longitudinal study, pilot data, or published survey reports) and can use them to calibrate LLM-based simulation.

Prompt structure: the few-shot examples are presented as pairs of (similar question, ground-truth distribution for the target group). The model sees five such pairs, followed by the target question, and is instructed to simulate an answer from the target group. The distribution expression method (sequence, verbalize, log-probabilities) is applied to the model's response to the target question only—the few-shot examples are input context, not scored outputs.

Performance improvement over persona steering: across all datasets and most models, few-shot steering produces statistically significant improvements in distributional alignment compared to persona steering (Figure 4). The average total variation decreases for all models except GPT-3.5 (which shows a slight degradation, though the paper does not explore why). Humans also benefit from few-shot examples, with both models and humans showing the same directional improvement. This is expected: providing concrete distributional data reduces the burden on the model's implicit demographic knowledge and gives it a numerical target to calibrate against.

Interaction with stereotyping: the paper finds that few-shot steering substantially reduces the stereotyping observed in persona steering for the NYT Book Opinions dataset (Figure 5, green vs. orange bars). When provided with actual distributional data from similar books, the models' marginal distributions over Likert ratings move closer to the human reference, suggesting that the stereotyping observed under persona steering is partly due to the model falling back on coarse demographic associations when specific distributional information is absent.

Dataset 1: OpinionQA

Source and selection: the OpinionQA dataset (Santurkar et al., 2023) consists of survey questions and response distributions from Pew Research Center surveys of US adults. The original dataset includes a steerability subset of 500 "contentious" questions where demographic subgroups (e.g., Democrats vs. Republicans) frequently disagree in their response distributions. The paper randomly samples 100 questions from this 500-question subset to create a manageable evaluation set spanning topics including science, politics, and personal relationships.

Demographic groups: six groups are evaluated: Democrat, Republican, Male, Female, Black, and White. These represent the intersection of political affiliation, gender, and race that are available in the PEW survey data and were used in the original OpinionQA analysis.

Ground-truth distributions: the ground truth is the empirical distribution of responses from PEW survey respondents belonging to each demographic group. These are not model estimates; they are actual survey results from nationally representative samples.

Why this dataset matters: it is the most widely-used benchmark in the LLM opinion alignment literature and serves as the point of comparison with prior work. By replicating the OpinionQA evaluation protocol while varying the distribution expression method and steering approach, the paper directly tests whether the conclusions of Santurkar et al. (2023) and subsequent work are robust to these methodological choices, or whether they are artifacts of the specific evaluation pipeline used.

Dataset 2: GlobalOpinionQA

Source and selection: GlobalOpinionQA (Durmus et al., 2024) consists of questions and answer distributions from two cross-national surveys: the World Values Survey and the PEW Global Attitudes Survey. The original dataset includes responses from 138 countries. Because the few-shot steering method requires selecting examples from the same country with sufficient data, the paper filters to only countries with at least 600 responses to other questions, reducing the set to 19 countries. From these, the 100 questions with the highest disagreement between pairs of countries are selected, where disagreement is measured by the total variation distance between the answer distributions of different countries.

Demographic groups: the groups are countries rather than within-country demographic categories. Example countries from the 19-country set include Pakistan, Japan, and Britain (as shown in Appendix A.3).

Question example (Appendix A.3): the question "Do you personally believe that drinking alcohol is morally acceptable, morally unacceptable, or is it not a moral issue?" reveals stark cross-national differences. Pakistan's distribution is 1% morally acceptable, 95% morally unacceptable, 2% not a moral issue, 2% depends on the situation. Japan's distribution is 67% morally acceptable, 6% morally unacceptable, 25% not a moral issue, 2% depends on the situation. Britain's distribution is 39% morally acceptable, 9% morally unacceptable, 47% not a moral issue, 5% depends on the situation. These distributions capture genuine cultural variation in moral values that an LLM would need to represent accurately for distributional alignment.

Why this dataset matters: it extends the evaluation beyond US-centric political opinions to cross-national cultural values, testing whether LLMs can represent diverse global perspectives rather than defaulting to Western (particularly American) cultural norms—a concern raised by Wang et al. (2024b) and others studying cultural bias in LLMs.

Absence of human baseline for GlobalOpinionQA: the paper explicitly does not collect human annotations for this dataset. The stated reason is that "estimating the opinions of respondents from different countries would require annotators with deep, contextually relevant knowledge of each country's sociocultural landscape and it is well established that annotations from Western populations do not accurately reflect non-western views" (Apicella et al., 2020; Arnett, 2008). This is a deliberate methodological choice to avoid confounding the evaluation with culturally mismatched human judgments. The cost is that for GlobalOpinionQA, we cannot directly compare LLM performance to human performance on the same task.

Dataset 3: NYT Book Opinions (New Contribution)

Motivation: the paper identifies a gap in existing benchmarks: all prior work on distributional alignment evaluates opinions that directly measure political values or cultural attitudes. In these domains, the relationship between demographic identity and opinion is strong, publicly signaled, and extensively discussed in training data. The paper constructs NYT Book Opinions to test whether alignment findings generalize to non-political, yet still subjective preferences—specifically, interest in reading particular books—where demographic correlations are weaker and opinions are expressed through a layer of abstraction (the book's content) rather than explicit value statements.

Book selection: 235 books were collected from The New York Times "Top Books" lists spanning the past two decades (as of 2024). For each book, the dataset includes the title, author, a summary, and the genre.

Annotation setup: 346 annotators were recruited on Prolific, filtered for English fluency, and paid $12 per hour. Each annotator rated 26 books (out of the 235 total) on a 4-point Likert scale in response to the question: "Given the summary of this book, how likely are you to read it?" The Likert scale ranges from 1 ("Very unlikely") to 4 ("Very likely"). The paper reports that annotators passed attention checks with 93% accuracy.

Demographic groups and sample sizes: the annotation included 131 Male, 206 Female, 165 Democrat, and 172 Republican annotators (Appendix A.4). This yields approximately 18 annotations per book per demographic group (165 Democrat annotators × 26 books per annotator / 235 books ≈ 18.3 annotations per book per Democrat, and similarly for other groups).

Ground-truth distributions: for each book and each demographic group, the opinion distribution is computed as the empirical fraction of annotators from that group selecting each of the four Likert ratings. This is a discrete probability distribution over $\{1, 2, 3, 4\}$ .

Evidence of demographic disagreement (Figure 3): the paper demonstrates that book preferences vary systematically by demographic group. The top four books preferred by Republicans over Democrats include Redeployment by Phil Klay (short stories by a former Marine who served in Iraq) and Washington's Crossing by David Hackett Fischer (a narrative from the American Revolutionary War). The top four books preferred by Democrats over Republicans include A Promised Land by Barack Obama and A Short History of Women by Kate Walbert. These differences are intuitive—they align with known political identity markers—but they are expressed through book preferences, not through direct political opinion questions. An LLM that can only simulate political opinions but not infer book preferences from demographic identity would fail on this dataset.

Quantifying agreement between groups: the paper reports Cohen's kappa between Democrats and Republicans as 0.05, indicating "little to no agreement," and between Men and Women as 0.15, indicating "a small amount of agreement" (Appendix A.4). This quantifies the baseline difficulty: the groups genuinely differ in their preferences, so a model that produces the same distribution for all groups will necessarily have poor distributional alignment.

Copyright and data release: the paper notes that New York Times articles are protected under copyright, but excerpts are collected under fair use exemption for research purposes. The released dataset includes links to the original New York Times content rather than reproducing full book summaries, respecting copyright while enabling reproducibility.

Why this dataset matters: it enables testing whether distributional alignment is a general capability or one that is specific to domains where opinions are strongly signaled by demographic identity. If LLMs perform well on OpinionQA but poorly on NYT Book Opinions, it suggests that their alignment capabilities are domain-dependent and that extrapolating from political surveys to other subjective preference domains is unjustified. The paper's results (Figure 4) confirm this: steerability—the reduction in total variation from no-steering to persona and few-shot steering—is smaller for NYT Book Opinions than for OpinionQA, indicating that aligning models to non-political subjective preferences is genuinely harder.

Human Baseline Annotation Protocol

Design: human annotators are recruited on Prolific and asked to perform exactly the same distributional alignment task that the LLMs are evaluated on. For each question from OpinionQA and NYT Book Opinions (GlobalOpinionQA is excluded for the reasons discussed above), annotators see the question text, answer choices, and a target demographic group, and are asked to estimate the percentage of people in that group who would select each answer choice. The estimates must sum to 100%.

Steering conditions: each annotator completes the task under three conditions: no steering (estimate the distribution for "Americans as a whole" or the general population), persona steering (estimate the distribution for a specific demographic group, e.g., "Democrats"), and few-shot steering (estimate the distribution for a specific demographic group, given five examples of how that group responded to similar questions). The instructions for each condition are shown in Figures 9, 10, and 11 of Appendix A.6.

Annotation details: each question receives four independent annotations (four different human estimates of the opinion distribution). For OpinionQA, there are 246 annotators across all steering conditions; for NYT Book Opinions, there are 374 annotators. The demographics of the annotator pool are reported: for OpinionQA annotations, the pool is 73% Democrat, 27% Republican, 33% Male, 67% Female, 84% White, 16% Black; for NYT annotations, the pool is 69% Democrat, 31% Republican, 37% Male, 63% Female (Appendix A.6).

In-group vs. out-group analysis: the paper examines whether humans are better at estimating the opinions of their own demographic group (in-group) vs. other groups (out-group). The results (Table 6, Appendix A.6) show that while persona steering produces a larger average difference between in-group and out-group estimates (suggesting in-group estimates may be slightly better), the differences are not statistically significant. For example, on OpinionQA with persona steering, in-group total variation is 0.297 ± 0.013 vs. out-group 0.322 ± 0.014. This null result is itself informative: human annotators are not reliably better at estimating their own group's opinion distribution than other groups' distributions, consistent with the Perception Gap literature showing that people systematically misperceive out-group opinions regardless of their own group membership.

Why this baseline matters: it provides a direct, apples-to-apples comparison between human and machine performance on the same task. The comparison is not between LLM-generated distributions and human-generated distributions (which would be a quality judgment) but between the accuracy of both humans and LLMs in matching the ground-truth survey data. The paper's key finding from this comparison (Table 2) is that the best LLMs with optimal expression methods (verbalization) achieve total variation scores (0.204–0.235) that are comparable to human performance (0.250), but this is framed as a negative result: "this is not particularly promising for the field of distributional alignment given that humans are known to be poor predictors of opinions of the opposite party" (Section 5.1). Matching a weak baseline does not indicate readiness for deployment.

Quality control: annotators are filtered for English fluency and must pass a reading attention check with 93% accuracy. The task is designed to take a reasonable amount of time per question (the paper does not specify exact timing but notes that compensation at $12/hour is "well over the federal minimum wage of$ 7.25").

Temperature Scaling Procedure (Oracle Calibration)

Definition: temperature scaling is a post-hoc calibration technique from Guo et al. (2017) that adjusts the confidence of neural network predictions by dividing the logits (pre-softmax output values) by a scalar temperature parameter $\tau$ before applying the softmax function. When $\tau > 1$ , the softmax output becomes more uniform (higher entropy); when $\tau < 1$ , the output becomes more peaked (lower entropy). The temperature is typically learned on a held-out calibration set by minimizing a proper scoring rule.

Application in this paper: for each dataset and each steering method separately, the authors find the temperature $\tau$ that minimizes the total variation between the temperature-scaled model probabilities and the ground-truth human distributions, using the formula from Equation 3 (Appendix A.2):

$\min_{\tau} \frac{1}{2} \left\lVert y_{g,q} - \hat{y}^{1/\tau}_{g,q,\text{norm}} \right\rVert_1$

where $y_{g,q}$ is the ground-truth distribution, $\hat{y}^{1/\tau}_{g,q,\text{norm}}$ is the temperature-scaled and renormalized model distribution, and $\tau$ is the temperature parameter being optimized.

What this means operationally: the raw logits from the model are divided by $\tau$ , the softmax is reapplied, the resulting probabilities across the answer choices are renormalized to sum to 1, and the total variation against the ground truth is computed. This process is repeated to find the $\tau$ that minimizes total variation.

Critical caveat—this is an oracle method: the optimization uses the ground-truth human distributions ( $y_{g,q}$ ), which would not be available in a real deployment scenario. Temperature scaling in this paper therefore represents an upper bound on what log-probability calibration can achieve—it answers the question "if we could perfectly calibrate the model's log-probabilities post-hoc using knowledge of the true distribution, how good could log-probability alignment be?" It does not represent a practical method that could be deployed without ground-truth labels. The paper's comparisons between temperature-scaled and raw log-probabilities should be interpreted as quantifying how much of the log-probability performance gap can be attributed to miscalibration vs. fundamental knowledge deficits.

Results of temperature scaling: temperature scaling substantially improves GPT-3.5 and GPT-4 performance, moving them from total variation values of 0.455–0.550 (raw log-probabilities) to 0.273–0.296 (temperature-scaled). This is a dramatic improvement—temperature-scaled GPT-4 log-probabilities (0.273) would rank 5th on the leaderboard, above several sequence-method results. However, temperature scaling provides minimal improvement for Llama-3-70B: its raw log-probability result is 0.495 and its temperature-scaled result is 0.469 (Table 1a), a much smaller gain. The Expected Calibration Error (ECE) values in Table 3 (Appendix A.2) confirm this pattern: temperature scaling reduces GPT-4's ECE from 0.28 to 0.07 (a 75% reduction) and GPT-3.5's ECE from 0.20 to 0.06 (a 70% reduction), but Llama-3-70B's ECE only drops from 0.13 to 0.11 (a 15% reduction). This suggests that Llama-3-70B's log-probability failures are due to something other than simple temperature miscalibration—potentially a more fundamental issue with how its token-level probabilities are structured.

Summary of Design Choices and Their Justifications

Total variation over KL divergence: avoids infinite values when distributions have zero-probability events, which naturally occur with finite survey samples and finite-length sequence emission. Total variation is also bounded in $[0, 1]$ and directly interpretable as the maximum probability difference for any event.
Three distribution expression methods spanning internal probabilities (log-probabilities), explicit sampling (sequence), and declarative knowledge (verbalization): disentangles knowledge from simulation capability and reveals measurement artifacts specific to RLHF-trained models.
Persona and few-shot steering as separate conditions: tests whether demographic labels alone are sufficient (they are not) and whether providing concrete distributional data improves estimation (it does, substantially and across both models and humans).
NYT Book Opinions as a non-political preference dataset: fills a gap in existing benchmarks and tests whether alignment capabilities transfer beyond domains where opinions are strongly correlated with and signaled by demographic identity.
Human baseline on the identical task: provides a direct performance reference and reveals that even the best LLM configurations only match a known-weak human standard, reframing apparent "good" performance as insufficient for deployment.
Temperature scaling as oracle calibration: establishes an upper bound on log-probability performance and decomposes the log-probability failure into a calibration component (fixable with temperature) and a residual component (not fixable with simple temperature adjustment, suggesting deeper issues).
Five-question few-shot selection with distinctiveness filtering: prevents the model from copying distributions from near-duplicate questions while providing topically relevant examples. The distinctiveness criterion (selecting the most different distributions among topically similar questions) is a non-obvious design choice that tests generalization rather than memorization.
30-token sequence length: balances discretization error against inference cost and autoregressive coherence, with the discretization error floor explicitly reported as a baseline (0.115–0.136) so that sequence-method results can be interpreted relative to this irreducible minimum.
Exclusion of GlobalOpinionQA from human baseline: avoids culturally mismatched human judgments that would confound the comparison, acknowledging the well-documented problem that Western annotators cannot accurately estimate non-Western opinion distributions.

4. Key Insights and Innovations

Innovation 1: The Knowledge-to-Simulation Gap as a Diagnostic Concept

The paper's most conceptually distinctive contribution is the introduction and empirical characterization of the knowledge-to-simulation gap: the observation that a language model can accurately describe what a demographic group believes (by verbalizing percentages directly) while simultaneously being unable to faithfully generate individual samples from that same distribution (via log-probabilities or sequential token emission). This is not merely a performance observation—it is a diagnostic decomposition that separates two capabilities prior work has implicitly conflated: possessing distributional knowledge versus being able to act as a calibrated sampler of that knowledge.

Prior work in LLM opinion alignment (Santurkar et al., 2023; Durmus et al., 2024) has evaluated models almost exclusively through log-probabilities—comparing the model's token-level probability mass across answer choices against the ground-truth human distribution. This paradigm treats the model's internal probability distribution as the only expression of its distributional knowledge. The paper's biased coin flip experiment (Section 3.1, Figure 2) demonstrates why this conflation is incorrect: when the model is explicitly told the ground-truth distribution (P(H) = p, P(T) = 1-p) and asked to express it, log-probabilities are miscalibrated while verbalized percentages and sequential sampling are accurate. The model has the knowledge; log-probabilities simply fail to measure it. This reframes a substantial fraction of prior negative findings—conclusions that LLMs cannot represent human opinion diversity—as potential measurement artifacts rather than genuine capability deficits.

What makes this a fundamental rather than incremental contribution is the formalization of the gap as a quantifiable metric (Equation 2) that varies substantially across model families and can be ranked independently of overall alignment quality. The finding that Anthropic Opus achieves the best verbalization score (0.226 total variation, top of the leaderboard in Table 1a) while exhibiting the largest knowledge-to-simulation gap (43.63% error increase, worst in Table 1b) demonstrates that the gap is not simply a proxy for model capability—it is an orthogonal property reflecting how different training procedures affect the relationship between internal knowledge representations and autoregressive sampling behavior. This has direct practical implications: a practitioner using Anthropic Opus for survey simulation should not have the model generate individual responses (sequence or log-probability methods), but should instead prompt it to verbalize the distribution and use an external sampler, thereby avoiding a 43.6% error penalty that has nothing to do with the model's actual understanding of the target group.

The gap also provides a unifying explanation for previously puzzling results. The observation that LLMs "know what they know" in factual settings (Kadavath et al., 2022) but produce poorly calibrated probabilities after RLHF finds its distributional analog here: the knowledge is encoded in the model's text-generation capabilities but the RLHF process simultaneously distorts the token-level probability distribution used to measure it. The paper's demonstration that temperature scaling substantially closes the gap for GPT-4 (from 0.550 to 0.273 total variation—a ~50% improvement) but not for Llama-3-70B (from 0.495 to 0.469—only ~5% improvement) further shows that the nature of the gap is training-procedure-dependent, not universal across all instruction-tuned models.

Innovation 2: Reframing Distributional Alignment as a Measurement Design Problem Rather Than a Model Capability Problem

The paper performs a Gestalt shift on the distributional alignment literature. Rather than asking "can LLMs simulate human opinion distributions?"—which presupposes a single correct way to measure the answer—the paper demonstrates that the answer depends substantially on how one asks the question, specifically along three axes (distribution expression method, steering method, and question domain) that prior work has neither systematically varied nor even recognized as interdependent variables.

This reframing is significant because it transforms the research agenda from capability evaluation (is the model good enough?) to measurement methodology (are we measuring what we think we're measuring?). The canonical approach in the field—measuring alignment via model log-probabilities with persona steering on political survey questions—is shown to be a worst-case configuration: log-probabilities are the poorest-performing expression method (scoring below the uniform baseline for all models in Table 1a), persona-only steering produces stereotypical distributions (Figure 5, with Democrats' reading interest overestimated by nearly a factor of two), and findings from political domains do not generalize to non-political preferences (Figure 4, where steerability is reduced on NYT Book Opinions). A researcher using this default pipeline would conclude that LLMs are fundamentally incapable of distributional alignment—a conclusion the paper shows is an artifact of the pipeline, not a property of the models.

What distinguishes this from a typical benchmarking contribution is the diagnostic decomposition the paper provides. Rather than simply proposing a new metric and ranking models, the paper identifies which specific design choices cause which specific failure modes. Log-probabilities fail because RLHF induces miscalibration (a claim supported by the biased coin experiment and temperature scaling results). Persona steering fails because demographic labels alone activate coarse stereotypes rather than fine-grained distributional knowledge (supported by the consistent improvement when few-shot examples are added). Political surveys inflate apparent alignment because opinions in these domains are strongly signaled by demographic identity markers and extensively discussed in training data (supported by the domain gap between OpinionQA and NYT Book Opinions). This decomposition allows practitioners to diagnose why their simulation pipeline is failing and intervene at the appropriate level—switching from log-probabilities to verbalization, augmenting persona prompts with few-shot examples, or recognizing that their domain may be inherently harder than the political surveys studied in prior work.

The paper's explicit refusal to propose a single "best" method is itself a methodological contribution. Instead of optimizing for one leaderboard metric under one configuration, the paper demonstrates that optimal choices depend on the use case: verbalization is best for evaluating knowledge, sequence emission is necessary for generating synthetic samples, and few-shot examples should be used whenever prior data exists. This equips practitioners with a design space rather than a recipe, which is more valuable for a problem as deployment-context-dependent as human simulation.

Innovation 3: Establishing That Non-Political Subjective Preferences Are a Distinct and Harder Distributional Alignment Problem

The construction of the NYT Book Opinions dataset and the comparative analysis across domains represents a negative result with significant implications: distributional alignment is substantially harder for preferences that are subjective but not directly political or cultural in nature. This is not an incremental finding—it challenges the external validity of the entire existing LLM opinion alignment literature, which has focused almost exclusively on political values and cultural attitudes (Santurkar et al., 2023; Durmus et al., 2024; Naous et al., 2024; Wang et al., 2024b).

The paper's evidence for this domain gap is multi-layered. First, the Cohen's kappa values quantifying group disagreement on NYT Book Opinions are near zero (0.05 between Democrats and Republicans, 0.15 between Men and Women), confirming that demographic groups do genuinely differ in their book preferences—the dataset is not simply capturing noise. Second, the differences are interpretable in ways that align with known political identity markers (Republicans preferring books about military service and American history, Democrats preferring books by Democratic political figures and feminist literature), confirming that the preferences are not random but reflect genuine demographic variation. Third, and most critically, Figure 4 demonstrates that steerability—the reduction in total variation when moving from no-steering to persona and few-shot steering—is smaller for NYT Book Opinions than for OpinionQA across all models and for human annotators. This means that even when given demographic labels (persona) or explicit distributional examples from similar questions (few-shot), both models and humans are less able to adjust their estimates toward the target group for book preferences than for political opinions.

Why does this domain gap matter beyond being "a new dataset"? It reveals a fundamental limitation in how LLMs acquire demographic knowledge. Political opinions are directly discussed in training data: news articles report that "Democrats support X policy by a margin of Y%," opinion pieces argue about partisan divides, and survey results are widely disseminated. An LLM can learn the correlation between Democratic identity and support for climate policy directly from explicit statements in its training corpus. Book preferences, by contrast, are rarely stated as explicit demographic generalizations—training data does not contain sentences like "Republicans are 20% more likely than Democrats to want to read books about military history." This means an LLM must infer implicit demographic patterns from individual book reviews, reading lists, and cultural commentary—a substantially harder learning problem that requires a different kind of distributional knowledge than explicit opinion polling data.

The implication is that the existing literature's focus on political and cultural values has likely overestimated LLMs' general capacity for distributional alignment by studying the easiest case. Practitioners considering using LLMs to simulate consumers for market research, to pilot surveys about product preferences, or to model audience engagement with entertainment content—all applications where preferences are expressed through layers of abstraction similar to the NYT Book Opinions setup—should not expect the levels of alignment observed on PoliticalQA-style evaluations. The problem is not that LLMs are bad at simulating humans; it is that they may only be adequate in the narrow domain where opinions are explicitly stated and strongly signaled by demographic identity, and the field has not previously had the data to detect this boundary.

Innovation 4: Human Baselines as a Reframing Device for Evaluating Simulation Quality

The paper's inclusion of a human baseline performing the identical distributional alignment task is methodologically routine—comparing machine performance to human performance is standard practice in NLP benchmarking. What is distinctive is how the result is interpreted: rather than celebrating that the best LLMs match or slightly exceed human performance (Table 2: GPT-4 verbalization at 0.204 vs. humans at 0.250), the paper frames this as evidence that distributional alignment is not solved, because humans themselves are known to be systematically poor at this task.

This interpretative move draws on the Perception Gap literature from political science (Yudkin et al., 2019; Levendusky and Malhotra, 2015), which documents that Americans substantially misperceive the opinion distributions of opposing political parties—Democrats overestimate the prevalence of extreme views among Republicans and vice versa. By positioning the human baseline not as a gold standard but as a known-weak reference point, the paper reframes the evaluation landscape. A model that merely matches human performance on distribution estimation is not ready for deployment, because deploying such a model would replicate the same systematic misperceptions that contribute to political polarization and cross-group misunderstanding in human society.

Section 5.1 makes this reframing explicit and provocative: "It would be highly questionable to base the result of social science surveys on participants guessing others' opinions, and our findings indicate that LMs offer little improvement over this baseline." This is the paper's clearest argument that distributional alignment is an open problem, not one nearing solution through better prompting alone. Even with the optimal configuration (verbalization + few-shot steering + politically charged questions), the best models achieve total variation scores around 0.15-0.20—meaning that on average, the predicted distribution differs from the true distribution by 15-20 percentage points in the worst case. For a survey question with two answer choices, this represents a 30-40 percentage point swing in estimated support, which would render any downstream analysis based on these distributions substantively unreliable.

The paper also provides the first empirical evidence comparing in-group vs. out-group human estimation accuracy in this specific task format (Table 6, Appendix A.6). The finding that differences are not statistically significant—Democrats are not reliably better at estimating Democratic opinion distributions than Republicans are—runs counter to what one might expect from the Perception Gap literature (which emphasizes out-group misperception rather than in-group accuracy). This null result suggests that distributional estimation is difficult even in the absence of the intergroup biases that the Perception Gap literature studies, and that the challenge is at least partly a general statistical reasoning problem (estimating a probability distribution from memory and general knowledge) rather than purely a motivated reasoning or intergroup bias problem. LLMs inherit this difficulty, and the paper's evidence suggests they have not solved it—they have approximately replicated it.

5. Experimental Analysis

Evaluation Methodology

Dataset. Three datasets are used: (1) OpinionQA (Santurkar et al., 2023)—100 questions randomly sampled from the 500-question "contentious" steerability subset, covering PEW survey responses from six US demographic groups (Democrat, Republican, Male, Female, Black, White); (2) GlobalOpinionQA (Durmus et al., 2024)—100 questions filtered from 19 countries (requiring ≥600 responses per country) by selecting those with highest total variation disagreement between country pairs, drawn from World Values Survey and PEW Global Attitudes; (3) NYT Book Opinions—newly collected, 235 books from The New York Times "Top Books" lists with 4-point Likert-scale reading interest ratings from 346 annotators (131 Male, 206 Female, 165 Democrat, 172 Republican), yielding approximately 18 annotations per book per demographic group. All datasets use ground-truth human opinion distributions rather than model-generated references.
Base model(s). Five instruction-tuned LLMs are evaluated: GPT-4 (gpt-4-0613), GPT-3.5-Turbo (gpt-3.5-turbo-0125), Anthropic Claude 3 Haiku, Anthropic Claude 3 Opus, and Meta Llama-3 70B Instruct. The paper restricts to larger models because smaller open-source models (Llama-3-8B, Llama-2 variants at 7B/13B/70B, Deepseek-coder models) exhibited poor instruction-following for the sequence and verbalization methods, with success rates as low as 0% for Deepseek-coder-6.7B and 3% for Llama-2-70B on sequence emission (Appendix A.9, Table 7). Models are chosen to span different developers, scales, and training procedures, enabling analysis of whether the knowledge-to-simulation gap is a general phenomenon or specific to particular training recipes.
Metrics. The primary metric is average total variation distance between the model-predicted opinion distribution $\hat{y}_{g,q}$ and the ground-truth human distribution $y_{g,q}$ , computed as $\frac{1}{|\mathcal{G}|} \sum_{g \in \mathcal{G}} \frac{1}{|\mathcal{Q}|} \sum_{q \in \mathcal{Q}} \frac{1}{2} ||y_{g,q} - \hat{y}_{g,q}||_1$ (Equation 1). Lower values indicate better distributional alignment, with 0 representing perfect match and 0.5 representing maximally different distributions. Total variation is chosen over KL divergence because KL becomes infinite when either the model or the human distribution assigns zero probability to an answer choice—a situation that naturally arises with finite survey samples (no respondents selecting a particular choice) and with finite-length sequence emission (the discretization error floor is computed by repeatedly drawing 30 samples from the ground-truth distribution and computing total variation against the true distribution, yielding an irreducible error of approximately 0.115–0.136 depending on the dataset). The knowledge-to-simulation gap (Equation 2) is a derived metric: $\text{KSS} = A(Y, \hat{Y}_{\mathcal{S}, \text{Sequence}}) / A(Y, \hat{Y}_{\mathcal{S}, \text{Verbalize}}) - 1$ , measuring the percentage increase in total variation error when the model must emit individual samples (sequence) rather than directly state the distribution (verbalization).
Baselines. Two non-learned baselines are reported: (1) Uniform—each answer choice is assigned equal probability (e.g., 25% each for a 4-choice question), producing a total variation of 0.363 on the combined leaderboard (Table 1a), 0.381 on OpinionQA, 0.223 on NYT Book Opinions, and 0.486 on GlobalOpinionQA (Table 10, Tables 4/8/9 in Appendix A.13); (2) Majority vote—all probability mass is placed on the single answer choice with the highest ground-truth frequency, producing total variation values substantially worse than uniform (0.712–0.731 across datasets). Additionally, the paper reports the discretization error for the sequence method as a performance floor—the minimum achievable total variation given the 30-token finite-sample approximation, computed by simulating 30 i.i.d. draws from the ground-truth distribution and measuring empirical frequency deviation (0.115 on NYT, 0.136 on OQA, 0.092 on GlobalOpinionQA). For log-probabilities, a temperature-scaled oracle variant is included: the temperature parameter $\tau$ is chosen to minimize total variation against the ground-truth distribution (Equation 3), providing an upper bound on what log-probability calibration can achieve when ground-truth labels are available (this is explicitly not a practical deployment method). Human annotators serve as a performance reference: crowd workers on Prolific complete the identical distribution estimation task under no-steering, persona-steering, and few-shot-steering conditions, with four annotations per question, compensated at $12/hour.
Generation budget / compute accounting. The paper does not measure compute in FLOPs or tokens—it measures "generation budget" implicitly through the expression method constraints. The sequence method always emits exactly 30 tokens per question. Model log-probabilities require a single forward pass per question. Verbalization produces variable-length text output per question. All methods are compared on equal footing per question: each model produces one output (one sequence, one set of log-probabilities, or one JSON verbalization) per question–group–steering combination, and the resulting distributions are compared to ground truth. There is no scaling analysis where generation budget is varied to produce compute-performance curves—the budget is fixed per expression method, and the analysis focuses on which method is most accurate at a fixed per-question cost.
Cross-validation / statistical protocol. No train-validation-test splitting is performed—all models are evaluated zero-shot and few-shot without any fine-tuning, so there is no risk of overfitting to the evaluation data. Confidence intervals are computed via bootstrapping with 1000 resamples over questions, reported as ± values in all tables (e.g., Anthropic Opus verbalization: 0.226 ± 0.006 in Table 1a). For the few-shot steering condition, the five most similar questions with the most distinct output distributions are selected as in-context examples, preventing models from simply copying distributions from near-duplicate questions (which are common in OpinionQA due to survey design variations). The distinctiveness filtering ensures that the few-shot evaluation measures generalization rather than memorization of similar survey items.

Main Quantitative Results

The Distributional Alignment Leaderboard (Table 1a)

The headline result is the ranking of models by distributional alignment performance, averaged across all three datasets, persona and few-shot steering, and all demographic groups. The leaderboard reveals a stark stratification by distribution expression method:

Verbalization dominates the top of the leaderboard. The five best-performing configurations are all verbalization results: Anthropic Opus (0.226 ± 0.006), GPT-4 (0.229 ± 0.006), Llama-3 70B (0.244 ± 0.006), Anthropic Haiku (0.254 ± 0.007), and GPT-3.5-Turbo (0.291 ± 0.007). The gap between the best verbalization result and the best non-verbalization result is substantial—GPT-4 temperature-scaled log-probabilities (the highest-ranked sampler) achieves 0.273 ± 0.006, which is approximately 21% worse than Opus's verbalization score.
Sequence emission occupies the middle tier but is bounded by discretization error. The best sequence-method result is GPT-4 at 0.278 ± 0.008, followed closely by Anthropic Haiku (0.309 ± 0.006), GPT-3.5-Turbo (0.318 ± 0.007), Anthropic Opus (0.325 ± 0.007), and Llama-3 70B (0.328 ± 0.008). All sequence results are substantially above the discretization error floor of 0.115 ± 0.004, indicating that even the best-performing sequence models incur a substantial additional error beyond what finite-sample approximation noise would produce. The gap between the sequence results and the discretization error (e.g., GPT-4: 0.278 vs. 0.115 floor) represents genuine misalignment—the model is systematically generating samples from a distribution different from the ground truth, above and beyond what sampling noise alone would produce.
Raw log-probabilities are catastrophically poor. GPT-3.5-Turbo log-probabilities (0.455 ± 0.008), Llama-3-70B log-probabilities (0.495 ± 0.008), and GPT-4 log-probabilities (0.550 ± 0.008) all perform worse than the uniform baseline of 0.363 ± 0.007. This means that a naive strategy assigning equal probability to every answer choice—representing zero knowledge about any demographic group—would substantially outperform these models' internal probability estimates at matching human opinion distributions. GPT-4, the most capable model in verbalization (0.229), is the worst of all models at log-probability expression (0.550)—a gap of more than 2× in total variation between what the model knows and what its token-level probabilities reveal.
Temperature scaling partially rescues log-probabilities, but unevenly across models. Temperature-scaled GPT-4 log-probabilities (0.273 ± 0.006) improve dramatically over raw log-probabilities (0.550)—a ~50% reduction in total variation—and would rank 5th overall on the leaderboard, above several sequence-method results. Temperature-scaled GPT-3.5 (0.296 ± 0.006) shows a similar improvement over its raw score (0.455). However, temperature-scaled Llama-3-70B (0.469 ± 0.009) improves only marginally over raw (0.495 ± 0.008), suggesting that Llama-3-70B's log-probability failures stem from a deeper issue than simple temperature miscalibration. The Expected Calibration Error values in Table 3 (Appendix A.2) confirm this asymmetry: temperature scaling reduces GPT-4's ECE from 0.28 to 0.07 (75% reduction) and GPT-3.5's ECE from 0.20 to 0.06 (70% reduction), but Llama-3-70B's ECE only drops from 0.13 to 0.11 (15% reduction).
The uniform baseline outperforms the majority vote baseline by a wide margin. Uniform achieves 0.363 ± 0.007 while majority vote achieves 0.712 ± 0.013—nearly double the error. This is expected: human opinion distributions on contentious survey questions are typically dispersed across multiple answer choices, and collapsing to the single most popular choice ignores the substantial minority views that the distributional alignment metric explicitly rewards representing. The fact that raw log-probabilities (0.455–0.550) fall between these two baselines indicates that models are not simply collapsing to a point estimate—they are producing some dispersion—but that dispersion is systematically miscalibrated relative to the ground truth.

The Knowledge-to-Simulation Gap by Model (Table 1b)

The knowledge-to-simulation gap (Equation 2) quantifies how much additional error is incurred when models must simulate samples (sequence method) rather than state distributions directly (verbalization). The results reveal substantial variation across model families:

GPT-3.5 Turbo: 9.17% gap. The smallest gap of any model. GPT-3.5's sequence emission is only marginally worse than its verbalization—this model is nearly as good at sampling from its distributional knowledge as it is at expressing that knowledge declaratively.
GPT-4: 21.35% gap. More than double GPT-3.5's gap. Despite being the more capable model overall, GPT-4 loses significantly more performance when translating knowledge into samples. This is consistent with the pattern observed in the biased coin experiment and in the calibration curves (Figure 6, Appendix A.2): GPT-4's token-level probabilities are substantially miscalibrated in ways that verbalization bypasses.
Anthropic Haiku: 21.49% gap. Nearly identical to GPT-4. Anthropic's smaller model exhibits a similar magnitude of simulation penalty despite different training infrastructure and architecture.
Llama-3 70B: 34.65% gap. A large gap. Llama-3's simulation error is over one-third larger than what its verbalization performance would predict. Combined with the finding that temperature scaling barely helps Llama-3's log-probabilities (Table 1a), this suggests that Meta's instruction tuning produces a specific failure mode where distributional knowledge is encoded in ways that the autoregressive sampling process cannot reliably access.
Anthropic Opus: 43.63% gap. The largest gap by a substantial margin—and this is the model that achieves the #1 rank on the main leaderboard for verbalization (0.226). Opus possesses the most accurate distributional knowledge of any tested model, yet it is the worst at converting that knowledge into simulated samples. This is the paper's most striking demonstration that knowledge and simulation are orthogonal capabilities: Opus knows the distributions better than any other model, but its autoregressive sampling behavior systematically distorts that knowledge, introducing a 43.6% error penalty that erases its knowledge advantage and drops it below GPT-4 and GPT-3.5 on sequence-based evaluation.

Steering Method and Dataset Interactions (Figure 4, Tables 8–10)

Figure 4 plots the average total variation for each dataset and steering method, averaged across demographic groups, for the 30-token sequential distribution output. Key findings:

Few-shot steering improves over persona steering for nearly all configurations. On OpinionQA with the sequence method (Table 10, Appendix A.13), GPT-4's persona-steered total variation is 0.238 ± 0.009 while its few-shot-steered total variation is 0.204 ± 0.009—a 14% improvement. Anthropic Opus improves from 0.282 ± 0.012 (persona) to 0.248 ± 0.010 (few-shot). Anthropic Haiku improves from 0.284 ± 0.012 to 0.262 ± 0.012. Llama-3 70B improves from 0.316 ± 0.013 to 0.300 ± 0.012. GPT-3.5 Turbo is the exception—it degrades slightly from 0.317 ± 0.012 (persona) to 0.339 ± 0.015 (few-shot), though the paper does not explore why.

The improvement extends to humans. Human annotators on OpinionQA with persona steering achieve approximately 0.30–0.32 total variation (Table 6, Appendix A.6: 0.297 ± 0.013 for in-group, 0.322 ± 0.014 for out-group) while few-shot steering reduces this to 0.278–0.283. On NYT Book Opinions, humans improve from 0.273–0.281 (persona) to 0.236–0.237 (few-shot). Both models and humans benefit from explicit distributional data; the relative improvement is comparable in magnitude.

Steerability is domain-dependent. The gap between no-steering and persona/few-shot steering is larger for OpinionQA than for NYT Book Opinions (Figure 4). On OpinionQA, all models show substantial reductions in total variation when steering is applied—the base model's default distribution (no steering) is far from any specific demographic group, and persona/few-shot steering moves it meaningfully closer. On NYT Book Opinions, the reduction from no-steering to steered is smaller in absolute terms. This means that models are less steerable toward specific demographic preferences when the preferences are about book interest rather than political values—the demographic signal is weaker in this domain, and both persona labels and few-shot examples provide less traction for distributional adjustment.

GPT-3.5 is anomalously insensitive to steering on NYT Book Opinions. Examining Table 10 in detail: on NYT Book Opinions with the sequence method, GPT-3.5 achieves 0.215 ± 0.006 with no steering, 0.226 ± 0.007 with persona steering, and 0.228 ± 0.005 with few-shot steering—all three are nearly identical. In contrast, GPT-4 on the same dataset/setting moves from 0.220 ± 0.006 (no steering) to 0.264 ± 0.009 (persona) to 0.243 ± 0.006 (few-shot), and Anthropic Opus moves from 0.316 ± 0.011 to 0.483 ± 0.010 to 0.334 ± 0.009. GPT-3.5's default distribution already approximates some average of the demographic groups for book preferences, and steering methods fail to differentiate it further—suggesting that GPT-3.5 may have limited distributional knowledge about demographic variation in reading preferences specifically.

GlobalOpinionQA results confirm the same patterns. Separate tables for GlobalOpinionQA persona steering (Table 8) and few-shot steering (Table 9) in Appendix A.13 show: verbalization remains the best method across both steering conditions; few-shot steering substantially improves performance (e.g., Anthropic Opus verbalization: 0.275 ± 0.021 persona vs. 0.208 ± 0.020 few-shot); and the sequence method continues to lag behind verbalization (e.g., Opus sequence: 0.338 ± 0.028 persona vs. 0.265 ± 0.025 few-shot). The discretization error floor for GlobalOpinionQA is 0.092 ± 0.004 (lower than OQA/NYT because the distribution of answer choices differs). The knowledge-to-simulation gap on GlobalOpinionQA (Table 5) ranges from 12.15% (GPT-3.5) to 28.68% (GPT-4), consistent with the US-centric datasets.

Stereotyping Under Persona Steering (Figure 5, Figure 13 in Appendix A.13)

Figure 5 disaggregates the NYT Book Opinions results by showing the marginal distribution of Likert ratings (1–4) for Democrat and Republican persona-steered models, few-shot-steered models, and the human reference distributions. The key finding is that persona-steered models produce stereotypical distributions:

Democrat-simulated reading interest is systematically overestimated. The persona-steered Democrat has a 13% chance of responding "Very unlikely to read" (Likert 1), compared to 33% for actual human Democrat annotators—a 20-percentage-point gap. The persona-steered Democrat has a 25% chance of responding "Very likely to read" (Likert 4), compared to 12% for human Democrats. The models are essentially encoding a stereotype that Democrats are more intellectually engaged or culturally curious than the data supports.
Few-shot steering substantially reduces but does not eliminate this gap. When provided with five examples of ground-truth distributions (green bars in Figure 5), the models' marginal distributions move closer to the human reference. The Democrat "Very unlikely" rate increases from the persona value toward the human 33%, and the "Very likely" rate decreases from 25% toward the human 12%. However, the gap is not fully closed—even with few-shot examples, the models do not perfectly reproduce the human marginal distributions across all Likert levels.
Humans exhibit a similar but smaller stereotyping pattern. Figure 13 in Appendix A.13 extends Figure 5 to include persona-steered and few-shot-steered human annotators. Persona-steered humans also overestimate Democrat reading engagement (purple bars), though the effect is smaller in magnitude than for models. Few-shot-steered humans (pink bars) move closer to the reference distribution, mirroring the model improvement pattern.
Republican-simulated distributions are less stereotyped but still deviate. The right panel of Figure 5 shows that Republican-simulated reading distributions are closer to the human Republican reference than Democrat simulations are to the Democrat reference, but notable deviations remain—particularly the "Very likely to read" category, which is overestimated by persona models.

Distributional Alignment with Human Performance (Table 2)

Table 2 narrows the comparison to the two datasets where human annotations exist (OpinionQA and NYT Book Opinions) and ranks models alongside human performance. The results contextualize LLM capabilities:

GPT-4 verbalization: 0.204 ± 0.003.
Anthropic Opus verbalization: 0.219 ± 0.004.
Llama-3 70B verbalization: 0.225 ± 0.004.
Anthropic Haiku verbalization: 0.235 ± 0.004.
GPT-4 sequence: 0.237 ± 0.004.
Humans (verbalization equivalent): 0.250 ± 0.004.
GPT-3.5 verbalization: 0.259 ± 0.005.

The best LLMs with the optimal expression method (verbalization) achieve total variation scores modestly better than the human baseline: GPT-4's 0.204 vs. humans' 0.250 represents a ~18% relative improvement. GPT-4 using the sequence method (0.237) is also slightly better than humans. However, the paper's interpretation is not that LLMs have "surpassed human performance" on this task. Rather, the framing in Section 5.1 emphasizes that "this is not particularly promising for the field of distributional alignment given that humans are known to be poor predictors of opinions of the opposite party." Matching a known-weak baseline does not constitute readiness for deployment.

Ablation Studies and Robustness Checks

Expected Calibration Error (ECE) analysis for log-probabilities (Table 3, Appendix A.2): The paper reports ECE values to quantify the miscalibration of log-probabilities. GPT-4: ECE 0.28 raw, 0.07 with temperature scaling. GPT-3.5: ECE 0.20 raw, 0.06 with temperature scaling. Llama-3-70B: ECE 0.13 raw, 0.11 with temperature scaling. The near-zero improvement for Llama-3-70B confirms that its log-probability failures are not simple calibration issues fixable by a global temperature parameter—the probability mass distribution itself is poorly structured for matching human opinion dispersions, independent of overall confidence calibration.

Temperature scaling calibration curves (Figure 6, Appendix A.2): The calibration curves visualize the relationship between model-predicted probability and empirical accuracy (in this context, alignment with ground-truth bin frequency). GPT-4 and GPT-3.5 show substantial improvement in calibration after temperature scaling (the curves move closer to the identity line). Llama-3-70B's calibration curve remains largely unchanged, with significant deviations from the diagonal persisting after scaling. This reinforces that the type of miscalibration differs qualitatively across model families.

Model log-probability per-token analysis for biased coin flips (Figure 12, Appendix A.7): A detailed breakdown of the biased coin flip experiment shows that the model's log-probabilities for 'H' are not consistent across positions in the 30-token sequence—the probability varies depending on where in the sequence the token appears, despite the instruction specifying a stationary Bernoulli process. This per-token conditional probability variation is a mechanism-level explanation for why sampling from log-probabilities produces distributions that deviate from the known ground truth: the model's autoregressive generation introduces sequential dependencies that corrupt the intended stationary distribution, even when the model "knows" the distribution should be i.i.d.

In-group vs. out-group human performance (Table 6, Appendix A.6): Across both OpinionQA and NYT Book Opinions, under both persona and few-shot steering, the differences between in-group and out-group human estimation accuracy are not statistically significant. For OpinionQA persona steering, in-group total variation is 0.297 ± 0.013 vs. out-group 0.322 ± 0.014—the confidence intervals overlap substantially. For NYT Book Opinions persona steering, in-group is 0.281 ± 0.010 vs. out-group 0.273 ± 0.010, with out-group actually nominally better. This null result indicates that the difficulty of the distributional estimation task is not primarily driven by intergroup bias—both in-group and out-group estimation are similarly challenging, suggesting the primary bottleneck is a general statistical reasoning difficulty rather than motivated misperception.

Open-source model instruction-following failure (Table 7, Appendix A.9): The paper tested additional smaller models but found they could not reliably follow the prompt format for sequence and verbalization methods. Llama-3-8B achieved only 45% success on sequence and 40% success on verbalization. Llama-2-70B: 3% success on sequence. Llama-2-13B: 11% sequence, 83% verbalization. Llama-2-7B: 10% sequence, 52% verbalization. Deepseek-coder models: near 0% on both. This establishes a practical lower bound on model scale for the benchmark—smaller instruction-tuned models lack the output format control needed to produce valid JSON or consistent-length token sequences, making the distributional alignment task infeasible below a certain capability threshold.

GlobalOpinionQA breakdown by steering method (Tables 8–9, Appendix A.13): Separating GlobalOpinionQA results into persona-only (Table 8) and few-shot-only (Table 9) reveals that the few-shot improvement observed in the US-centric datasets generalizes to cross-national cultural values. With persona steering, the best verbalization score is Anthropic Opus at 0.275 ± 0.021; with few-shot steering, Opus improves to 0.208 ± 0.020—a 24% reduction in total variation. GPT-4 improves from 0.319 ± 0.022 (persona) to 0.237 ± 0.023 (few-shot). The sequence method shows similar relative improvements. This robustness across cultural contexts suggests that providing explicit distributional data is universally beneficial, not domain-specific.

Cohen's kappa analysis for NYT Book Opinions (Appendix A.4): The paper quantifies agreement between demographic groups using Cohen's kappa on the Likert-scale book ratings. Democrat–Republican kappa: 0.05 ("little to no agreement"). Male–Female kappa: 0.15 ("small amount of agreement"). These low values confirm that the groups genuinely differ in their preferences—the dataset captures real demographic variation rather than noise—while also establishing that these differences are subtler than in political opinion domains, where intergroup agreement would be even lower for contentious questions. This validates the NYT Book Opinions dataset as measuring a harder distributional alignment problem: weaker demographic signal means models must rely on finer-grained associations that may not be robustly encoded in training data.

Critical Assessment

Do the experiments support the claim that model log-probabilities systematically underestimate distributional alignment?

Supported, with the qualification that the "true" alignment is only demonstrated via verbalization, not validated externally. The evidence is clear and multi-layered: the biased coin flip experiment (Figure 2) shows miscalibration of log-probabilities when the ground truth is known exactly; the main leaderboard (Table 1a) shows all raw log-probability results falling below the uniform baseline; temperature scaling substantially improves GPT models' log-probability scores (Table 1a, Figure 6) but not Llama-3-70B's; and verbalization consistently outperforms log-probabilities by large margins across all models, datasets, and steering conditions. The claim that log-probabilities "underestimate" alignment depends on accepting verbalization as an accurate expression of the model's distributional knowledge. The paper provides no independent verification that verbalized percentages are meaningfully more accurate than log-probabilities in an absolute sense—only that they are closer to the ground-truth distributions. A skeptic could argue that verbalization simply produces a different type of error that happens to be measured as smaller by total variation, rather than tapping into genuinely more accurate distributional knowledge. However, the biased coin experiment (where ground truth is perfectly known) directly demonstrates that verbalization is better calibrated than raw log-probabilities in a setting where accuracy can be definitively assessed, lending credence to the interpretation that verbalization better reflects knowledge in the survey setting as well.

Do the experiments support the existence and magnitude of the knowledge-to-simulation gap?

Supported, with the gap's magnitude potentially overstated due to asymmetries in the comparison. The gap is computed by comparing sequence emission to verbalization (Equation 2). However, these two methods involve different output formats and different parsing procedures—verbalization requires JSON parsing of percentages, while sequence emission requires counting token frequencies from a 30-token string. The sequence method has an inherent discretization error floor of 0.115 that verbalization does not share. The reported gap of 43.63% for Anthropic Opus represents the percentage increase from verbalization total variation (0.226) to sequence total variation (0.325). If we subtract the discretization error from the sequence result (treating it as irreducible noise), the "clean" sequence performance would be 0.325 − 0.115 = 0.210, which would actually be better than the verbalization score of 0.226—reversing the gap entirely for Opus. The paper does not perform this decomposition or discuss whether the discretization error should be subtracted before computing the gap. This means the reported knowledge-to-simulation gap values conflate two sources of error: genuine simulation miscalibration and the finite-sample approximation inherent to 30-token sequences. The gap is likely real—the biased coin experiment shows it in a setting without discretization issues—but its magnitude in the survey experiments is inflated by not accounting for the sequence method's irreducible noise floor.

Do the experiments support that few-shot steering improves over persona steering?

Supported across most configurations, but with unexplained exceptions. Figure 4 and the full results in Tables 8–10 consistently show that few-shot steering produces lower total variation than persona steering for nearly all models and datasets. The improvement is observed in both models and humans, lending cross-validation to the finding. However, GPT-3.5 is an anomaly: on OpinionQA, its few-shot sequence result (0.339) is worse than its persona sequence result (0.317), and on NYT Book Opinions, its few-shot result on the verbalization method (0.226) is only marginally better than its persona result (0.239) while essentially unchanged from no-steering (0.219). The paper does not investigate why GPT-3.5 fails to benefit from few-shot examples in these settings. This raises the possibility that the few-shot benefit is model-dependent and may not generalize to all instruction-tuned LLMs. A deeper analysis—for instance, measuring whether GPT-3.5 is copying the few-shot distributions too literally (despite the distinctiveness filtering) or ignoring them entirely—would strengthen confidence in the generality of the finding.

Do the experiments support that non-political subjective preferences are a harder distributional alignment problem?

Partially supported, but the evidence is confounded by differences in dataset construction. Figure 4 shows that steerability (reduction in total variation from no-steering to steered) is smaller for NYT Book Opinions than for OpinionQA. However, NYT Book Opinions and OpinionQA differ in multiple ways beyond the political vs. non-political dimension: (1) NYT has 4 answer choices (Likert scale) while OpinionQA questions have varying numbers of choices (often 2–5+); (2) NYT has 235 questions while OpinionQA in this benchmark has 100; (3) NYT annotations are from Prolific workers while OpinionQA distributions come from nationally representative PEW surveys—the sampling methodology, sample size, and representativeness of the ground-truth distributions differ substantially; (4) the "demographic groups" in NYT are constructed post-hoc from annotator self-reports, with unequal group sizes (165 Democrats vs. 172 Republicans), while OpinionQA uses PEW's survey-weighted demographic categories. The domain (political vs. non-political) is not isolated from these other variables. An ideal experiment would hold the survey methodology constant while varying only question content—for example, collecting PEW-style political questions and non-political preference questions using the same sampling frame and demographic categories. The current evidence is suggestive but cannot definitively attribute the steerability gap to the political vs. non-political distinction rather than to differences in data quality, sample size, or answer format.

Do the experiments support that LMs offer little improvement over a weak human baseline?

Supported for the specific human baseline collected, but the baseline may not represent the strongest possible human performance. The human annotators were Prolific crowd workers paid $12/hour, given a few examples of the task, and asked to produce percentage estimates for demographic groups. This is a reasonable but not optimized human baseline. Professional pollsters, political scientists, or domain experts with access to reference materials would likely perform substantially better. The paper's claim that "LMs offer little improvement over this baseline" is accurate for the specific baseline studied, but framing this as evidence that distributional alignment is "not solved" requires the additional (reasonable but unstated) assumption that the human baseline represents a minimum acceptable performance threshold for deployment. If a deployed survey simulation system merely replicated human-level accuracy at guessing demographic distributions, would that be sufficient? The paper argues no—basing social science on participants guessing others' opinions would be "highly questionable"—but does not establish what level of accuracy would be sufficient, nor whether LLMs are on a trajectory to reach it.

Missing experiments and genuine weaknesses

No difficulty analysis. Unlike the referenced example paper (which analyzes results by prompt difficulty quintiles), this paper does not attempt to characterize which questions are "hard" or "easy" for distributional alignment. A breakdown by question characteristics—topic, number of answer choices, entropy of the ground-truth distribution, degree of intergroup disagreement—would reveal whether alignment failures are concentrated in particular question types or uniform across the dataset. This is particularly important for the knowledge-to-simulation gap: is the gap driven by a few pathological questions or is it uniform?

No analysis of model uncertainty or confidence. The paper evaluates distributional alignment by comparing a single predicted distribution to ground truth, but does not elicit or evaluate the model's uncertainty about its own estimates. A model that verbalizes "A: 25%, B: 75%" with high confidence when the ground truth is "A: 40%, B: 60%" is making a different kind of error than a model that expresses similar percentages but acknowledges low confidence. The paper's framework provides no mechanism for models to express epistemic uncertainty, which limits the diagnostic value of the alignment metric—a large total variation could reflect either ignorance (the model doesn't know and guesses) or miscalibration (the model is confident and wrong).

No multi-turn or iterative refinement. All evaluations are single-turn: the model produces one distributional estimate per question. The paper does not explore whether models can improve their estimates through iterative prompting (e.g., "Re-examine your estimate: do you think 25% of Democrats would be very unlikely to read this book after seeing this summary?"). Given the existence of the knowledge-to-simulation gap, an iterative refinement process that allows the model to compare its verbalized knowledge against its simulated samples might close the gap—but this is not tested.

No cross-model correlation analysis. The paper does not examine whether models that are better at verbalization also tend to be better at sequence emission (which would suggest a shared knowledge substrate) or whether the two capabilities are largely independent (which would suggest different underlying mechanisms). A simple correlation plot of verbalization rank vs. knowledge-to-simulation gap across the five models would reveal whether the gap is systematically related to model quality.

Single survey methodology for human baseline. The human annotators saw the same question format and were asked to produce percentage estimates directly. This is a specific type of human judgment task that may not capture how humans naturally reason about group opinion distributions. Alternative human baselines—such as asking humans to predict what fraction of a group would endorse each statement, or asking them to estimate the shape of the distribution without numerical percentages—might produce different results and either reduce or increase the observed human–LLM gap.

Small annotation samples per book for NYT. With approximately 18 annotations per book per demographic group, the ground-truth distributions for individual books are noisy. A book with 18 Democrat annotations might show 40% "Very likely" based on 7 annotators—a proportion that could easily be 30% or 50% with a different sample of annotators. This noise in the ground truth introduces variance in the total variation metric that is not accounted for in the reported bootstrapped confidence intervals (which bootstrap over questions, not over annotator samples). The true uncertainty in the NYT Book Opinions alignment estimates is likely larger than the reported standard errors.

No analysis of the effect of the number of few-shot examples. The paper uses exactly five few-shot examples in all experiments, with no ablation varying this number (1, 3, 5, 10, 20 examples). It is unknown whether the improvement from persona to few-shot saturates at 5 examples, or whether additional examples would yield further gains. Similarly, the distinctiveness filtering threshold (top 5 most distinct among top 10 most similar) is not ablated.

Oracle temperature scaling is not deployable. The temperature scaling results (0.273 for GPT-4, 0.296 for GPT-3.5) represent an upper bound obtained by minimizing total variation against ground-truth labels. These numbers cannot be achieved in practice without access to the very distributions the model is trying to predict. The paper is transparent about this, but readers may overlook the caveat and misinterpret temperature-scaled log-probabilities as a practical method rather than an oracle diagnostic. A non-oracle temperature scaling method (e.g., using a held-out calibration set of different questions) would have provided a fairer practical baseline.

6. Limitations and Trade-offs

The Difficulty Estimation Cost Is Untreated, Making the Benchmark a Diagnostic Tool Rather Than a Deployable Pipeline

The assumption or constraint. The paper's contribution is a benchmarking framework for evaluating distributional alignment—it measures how well models can match opinion distributions under varying design choices. However, the paper explicitly does not account for the cost of selecting the best configuration for a given use case. The optimal configuration (verbalization + few-shot steering) requires practitioners to have access to prior ground-truth distributional data for similar questions to construct the few-shot examples. The paper acknowledges this directly: the few-shot steering condition represents "instances where practitioners have access to existing survey data for similar questions" (Section 3.2). For the NYT Book Opinions dataset, this required collecting annotation data from 346 annotators across 235 books, a substantial data collection effort. For OpinionQA and GlobalOpinionQA, the ground-truth distributions come from nationally representative surveys conducted by Pew Research Center and the World Values Survey—institutional-scale efforts that individual practitioners cannot replicate.

The consequence. A practitioner seeking to deploy LLM-based survey simulation for a novel domain (e.g., consumer product preferences, entertainment media preferences, policy attitudes in a specific community) would not have the ground-truth distributional data needed to construct few-shot examples. The paper provides no guidance on what to do in this regime: should the practitioner fall back to persona-only steering (which produces stereotypical outputs, Figure 5)? Should they collect a small pilot survey to construct few-shot examples, and if so, how many examples are needed? The paper also does not evaluate how performance degrades as the number of few-shot examples decreases—all experiments use exactly five examples, with no ablation on example count. A practitioner with access to only 1–2 examples from a prior survey wave has no basis for estimating how much alignment improvement to expect.

What evidence exists in the paper. Figure 4 and the full steering results (Tables 8–10 in Appendix A.13) demonstrate that few-shot steering consistently outperforms persona steering across models, datasets, and demographic groups—but this comparison assumes the existence of the few-shot data. The paper's human baseline required 246 annotators for OpinionQA and 374 annotators for NYT Book Opinions, each completing multiple questions (Appendix A.6)—this annotation cost is not factored into any comparison between models and humans. The paper never evaluates a "limited data" regime where few-shot examples are scarce, nor does it propose a method for selecting an expression method or steering approach without access to ground-truth labels for evaluation.

Mitigation status. The paper does not address this limitation. It does not propose a method for estimating distributional alignment quality without ground-truth labels, does not provide a calibration procedure that works without oracle access to true distributions, and does not evaluate how few-shot performance scales with the number of provided examples. A practitioner reading this paper learns that verbalization + few-shot is the best current approach, but receives no guidance on what to do when few-shot examples are unavailable or when the domain is sufficiently novel that prior survey data does not exist.

The Knowledge-to-Simulation Gap Is Partially an Artifact of the Sequence Method's Discretization Error, and the Reported Magnitudes Are Therefore Inflated

The assumption or constraint. The knowledge-to-simulation gap (Equation 2) compares the model's total variation error when emitting a 30-token sequence against its error when verbalizing percentages directly. The sequence method has an inherent discretization error floor: even if the model perfectly intended to sample from the ground-truth distribution, the finite sequence of 30 tokens would produce an empirical frequency distribution that deviates from the true distribution by approximately 0.115–0.136 total variation depending on the dataset (Table 1a, Table 10). This error is irreducible—no amount of model improvement can push the sequence method's total variation below this floor, because it arises from finite-sample approximation rather than model misalignment.

The consequence. The reported knowledge-to-simulation gap values in Table 1b conflate two distinct sources of error: (1) genuine simulation miscalibration (the model actually samples from a distribution different from what it knows) and (2) the irreducible discretization noise from the 30-token finite sample. For a model like Anthropic Opus (verbalization: 0.226, sequence: 0.325, gap: 43.63%), subtracting the discretization error floor of 0.115 from the sequence result yields 0.325 − 0.115 = 0.210, which is actually lower than the verbalization error of 0.226. This would imply no genuine simulation penalty—Opus's sequence emission might be perfectly calibrated to its knowledge, with all additional error attributable to finite-sample noise. The paper does not perform this decomposition or discuss whether the discretization error should be accounted for before interpreting the gap. The gap metric as defined measures the total operational penalty of using sequences rather than verbalization, but it does not isolate the simulation-specific component of that penalty from the estimation component, which limits its diagnostic value. A practitioner who subtracts the discretization floor from their sequence results might reach different conclusions about whether their model has a simulation problem or an estimation problem.

What evidence exists in the paper. The paper reports the discretization error explicitly (Table 1a: 0.115 ± 0.004 for the combined leaderboard) and notes that this represents an error floor. However, the text does not discuss the relationship between the discretization error and the knowledge-to-simulation gap. The gap values in Table 1b are computed directly from the raw total variation scores without any adjustment for the floor. The biased coin flip experiment (Section 3.1, Figure 2) demonstrates a genuine simulation gap in a setting with no discretization issues (the 30-token sequence in the coin flip experiment also has finite-sample noise, but the comparison in Figure 2 shows the sequence method closely tracking the true probability across all $p$ , suggesting the miscalibration for coin flips is genuinely smaller than for survey responses). This suggests that part of the gap is real, but the paper provides no method for decomposing how much of the gap in the survey setting is due to simulation miscalibration vs. the discretization floor.

Mitigation status. Not addressed. The paper could have reported the gap with the discretization error subtracted (i.e., $\text{KSS}_{\text{adjusted}} = (A_{\text{Seq}} - \text{disc\_error}) / A_{\text{Verb}} - 1$ ) to isolate the simulation-specific penalty, or could have used longer sequences to reduce the discretization floor and recomputed the gap. Neither approach is attempted. The paper also could have varied sequence length to show how the gap changes as the discretization error decreases, which would directly reveal whether the gap is primarily driven by finite-sample noise or by genuine sampling biases. This is a missed opportunity to strengthen the paper's central diagnostic concept.

All Evaluations Are on a Single Task Format (Closed-Ended Multiple-Choice Survey Questions), Leaving Open-Ended Opinion Simulation Unaddressed

The assumption or constraint. The entire benchmark—all three datasets, all three expression methods, all human and model evaluations—operates on multiple-choice survey questions where respondents select from a fixed set of answer options. The paper acknowledges this explicitly in the Limitations section:

"Our analyses are restricted to opinions expressed in multiple-choice format, which can collapse the nuances of opinions and alter the opinion expressed, as LLMs have also been shown to express different opinions when prompted to respond with open-ended text" (Wang et al., 2024c; Lyu et al., 2024)

The consequence. Many realistic applications of LLM-based human simulation involve open-ended responses: survey pretesting often uses open-ended questions to identify unexpected response categories; agent-based simulations require agents to produce natural-language dialogue, not multiple-choice selections; and synthetic data generation for qualitative research requires free-text responses. The paper provides no evidence about whether the knowledge-to-simulation gap, the advantage of verbalization over log-probabilities, or the benefit of few-shot over persona steering would generalize to open-ended settings. Worse, the paper identifies specific challenges that prevented them from extending the benchmark to open-ended responses:

"we have found complex challenges in studying long-form opinions, such as (1) strict refusal policies that limit an LLM's ability to generate long-form responses to potentially harmful or generally controversial questions... (2) challenges in defining the input distribution... (3) long-form measurement of opinions encounters the same challenges as the automated evaluation of open-ended text generation, including cost, construct validity, and bias"

These are not difficulties that the paper solves; they are reasons the paper's approach may not extend to the very settings where LLM simulation would be most practically useful. A practitioner needing to generate synthetic open-ended survey responses cannot apply the distributional alignment benchmark or its findings directly.

What evidence exists in the paper. The limitation is acknowledged in Section 8 (Limitations), but no experiments address the multiple-choice vs. open-ended gap. The argument for focusing on closed-ended questions is practical—leveraging existing datasets, enabling precise and reproducible evaluation, and allowing calibration techniques to be applied—but this argument does not mitigate the scope limitation. The paper cites external work showing that "LLMs have also been shown to express different opinions when prompted to respond with open-ended text" (Wang et al., 2024c; Lyu et al., 2024), which directly suggests that findings from the multiple-choice setting may not transfer to open-ended formats.

Mitigation status. The paper acknowledges the limitation and provides methodological justification for the constraint (Section 8), but does not propose any approach for extending the framework to open-ended settings. The knowledge-to-simulation gap concept could in principle be extended—verbalization would become "describe the distribution of opinions this group would express in free text," and sequence emission would become "generate 30 free-text responses." However, the evaluation problem for open-ended responses (how to measure whether a set of generated responses matches the distribution of human responses) is not addressed. This is flagged as future work implicitly by the paper's acknowledgment of the challenges.

The Benchmark Evaluates Only Five Models from Three Developers on English-Language, Western-Centric Surveys, Limiting Claims About Generality

The assumption or constraint. All five evaluated models (GPT-4, GPT-3.5, Anthropic Opus, Anthropic Haiku, Llama-3 70B) are large instruction-tuned models from US-based developers, trained predominantly on English-language data, and evaluated exclusively on English-language survey questions. The three datasets cover US political opinions (OpinionQA), cross-national cultural values as perceived through Western-designed survey instruments (GlobalOpinionQA), and US annotators' book preferences (NYT Book Opinions). The paper acknowledges a related limitation about demographic scope:

"Beyond evaluating six demographic groups for OpinionQA and four demographic groups for NYT-Books, there are many other demographic groups that we have not yet explored."

However, the acknowledgment does not address model diversity or linguistic diversity.

The consequence. The paper's findings about which expression methods work best, how large the knowledge-to-simulation gap is, and whether few-shot steering improves over persona steering may be specific to English-language instruction-tuned models from a particular era of LLM development. A model trained with different RLHF procedures, a model trained primarily on non-English data, or a model from a different cultural context might exhibit different calibration properties, different stereotype patterns, or different sensitivity to steering methods. The paper's claim that the biased coin flip experiment reveals "a larger knowledge-to-simulation gap between model log-probabilities and the verbalization of knowledge" (Section 4.2) implicitly generalizes across all current LLMs, but the evidence comes from only five models, with the anomalous behavior of GPT-3.5 (which shows no clear benefit from few-shot steering on NYT Book Opinions, Table 10, and has the smallest knowledge-to-simulation gap at 9.17%, Table 1b) demonstrating that findings can be model-specific.

Furthermore, the GlobalOpinionQA dataset was filtered to 19 countries with sufficient data and evaluated only for countries with at least 600 responses. The 19 countries represent a fraction of the original 138, and the filtering criteria may have selected for countries with more robust survey infrastructure and stronger presence in English-language training data—precisely the countries where LLMs would be most likely to perform well. The paper does not analyze whether model performance correlates with a country's representation in training data, GDP, or English proficiency.

What evidence exists in the paper. Table 7 in Appendix A.9 shows that smaller open-source models (Llama-2 variants, Llama-3-8B, Deepseek-coder) could not reliably follow the prompt format for the sequence and verbalization methods, with success rates as low as 0%. This means the benchmark cannot evaluate models below a certain capability threshold—the findings about distributional alignment are conditional on using large, instruction-tuned models that can produce structured output. Whether smaller or differently-architected models might exhibit different patterns is unknown. The paper also shows that temperature scaling helps GPT models substantially more than Llama-3-70B (Table 3, Figure 6 in Appendix A.2), which directly demonstrates model-specific behavior in a core result.

Mitigation status. The paper is transparent about evaluating "GPT-4, GPT-3.5, Anthropic Haiku, Anthropic Opus, Llama-3 70B Instruct" (Section 4), and the Limitations section acknowledges the restricted scope of demographic groups. However, the paper does not discuss the limitation in terms of model diversity, language diversity, or cultural diversity of the evaluation instruments themselves. The GlobalOpinionQA exclusion of the human baseline (Section 3.4) implicitly acknowledges the cultural specificity problem for human annotations, but the same reasoning is not applied to model evaluation: if Western annotators cannot reliably estimate non-Western opinion distributions, on what basis do we trust that Western-developed LLMs can? The paper does not engage with this question.

The Human Baseline Matches LLM Performance But Is Not Calibrated to Represent a Meaningful Sufficiency Threshold

The assumption or constraint. The paper recruits crowd workers to estimate opinion distributions and compares their performance to LLMs, finding that the best models with optimal expression methods achieve total variation scores slightly better than humans (Table 2: GPT-4 verbalization at 0.204 vs. humans at 0.250). The paper interprets this not as evidence that LLMs have surpassed human performance, but as evidence that the task remains unsolved because humans are known to be poor at it:

"this is not particularly promising for the field of distributional alignment given that humans are known to be poor predictors of opinions of the opposite party" (Yudkin et al., 2019; Levendusky and Malhotra, 2015)

The consequence. The human baseline, as constructed, cannot distinguish between three importantly different interpretations: (1) LLMs have reached human-level performance and are therefore ready to substitute for humans in survey simulation (the optimistic interpretation the paper rejects); (2) LLMs and humans perform similarly because both are fundamentally limited by the same information constraint—the difficulty of inferring group opinion distributions from general knowledge—and neither is adequate for deployment (the paper's preferred interpretation); (3) LLMs and humans perform similarly because the specific human baseline collected is weak relative to what a motivated, knowledgeable human could achieve, and a stronger human baseline would outperform LLMs substantially. The paper provides no evidence that the human baseline represents an upper bound on what humans can achieve at this task. Professional pollsters, domain experts with survey methodology training, or individuals with access to reference materials might substantially outperform the Prolific crowd workers evaluated here. If a stronger human baseline achieved, say, 0.10 total variation while LLMs remained at 0.20, the conclusion would shift from "LLMs match a weak baseline" to "LLMs substantially underperform a strong baseline."

What evidence exists in the paper. The human annotation protocol is described in Section 3.4 and Appendix A.6. Annotators were Prolific workers paid $12/hour, filtered for English fluency and 93% attention check accuracy, and provided with instructions and examples of the distributional estimation task (Figures 9–11). This is a reasonable crowdsourcing setup, but the paper provides no calibration of how these annotators' performance compares to expert performance, to performance with unlimited time and resources, or to the best possible human performance given the available information. The in-group vs. out-group analysis (Table 6, Appendix A.6) finds no statistically significant difference—in-group estimates are not reliably better than out-group estimates—which is consistent with the Perception Gap literature's finding that humans are poor at this task in general, but also suggests that the collected annotators may not have been sufficiently familiar with the specific survey domains to leverage in-group knowledge effectively.

Mitigation status. The paper does not attempt to construct a stronger human baseline (e.g., political science graduate students, professional pollsters, or annotators allowed to search for relevant reference data). It also does not establish a performance threshold that would constitute "sufficient for deployment"—at what total variation value would LLM-based survey simulation be considered reliable enough to substitute for human data collection? Without such a threshold, the fact that LLMs approximately match human performance on the same task is uninterpretable as a deployment recommendation. The paper's conclusion that "distributional alignment remains fundamentally unsolved" is a judgment the evidence supports but does not quantify.

The Benchmark Does Not Measure or Account for Distribution Expression Methods That Fail to Produce Valid Outputs, Restricting the Evaluated Model Set to Large Instruction-Tuned Models

The assumption or constraint. The benchmark requires models to produce outputs in specific structured formats: a 30-token sequence of answer choice letters for the sequence method, or a valid JSON string with percentage values for the verbalization method. The paper reports that many models—particularly smaller and older ones—fail to follow these output format instructions reliably:

"Smaller models struggled to follow the sequence distribution expression method, thus restricting our model selection" (Section 4)

Appendix A.9, Table 7 quantifies this: Llama-3-8B achieves only 45% success on the sequence method and 40% on the verbalization method. Llama-2-70B achieves 3% on sequence. Deepseek-coder-6.7B achieves 0% on both. The paper therefore evaluates only the five largest models that could reliably follow instructions.

The consequence. The benchmark's findings—that verbalization outperforms sequence emission, that the knowledge-to-simulation gap exists, that few-shot steering improves over persona steering—apply only to models that are already capable enough to produce valid structured outputs. This is a form of survivorship bias: the models that can even be evaluated are those that have already crossed a capability threshold that smaller models have not. The paper cannot determine whether distributional alignment is a capability that emerges only at a certain scale, or whether smaller models might perform well at the task if the output format were adapted to their capabilities (e.g., allowing free-text descriptions of distributions rather than strict JSON). Furthermore, the exclusion of models below the capability threshold means the paper provides no evidence about whether the knowledge-to-simulation gap is a property of all LLMs or only of large instruction-tuned models—smaller models might have a smaller gap (because their simpler training procedures produce less RLHF-induced miscalibration) or a larger gap (because they have weaker output control). Without evaluating models at the boundary where format-following becomes reliable, the paper cannot characterize how the phenomena it studies scale with model capability.

What evidence exists in the paper. Table 7 in Appendix A.9 reports the success rates for eight additional models. The paper is transparent about this limitation: it explicitly notes that "smaller models struggled to follow the sequence distribution expression method, thus restricting our model selection." However, it does not discuss the implications of this restriction for the generality of its findings. It also does not attempt alternative prompting strategies that might improve format adherence for smaller models (e.g., few-shot examples of the output format, chain-of-thought prompting before structured output, or iterative refinement).

Mitigation status. The paper acknowledges the limitation but does not address it. The restriction to five large models is a practical necessity given the format requirements of the benchmark, but the benchmark's format requirements are themselves a design choice. An alternative evaluation approach—for instance, extracting answer choice distributions from free-text model responses through parsing or LLM-based evaluation—could have included smaller models and tested whether the knowledge-to-simulation gap scales with model size. The paper does not explore this design tradeoff or suggest how future benchmarks might accommodate a broader range of model scales.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper performs a methodological reframing rather than a paradigm shift. It does not introduce a new model, training procedure, or deployment architecture. Instead, it demonstrates that the answer to "can LLMs simulate human opinion distributions?" depends substantially on how one asks the question—specifically, on choices about the distribution expression method, steering method, and question domain that prior work has neither systematically varied nor recognized as interdependent. The magnitude of this reframing is significant: the paper provides evidence that the canonical evaluation pipeline in the field (log-probabilities with persona steering on political survey questions) is a worst-case configuration that systematically underestimates what models can do. A researcher using that pipeline would conclude that GPT-4 is worse than a uniform baseline at distributional alignment (Table 1a: 0.550 vs. 0.363 total variation); the same researcher, switching to verbalization, would find GPT-4 among the best-performing configurations (0.229, ranked #2 on the leaderboard). This is not an incremental improvement in method—it is a demonstration that the measurement apparatus itself has been producing misleading readings.

The paper resolves a latent contradiction in the literature that the authors identify explicitly in Section 1: some work argues LLMs can faithfully simulate demographic groups (Grossmann et al., 2023), while other work shows such simulations are inaccurate and stereotypical (Liu et al., 2024; Wang et al., 2024a). The resolution this paper offers is not that one side is correct and the other wrong, but that both conclusions are artifacts of different measurement choices applied to different difficulty regimes. The optimistic findings come from evaluating models in the domain where alignment is easiest (political opinions, which are strongly signaled by demographic identity markers and extensively represented in training data) using methods that, while imperfect, do not exhibit the worst calibration pathologies. The pessimistic findings come from evaluating models using log-probabilities (which RLHF training has miscalibrated) on domains where demographic associations are weaker or where persona-only steering elicits stereotypes rather than accurate distributions. The paper's framework provides a vocabulary for articulating which conditions produce which conclusions, transforming an apparent contradiction into a set of boundary conditions.

The paper shifts research attention in two specific directions, both of which become more attractive than they were before this work:

Toward measurement methodology rather than model capability improvement. Before this paper, a natural response to finding that LLMs poorly match human opinion distributions would be to train better models—larger pretraining datasets, improved RLHF procedures, or specialized fine-tuning. The paper's evidence suggests that a substantial fraction of the observed misalignment is not a capability deficit but a measurement artifact: models possess distributional knowledge (as demonstrated by verbalization) that existing evaluation methods (log-probabilities) fail to surface. This redirects effort from improving models to improving how we elicit and measure what models already know. Work on better prompting strategies, output format design, and calibration techniques becomes higher-priority relative to work on larger-scale pretraining.

Toward domain-specific evaluation rather than universal claims about LLM simulation capability. By demonstrating that steerability is substantially lower for NYT Book Opinions than for OpinionQA (Figure 4), and that models exhibit stereotypical behavior in the non-political domain (Figure 5), the paper establishes that distributional alignment is not a monolithic capability that transfers uniformly across domains. This makes it more difficult—and less responsible—to make general claims about whether "LLMs can simulate humans" based on evaluations in a single domain. Future work claiming that LLMs are adequate for human simulation must specify for what kinds of questions and what kinds of demographic groups, and evaluations confined to political surveys cannot support claims about consumer preference simulation, market research, or other non-political use cases.

Some research directions become less attractive as a result of this paper. The straightforward extension of the OpinionQA paradigm—evaluating more models and more demographic groups using log-probabilities—is shown to produce results that may be dominated by measurement artifact rather than genuine capability differences. The paper's evidence that log-probabilities systematically underestimate alignment (and that temperature scaling, an oracle method, partially rescues them) means that incremental work using log-probability evaluation without addressing the expression method question is likely to produce findings that are artifacts of the measurement pipeline rather than discoveries about model capabilities. Similarly, persona-only steering without comparison to few-shot alternatives is shown to produce stereotypical distributions (Figure 5) that may reflect prompting limitations rather than model knowledge limitations—work that evaluates persona steering in isolation without a few-shot comparison or a verbalization baseline risks misattributing prompting failures to model incapabilities.

Follow-Up Research This Work Enables

Closing the knowledge-to-simulation gap through improved sampling methods. The paper identifies and names a specific gap—models can state distributions they cannot sample—but does not propose methods for closing it. A natural next step is to develop decoupled simulation pipelines that leverage the paper's finding that verbalization is substantially more accurate than sequence emission. The simplest such pipeline would: (1) prompt the model to verbalize the distribution in JSON format; (2) parse the JSON to extract numerical probabilities; (3) use an external pseudorandom number generator to draw samples from that distribution. This would bypass the autoregressive sampling step entirely, eliminating the 43.6% error penalty observed for Anthropic Opus and the 34.7% penalty for Llama-3-70B. A strong follow-up would evaluate this decoupled pipeline against direct sequence emission on the same datasets and models, measuring whether the verbalize-then-sample-externally approach recovers the verbalization-level alignment performance at the sampling cost of the sequence method. If successful, this would provide an immediate practical improvement for any practitioner using LLMs to generate synthetic survey responses, without requiring any model retraining or fine-tuning. A negative result—finding that externally-sampled responses from verbalized distributions still fail to match human distributions in some important way—would indicate that the verbalized distributions themselves are flawed in ways the total variation metric does not capture (e.g., missing correlations between answers across questions, or failing to represent within-group heterogeneity beyond aggregate percentages).

Difficulty estimation: characterizing which questions produce the largest knowledge-to-simulation gap. The paper reports aggregate gap values across all questions but does not analyze whether the gap is concentrated in particular question types or is uniform. A diagnostic follow-up would compute the knowledge-to-simulation gap per question and then characterize the questions where the gap is largest. Candidate explanatory variables include: the entropy of the ground-truth distribution (do models struggle to sample from high-entropy distributions where many answer choices have non-trivial probability?); the number of answer choices (does the gap grow with more choices, as the autoregressive generation has more opportunities to drift?); the degree of intergroup disagreement on the question (does the gap shrink when groups are highly polarized, because the distribution becomes more concentrated and easier to sample?); and the topic domain (is the gap larger for questions about science vs. politics vs. personal relationships?). This analysis would reveal whether the gap is a general sampling pathology or a symptom of specific distributional properties that the model's autoregressive process handles poorly. If the gap is concentrated in high-entropy, many-answer-choice questions, practitioners could apply the decoupled pipeline selectively—using external sampling only for questions above an entropy threshold, and relying on direct sequence emission for questions with concentrated distributions where the gap is small. The paper's existing data—with 100 questions per dataset, five models, and both verbalization and sequence results for each—is sufficient to conduct this analysis without new data collection, making this a low-cost, high-insight follow-up.

Quantifying how many few-shot examples are needed to saturate the improvement over persona steering. The paper uses exactly five few-shot examples in all experiments, finding that this consistently outperforms persona steering (Figure 4). However, the paper does not vary the number of examples. A scaling analysis—evaluating distributional alignment with 0 (persona-only), 1, 2, 3, 5, 10, and 20 few-shot examples—would answer two practically important questions. First, how many examples are enough? If most of the improvement occurs between 0 and 3 examples and additional examples yield diminishing returns, practitioners with limited prior survey data can invest in collecting a small number of reference distributions and expect near-optimal performance. If improvement continues linearly or logarithmically with example count, the practical recommendation would differ: invest in collecting as many examples as feasible. Second, does the benefit of few-shot examples interact with the distribution expression method? The paper's few-shot results are reported for the sequence method (Figure 4), but it is unknown whether verbalization benefits more or less from few-shot examples than sequence emission does. If verbalization's advantage over sequence emission shrinks when many few-shot examples are provided (because the examples provide enough distributional information to calibrate the sampling process), then in high-data regimes the choice of expression method may matter less. If verbalization's advantage persists regardless of example count, then the expression method recommendation (prefer verbalization) holds universally. This experiment requires no new datasets or models—only varying a prompt parameter and recomputing the existing metrics.

Extending the NYT Book Opinions finding to other non-political preference domains. The paper demonstrates that distributional alignment is harder for book preferences than for political opinions, but the finding is based on a single non-political domain. To determine whether this is a general phenomenon or specific to books, one would construct analogous preference datasets in other domains: music preferences (e.g., "Given this description of an album, how likely are you to listen to it?"), film preferences, restaurant or food preferences, hobby and leisure activity preferences, or consumer product preferences. Each domain would require collecting ground-truth human annotations from demographically identified annotators, following the NYT Book Opinions annotation protocol. A consistent finding across multiple non-political domains—that steerability is reduced relative to OpinionQA, that persona steering produces stereotypical distributions, and that the knowledge-to-simulation gap is comparable—would establish the domain-dependence of distributional alignment as a robust phenomenon. A finding that some non-political domains (e.g., music preferences) show strong steerability while others (e.g., book preferences) do not would suggest that the relevant variable is not "political vs. non-political" but something else—perhaps the strength of demographic signaling in the domain's public discourse, or the availability of explicit distributional information in training data. This would refine the paper's diagnostic framework from a binary domain distinction to a more nuanced characterization of which properties make a domain amenable to LLM-based simulation.

Training verifier models to predict distributional alignment quality without ground-truth labels. A fundamental limitation of the paper's framework is that evaluating distributional alignment requires ground-truth human distributions, which are expensive to collect and unavailable in novel domains. A practically valuable follow-up would train a distributional alignment quality estimator—a model that takes as input a question, a target demographic group, and an LLM's predicted distribution (from any expression method), and outputs an estimate of the total variation distance between that predicted distribution and the (unknown) ground truth. Such an estimator could be trained on the paper's existing data: for each of the 100 OpinionQA questions × 6 groups × 5 models × 3 expression methods × 3 steering methods ≈ 27,000 (question, group, model, method, steering) tuples, the ground-truth total variation is already computed. A regression model trained on features of the question (topic, number of answer choices, embedding), the predicted distribution (entropy, concentration, agreement with other models' predictions), and the configuration (expression method, steering method, model identity) could predict the alignment error. If accurate, this estimator would allow practitioners to assess whether their LLM-based simulation is reliable for a given domain without collecting ground-truth labels—they would simply run the estimator on their model's outputs and receive a predicted error bound. This would transform the paper's framework from a retrospective benchmarking tool into a prospective deployment diagnostic. The key evaluation would be whether the estimator generalizes across datasets: trained on OpinionQA, does it predict alignment quality accurately on NYT Book Opinions? If not, domain-specific estimators may be needed, which would be a practically important finding about the limits of transfer.

Stress-testing the knowledge-to-simulation gap under distribution shift: model updates, prompt variations, and temporal drift. The paper evaluates a snapshot of five models at a single point in time using fixed prompts. An important robustness check—with implications for whether the gap is a stable property of model families or an ephemeral artifact of specific training runs—would measure how the gap changes under three types of variation. First, model updates: evaluate the same model family before and after an RLHF tuning iteration or a major version update (e.g., GPT-4 initial release vs. GPT-4 after several months of deployment). If the gap widens with additional RLHF (consistent with the hypothesis that RLHF miscalibrates log-probabilities while preserving verbalized knowledge), this is evidence of a systematic tradeoff in current alignment techniques. Second, prompt sensitivity: vary the phrasing of the verbalization and sequence instructions (e.g., "estimate the percentage of Democrats who would choose each option" vs. "what fraction of Democrats believe each of the following" vs. "return a probability distribution over answer choices for Democrats") and measure whether the gap is stable or prompt-dependent. If small prompt changes produce large gap changes, the phenomenon is less fundamental and more an artifact of specific instruction formats. Third, temporal drift: compare model estimates of opinion distributions against survey data from different time periods (e.g., PEW surveys from 2018 vs. 2022). If the knowledge-to-simulation gap grows for older survey data relative to newer data (because the model's training data is more representative of recent opinions), this would indicate that the gap is partly a staleness problem—the model knows outdated distributions and simulation compounds the staleness error. These stress tests would distinguish whether the gap is a deep property of autoregressive language models or a surface-level calibration issue specific to current instruction-tuning practices.

Practical Applications and Downstream Use Cases

Two-stage synthetic survey respondent generation for social science pretesting. A social scientist designing a new survey instrument wants to pretest questions before expensive human data collection. The paper's findings directly recommend a two-stage pipeline: first, for each demographic group of interest, prompt the LLM (using few-shot examples if prior survey data exists for similar questions, otherwise falling back to persona steering with the caveat that stereotypical distributions may result) to verbalize the expected answer distribution in JSON format. Second, use an external pseudorandom sampler to draw individual synthetic responses from that distribution, generating a synthetic dataset of arbitrary size without the 43.6% simulation penalty observed for Anthropic Opus or the 34.7% penalty for Llama-3-70B when using direct sequence emission. The scientist can then analyze the synthetic data for floor/ceiling effects, test whether answer distributions vary as expected across experimental conditions, and identify questions where the expected distribution is pathologically concentrated (suggesting the question may not discriminate between respondents). The benefit relative to current practice—which typically uses either raw model samples (subject to the knowledge-to-simulation gap) or no pretesting at all—is that the verbalize-then-sample pipeline recovers the model's best available distributional knowledge (total variation ~0.20-0.23 for GPT-4 and Opus on OpinionQA) rather than degrading it through miscalibrated sampling. The cost is minimal: verbalization requires a single API call per question per group, and external sampling is computationally free.

Auditing LLM-based simulation systems for stereotyping before deployment. An organization deploying LLMs to generate synthetic survey responses or populate agent-based simulations needs to ensure the system does not systematically misrepresent specific demographic groups. The paper's disaggregated analysis approach (Section 4.2, Figure 5) provides a template: for each demographic group and question domain, compute the marginal distribution of model-generated responses and compare it against available ground-truth human distributions (even if only from pilot data or published surveys). The paper's finding that persona-steered models overestimate Democrat reading interest by 20 percentage points (13% vs. 33% "very unlikely to read") and underestimate Republican reading disinterest—a stereotypical distortion—was detected by this marginal distribution comparison. Organizations can implement this as an automated audit: before deploying a simulation system, generate responses for all target demographic groups, compute marginal distributions over answer choices, and flag any group–answer combination where the model's predicted probability deviates from the available reference by more than a threshold (e.g., 15 percentage points). The key practical insight from the paper is that this audit should be performed on the output of the simulation pipeline as deployed (whether sequence emission or decoupled external sampling), not on verbalized distributions (which may hide the stereotyping that emerges during sampling). If the audit reveals systematic stereotyping, the organization can mitigate it by switching to few-shot steering (Figure 5, green bars, showing substantial reduction in the Democrat reading interest stereotype) or by calibrating the verbalized distribution against reference data before sampling.

Selecting between LLM-based simulation and human data collection based on domain difficulty. A market research firm is deciding whether to use LLMs to simulate consumer preferences for a new product category or to invest in a traditional survey panel. The paper's finding that steerability is domain-dependent (Figure 4)—alignment is substantially better for directly value-laden questions (OpinionQA) than for preferences expressed through a layer of abstraction (NYT Book Opinions)—provides a decision heuristic. If the product category has strong demographic signaling (e.g., political merchandise, where preferences are highly correlated with political identity and extensively discussed in public discourse), LLM-based simulation may approach the alignment levels observed on OpinionQA (total variation ~0.18-0.24 with verbalization + few-shot, Table 10). If the product category involves subtle, non-politicized preferences (e.g., book genres, as in the NYT dataset; or analogous domains like music subgenres, hobby interests, or aesthetic preferences), the firm should expect alignment closer to the NYT Book Opinions results (total variation ~0.21-0.25 with the best methods, Table 10, but with larger stereotype effects and less benefit from steering). Critically, the paper's human baseline (Table 2) shows that even humans are poor at estimating distributions in these domains (~0.25 total variation for NYT + OQA combined), so the relevant comparison is not "LLMs vs. perfect ground truth" but "LLMs vs. the alternative methods available within the project's budget and timeline." If the alternative is no data at all (because a survey panel is too expensive or too slow), LLM-based simulation with the best available configuration (verbalization, few-shot if prior data exists) may provide useful directional information despite its errors. If the alternative is a modestly-sized human panel that can directly measure preferences, the paper's findings suggest the human panel will likely produce more accurate distributions—but at higher cost and longer timeline, a tradeoff the paper quantifies but does not resolve.

Improving the calibration of model log-probabilities for applications where sampling is unavoidable. Some downstream applications cannot use the verbalize-then-sample-externally pipeline because they require the model to make online probabilistic decisions—for instance, an LLM-based agent in a simulation that must decide in real-time whether to take an action based on its estimate of how a demographic group would respond. In these settings, the model's token-level probabilities (log-probabilities) are the only available distributional signal at decision time. The paper's temperature scaling analysis (Table 1a, Table 3, Figure 6 in Appendix A.2) shows that applying a learned temperature parameter $\tau$ can substantially improve GPT-4's log-probability calibration (from 0.550 to 0.273 total variation) while providing minimal benefit for Llama-3-70B (0.495 to 0.469). The practical implication is that practitioners using GPT-family models for online probabilistic simulation can apply temperature scaling with $\tau$ values calibrated on a held-out set of questions (using available ground-truth distributions from similar domains) to recover a substantial fraction of the model's distributional knowledge that raw log-probabilities obscure. The paper provides the formula (Equation 3) and demonstrates that the optimal $\tau$ varies by dataset and steering method—a practitioner would need to calibrate $\tau$ for their specific domain, not use a universal constant. For Llama-family models, temperature scaling provides minimal benefit, and practitioners using these models for online probabilistic simulation should expect substantially degraded distributional alignment and should consider architectural alternatives (e.g., having the model verbalize a distribution and then sample from it externally, accepting the latency cost of an extra API call) rather than relying on log-probabilities.