Steering Language Models with Game-Theoretic Solvers

ArXiv: 2402.01704

🎯 Pitch

Large language models can be made less exploitable in strategic dialogue by having game-theoretic solvers compute equilibrium strategies over abstract actions like "angry" or "enthusiastic," then feeding those strategies back to the LLM as guidance—and a neural network trained to mimic these equilibrium policies beats the unguided baseline 76% of the time in fruit trading, though it curiously fails in meeting scheduling when historical dialogue embeddings prove insufficient.

1. Executive Summary

This paper introduces a framework that binds natural language dialogue to the formal machinery of extensive-form game theory, enabling equilibrium solvers to guide large language models toward strategic conversational behavior. Across three procedurally generated dialogue domains—scheduling meetings, trading fruit, and public debate—using PaLM 2 models, the authors evaluate two solver mechanisms: Counterfactual Regret Minimization (CFR, which computes an equilibrium strategy over a fixed set of dialogue actions such as tones or argument styles) and Policy-Space Response-Oracles (PSRO, which iteratively expands the action set by generating novel instructions like "angry" or "enthusiastic" as approximate best responses). A CFR-guided LLM achieves positive CFR Gain in all three domains (e.g., +0.106 in debate, +0.038 in fruit trading) while maintaining low NashConv values, establishing that solver-steered LLMs are less exploitable than unguided baselines—and a neural network trained via imitation learning to predict CFR equilibrium policies outperforms the baseline LLM ~76% of the time in fruit trading, though this advantage reverses in meeting scheduling when the embedding of dialogue history carries insufficient information for predicting optimal actions.

2. Context and Motivation

The Core Problem: LLMs Generate Plausible Language Without Strategic Coherence

This paper addresses a fundamental disconnect between two research traditions that should, in principle, be deeply complementary. On one side, large language models have achieved remarkable fluency in natural language generation, producing text that appears human-like across conversational tasks. On the other side, game theory provides a rigorous mathematical framework for reasoning about optimal behavior in multi-agent interactions where outcomes depend on the choices of others. The problem is that LLMs, despite their surface plausibility, often fail to exhibit coherent and consistent reasoning strategies when placed in strategic settings. The paper cites evidence for this: Jiang et al. (2020) show that LLMs struggle with rational reasoning even when their outputs appear plausible, and Turpin et al. (2024) demonstrate that methods like chain-of-thought prompting are "often unreliable and unfaithful to the generations of models and susceptible to small variations in inputs that drastically alter behavior."

This gap is not merely an academic curiosity. It has concrete implications for any application where an LLM must interact with users or other AI agents in scenarios involving negotiation, persuasion, resource allocation, or argumentation—situations where the quality of the outcome depends on the strategic sophistication of the language generated. An LLM that achieves high BLEU scores or human-likeness ratings but is systematically exploitable by strategic counterparts cannot be trusted in consequential multi-agent deployments.

The problem exists because the space of natural language is vast and unstructured, while traditional game-theoretic models operate over small, discrete action sets—moves in chess, bets in poker, build orders in StarCraft. The paper states this explicitly:

"However, most of these success stories are on games in the colloquial sense of the word. These are card, board, or video games that often have action spaces far smaller than the space of natural language words. Language also comes with the underlying complexities of strategizing about beliefs and intents of players over the semantics of the generated words."

The question the paper poses is therefore: can we create a generic binding between natural language dialogue and the formal machinery of game theory such that equilibrium solvers—which already know how to compute optimal strategies in abstract games—can guide an LLM's language generation toward more strategic outcomes?

Why the Disconnect Exists: The Action Space Problem

Game-theoretic solvers like Counterfactual Regret Minimization (CFR) and its variants have driven superhuman performance in poker, Diplomacy, and StarCraft by systematically searching over possible action sequences and identifying strategies that minimize exploitability. But these successes depend on a property that natural language dialogue does not naturally possess: a finite, enumerable action space that can be explicitly represented in a game tree. In poker, the actions are "fold," "call," and "raise" with discrete amounts. In Diplomacy, while the action space is larger, it is still a structured set of move orders over a finite map.

Natural language has no such structure. An LLM can generate any sequence of tokens, and the semantic content of those tokens—the promises, threats, information revelations, arguments, and emotional appeals they contain—constitutes a combinatorially explosive action space. Previous game-theoretic work on language has therefore been forced to either severely restrict the language (e.g., template-based generation with hand-crafted state features) or abandon game-theoretic guarantees in favor of end-to-end learned policies that may not converge to equilibria.

The paper identifies this as the central bottleneck: how do you map the open-ended space of natural language utterances onto the closed, finite game trees that equilibrium solvers require?

Where Prior Approaches Fall Short

The paper situates its contribution by identifying specific limitations in several lines of prior work. These are discussed across Sections 1, 5, and the Limitations section.

End-to-end negotiation agents (Lewis et al., 2017). The most directly comparable prior work is Lewis et al. (2017), which trains end-to-end neural agents for negotiation dialogues. While influential, this approach suffers from several constraints that the current paper aims to overcome. The paper notes that Lewis et al. (2017) assumes "a single game setting with a hard-coded reward model," that "item valuations are observable by all agents," and crucially that "training is performed with a single best-response step due to challenges of convergence (a known pathology within computational game theory)." In other words, the prior work sacrifices several key properties of game-theoretic rigor: private information (imperfect information in game-theoretic terms), flexible reward specification, and convergent multi-step equilibrium computation. The current paper explicitly targets these gaps by designing a framework that supports private valuations, LLM-based reward models that can be specified in natural language, and principled equilibrium solvers with known convergence guarantees.

The paper further positions itself against prior dialogue system work by citing He et al. (2018), who posed desiderata for future work including avoiding "laborious dialogue state design and annotation" and the need to "connect with game theory." The authors argue their framework directly addresses both calls.

In-context reasoning without strategic guarantees. A large body of work has attempted to improve LLM reasoning through prompting techniques—chain-of-thought (Wei et al., 2022), scratchpads (Nye et al., 2021), training on human reasoning traces (Rajani et al., 2019; Shwartz et al., 2020), or bootstrapping from the model's own reasoning (Zelikman et al., 2022). While these approaches can improve reasoning on individual tasks, the paper points to evidence that they are fragile: Turpin et al. (2024) show that such reasoning traces are "often unreliable and unfaithful" and susceptible to adversarial perturbations. More fundamentally, these approaches improve an LLM's reasoning about a fixed problem, but do not address the interactive, adversarial nature of multi-agent strategic settings. A chain-of-thought that reasons soundly about a scheduling problem does not protect an agent from being exploited by a negotiating partner who strategically withholds or misrepresents information—because the prompting approach has no model of the opponent's strategic incentives.

Game-theoretic solvers unconnected to language. On the other side, the game theory community has developed sophisticated solvers (CFR, PSRO, and their many variants) that achieve strong guarantees in board games, card games, and video games. But these solvers operate over symbolic action spaces and require the game's payoff structure to be pre-specified. They have no native ability to generate or understand natural language. Prior efforts to combine game-theoretic reasoning with language, such as the Diplomacy-playing agent Cicero (FAIR et al., 2022), use language as an auxiliary channel alongside a structured game engine—the game board, unit positions, and move orders are the primary objects of strategic reasoning, and language is used to coordinate, persuade, and deceive around a game state that is already formalized. The current paper asks a different question: what if the language itself is the game, rather than a communication layer on top of a separate formal game?

No framework for "dialogue as game." The paper's core observation is that no prior work has provided a general, reusable binding from conversational dialogue tasks to the formal definition of an extensive-form game. Such a binding would need to specify, for any given dialogue domain: (1) what constitutes a player, (2) what information is public vs. private, (3) what the action space is (and how finiteness is achieved), (4) how terminal states are defined, and (5) how payoffs are computed from natural language outcomes. Without this binding, each new dialogue domain requires ad-hoc engineering, and there is no systematic way to apply the full power of equilibrium solvers to language tasks.

How This Paper Positions Itself

The paper does not propose a new equilibrium solver, a new language model architecture, or even a new training objective. Its contribution is architectural and conceptual: it defines a mapping—what the authors call a "binding"—from the elements of conversational dialogue to the tuple ⟨N, c, A, H, Z, u, τ, S⟩ that defines an extensive-form game (Section 2). This binding is designed to be domain-agnostic while making deliberate, explicit design choices about what is finite, what is private, and how utility is computed.

The key design insight is how the binding achieves finiteness of the action space, which is the critical requirement for game-theoretic solvers to operate. Rather than attempting to enumerate possible natural language utterances (which is infeasible), the framework defines actions as instructions from a finite set that steer the LLM's generation. Concretely, an action might be "use an assertive tone" or "propose Wednesday" or "make a logos-style argument." The LLM is then responsible for generating the actual natural language text that instantiates that instruction in context. This decouples strategic decision-making (what instruction to follow) from language generation (how to execute that instruction in fluent text), and it is this decoupling that makes the framework tractable for equilibrium solvers.

This design also creates a natural entry point for iterative action-space expansion via PSRO. Starting from a small, human-specified set of instructions (e.g., "calm," "assertive," "submissive"), the PSRO procedure can generate and evaluate new instructions as approximate best responses, growing the action space to include instructions like "angry," "enthusiastic," or "relaxed" that may not have been anticipated in the initial design. This mechanism, described in Section 3.3 and Algorithm 1, blurs the line between game solving and open-ended language exploration: the solver doesn't just find the best strategy over a fixed action set, it discovers new strategic dimensions of language that exploit weaknesses in the current equilibrium.

The paper's positioning relative to the two bodies of relevant work can be summarized as follows: it borrows the rigor and convergence guarantees of game-theoretic solvers (CFR for fixed-action games, PSRO for expandable action sets) and combines them with the generative fluency of LLMs, but does so through a novel interface layer—the instruction set—that translates between the discrete mathematics of game theory and the continuous, high-dimensional space of natural language. The authors are explicit about this mediating role:

"By doing so in this work, we open the door for a broad body of game theory and multi-agent research—not just on algorithms but also solution concepts and principled strategic reasoning—to pour into the AI guided interactions that pervade humans' daily life."

The paper also positions itself as an inference-time steering mechanism, not a training-time intervention (as noted explicitly in the Limitations section). The LLM itself is frozen; all strategic adaptation happens through the solver's choice of instruction at each turn, combined with the LLM's ability to condition on that instruction in its prompt. This is both a strength (no expensive retraining) and a limitation (it depends on the LLM's instruction-following fidelity, which the paper thoroughly evaluates in Section 4 and Appendix B.1, finding mixed results—PaLM2-L follows "submissive" instructions only 17% of the time in fruit trading, for example).

The Practical and Conceptual Significance

The framework addresses a real deployment challenge. As LLMs are integrated into customer service, virtual assistants, negotiation support, and multi-agent systems, they increasingly find themselves in strategic situations where the optimal utterance depends on what the other party is likely to do. A meeting-scheduling assistant that always reveals its user's availability honestly is exploitable by a counterpart that strategically misrepresents theirs. A debate agent that always uses the same argumentative style is predictable. The paper's framework provides a principled way to equip LLMs with strategic policies that are less exploitable—measured quantitatively through NashConv and CFR Gain (Table 1)—while still operating in natural language.

Conceptually, the work bridges two communities that have largely operated in parallel: the natural language processing community, which has focused on fluency, coherence, and factual accuracy of generation, and the multi-agent game theory community, which has focused on optimality, equilibrium, and exploitability. The binding proposed in Section 2 is, in effect, a conceptual API that allows game-theoretic reasoning to be applied to any dialogue task that can be expressed in the framework's terms, without requiring the NLP researcher to become a game theorist or vice versa.

3. Technical Approach

3.1 Reader Orientation

The paper builds a system that maps a natural language conversation between two LLMs onto a formal extensive-form game tree, allowing off-the-shelf equilibrium-finding algorithms from game theory to compute the optimal strategic instruction (e.g., which argument style to use, which day to propose, which tone to adopt) at each turn of dialogue, and then feeds that instruction back into the LLM's prompt so the LLM generates language consistent with the equilibrium strategy. The core problem it solves is that LLMs produce fluent but strategically naïve dialogue—they don't inherently model their conversation partner's incentives or optimize their utterances to maximize their own payoff in competitive or mixed-motive settings—and the solution has the shape of a mediating instruction layer that sits between the continuous, open-ended space of natural language and the discrete, finite action spaces that game-theoretic solvers require.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components that operate in a loop:

1. Game Configuration — a procedurally generated specification for a particular dialogue instance, encoding each player's private information (e.g., fruit endowments and valuations, meeting day availability and preferences, debate stance and topic), the finite set of actions available (e.g., tones, argument styles, days of the week), the procedure for determining when the conversation terminates, and the instructions for an LLM-based reward model that will compute payoffs at the end of the interaction.

2. Dialogue LLM (DLLM) — a frozen PaLM-2 model that receives a prompt containing the game context (private information), the dialogue history so far, and—crucially—an action instruction string selected by the solver (or the special token "Any" for the unguided baseline). It generates the next natural language utterance in the conversation.

3. Game-Theoretic Solver (SGT) — either Counterfactual Regret Minimization (CFR) operating over a fixed action set, or Policy-Space Response-Oracles (PSRO) which iteratively expands the action set by generating novel instruction strings. The solver takes as input the game tree structure (built from the configuration) and the payoff values at leaf nodes, and outputs a distribution over actions for each information state in the game.

4. LLM Reward Model (RLLM) — a separate frozen PaLM-2 model (specifically PaLM2-L for the main experiments) that takes the full dialogue transcript and private information of both players, processes it through a chain-of-thought prompt, and computes a numerical payoff for each player. This payoff populates the leaf nodes of the game tree, enabling the solver to assign value to different dialogue trajectories.

5. Neural Equilibrium Solver (for imitation learning) — a two-layer MLP trained via supervised learning to predict the CFR-computed equilibrium distribution over actions from a T5 embedding of the information state. At inference time, this neural network replaces the expensive CFR solver, enabling fast equilibrium lookups for new dialogue instances.

Information flows cyclically: the game configuration defines the initial state → the solver (or neural network proxy) selects an action given the current information state → the DLLM generates an utterance conditioned on that action and the dialogue history → the conversation advances to the next state → this repeats until a termination condition is met → the RLLM evaluates the full transcript and assigns payoffs → the solver uses these payoffs to update its value estimates for states and actions.

3.3 Roadmap for the Deep Dive

• First, the binding from dialogue to extensive-form game, because this is the foundational contribution—without it, none of the solver machinery can be applied. We will walk through each element of the game tuple ⟨N, c, A, H, Z, u, τ, S⟩ and explain how it is instantiated for dialogue.
• Second, the action space design, because this is where the key architectural decision lives: actions are not natural language utterances but finite instruction strings that steer the LLM. Understanding why this design choice is necessary, how it enables solver tractability, and what it sacrifices is essential.
• Third, how game trees are built procedurally from dialogue configurations, because this makes concrete how "dialogue as game" actually operates in practice—what chance nodes represent, how branching works, and how termination is determined.
• Fourth, CFR as a solver for fixed-action dialogue games, because it is the primary solver used in the quantitative evaluation (Table 1, NashConv and CFR Gain metrics).
• Fifth, PSRO as a solver for expandable-action dialogue games, because it represents the framework's most forward-looking capability—the ability to discover novel strategic instructions beyond human specification.
• Sixth, the imitation learning pipeline, because it addresses the practical bottleneck of solver computation cost and enables generalization to new game instances.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a conceptual framework paper that proposes a binding between natural language dialogue and extensive-form game theory, then empirically validates that equilibrium-guided LLM generation produces more strategic (less exploitable) dialogue than unguided generation. The core idea is that by defining a finite instruction set as the action space—decoupling strategic choice from language generation—the full machinery of game-theoretic equilibrium solvers becomes applicable to conversational tasks without requiring explicit enumeration of possible utterances.

The Formal Binding: Dialogue as an Extensive-Form Game

The paper defines extensive-form games as the target formalism. The authors state in Section 2:

"Extensive-form, imperfect-information games represent one of the most general classes of games in game theory. They allow us to represent sequential (i.e., temporally-extended) interactions between players and their underlying valuations or preferences that influence the actions they take in a game."

The choice of extensive-form games (rather than normal-form or Bayesian games) is deliberate: extensive-form games naturally model sequential decision-making with information asymmetry—exactly the structure of a multi-turn conversation where each player knows their own private valuation but not their partner's.

The binding maps a dialogue task onto the standard game tuple:

$\langle N, c, A, H, Z, u, \tau, S \rangle$

where each element is instantiated for dialogue as follows.

Players (N): The set $N = \{1, 2, \cdots, n\}$ of players is simply the set of dialogue participants. In all experiments, $n = 2$. There is an additional special player $c$, called "chance" or "nature," which represents stochastic elements in the game—in this framework, chance nodes represent the LLM's generation process itself. When an LLM receives a prompt and generates a response, the specific tokens produced are stochastic (due to sampling), and this stochasticity is modeled as a chance outcome in the game tree.

Histories (H): A history $h \in H$ is a sequence of actions taken from the start of the game, including both player actions (chosen instructions) and chance outcomes (the LLM's generated utterances). Concretely, a history in a fruit trading dialogue might be: [Player 1 chooses "assertive" instruction → chance node produces the specific assertive utterance → Player 2 chooses "submissive" instruction → chance node produces the specific submissive response]. Each history is a node in the game tree, and the set $H$ is finite because the tree depth is bounded.

Player identity function (τ): The function $\tau: H \rightarrow N \cup \{c\}$ indicates whose turn it is at each history. At decision nodes where a player must choose an instruction, $\tau(h)$ returns that player's index. At nodes where the LLM generates text (stochastically), $\tau(h) = c$ returns the chance player. The paper notes that each player has "perfect recall"—they remember all their own past actions and all public observations—which is a standard assumption that makes equilibrium computation tractable.

Infostates (S): An infostate $s \in S$ represents a partition of histories such that the player whose turn it is cannot distinguish between histories within the partition. This is how the framework models private information. The paper states:

"In our games, we implicitly define these partitions by defining what information is public versus private to player $\tau(s)$, and a player's infostate therefore includes observable dialogue history, action history and private information."

Importantly, all sent messages are appended to a public thread, making the dialogue history public knowledge to both players. However, a player's own past actions—specifically, their chosen instructions (e.g., "submissive" tone)—are private: the other player sees the resulting utterance but not the instruction that generated it. A player's private valuation (e.g., how much they value each fruit or each meeting day) is also private. The infostate for a player at decision time therefore includes: the full public dialogue history, the player's own private valuations, and the player's own history of chosen instructions, but not the opponent's private valuations or chosen instructions.

Actions (A): The set $A$ is a finite set of infostate-dependent actions. The paper is explicit:

"In the case of dialogue, an LLM can receive a game context and natural language instructions as input; an instruction in this case constitutes the action (strategy) to be taken and is assumed to come from a finite instruction set."

The finiteness of $A$ is the critical enabler. For the debate domain, $A$ might be {logos, ethos, pathos, any}. For fruit trading, $A$ might be {calm, assertive, submissive, any}. For meeting scheduling, $A$ might be {Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday}. The paper also notes that one could extend the action set beyond instructions to include choices like "which LLM (e.g., out of a set of models of different sizes) to call;" the only requirement is finiteness.

Terminal histories (Z): The set $Z \subseteq H$ includes histories where the dialogue has ended. Two termination conditions are specified. First, a limit on the "allowable tree-depth"—meaning a maximum number of messages per player, configured as a parameter num_max_replies (shown as 1 in the example config in Appendix F, though the paper does not specify the exact value used in experiments). Second, an LLM-based termination prompt that determines whether a conversation has reached a natural endpoint:

"determined by either a limit on the allowable tree-depth (we only allow a finite number of messages per player) or by an LLM tasked with determining whether a conversation has ended e.g., a deal has been struck and no further conversation is required."

The config specification in Appendix F shows llm_termination_prompt as a structured object containing prompts for preprocessing the dialogue history and determining terminality.

Utility function (u): The function $u: Z \rightarrow \Delta_u^n \subseteq \mathbb{R}^n$ with $\Delta_u = [u_{\text{min}}, u_{\text{max}}]$ assigns each player a real-valued payoff at terminal nodes. The paper:

"construct[s] a prompt specific to the domain at hand and let[s] an LLM quantify the reward to each player following previous work (Kwon et al., 2023; Wei et al., 2022)."

The payoff is bounded within $[u_{\text{min}}, u_{\text{max}}]$, with the bounds specified in the game configuration (Appendix F shows these as parameters configurable per game). In the fruit trading domain, payoffs represent the difference between the value of a player's basket after trading and before trading. In meeting scheduling, payoffs represent how much the player values the agreed-upon day. In debate, payoffs are binary (0 or 1) based on whether the LLM reward model judges the argument valid.

Why this binding is neither unique nor the only possible one: The paper acknowledges up front:

"The exact binding we present is not unique. We will explore an approach later, PSRO, that can help to modify and improve an initial selected binding automatically."

This admission is important because it frames the binding as a design choice rather than a necessary mapping, and it positions PSRO as a mechanism for iteratively refining that choice. Different bindings could define actions differently (e.g., at a finer or coarser granularity, or over different aspects of language), could model information asymmetries differently, or could use different reward structures.

The Action Space Design: Instructions as Mediators

The central architectural decision in this framework is what constitutes an action. An action is not a natural language utterance. An action is a short instruction string—a tone label like "assertive," an argument style like "logos," a day of the week like "Wednesday," or a special token "Any" that provides no strategic guidance. This instruction is inserted into the LLM's prompt alongside the game context and dialogue history, and the LLM generates the actual utterance conditioned on that instruction.

This design resolves the fundamental tension between the finite action spaces required by game-theoretic solvers and the infinite space of possible natural language utterances. The solver reasons over choices in the instruction space (which is small, discrete, and enumerable), while the LLM handles the translation from instruction to fluent language.

The content of the action space is domain-specific and defined in the game configuration. Appendix F shows the config for fruit trading:

given_prompt_actions = collections.OrderedDict()
tones = ["calm", "assertive", "submissive", "any"]
given_prompt_actions[header.action_keys[0]] = tones

This specifies a one-dimensional action space over four tones. The meeting scheduling config (not shown but described in the text) has a seven-dimensional action space over days of the week. The debate config has a three-dimensional action space over argument styles {logos, ethos, pathos} plus "Any." The action "Any" serves as the uninformative baseline—when selected, the LLM receives the prompt with the instruction "Any," meaning no specific strategic guidance, and the resulting generation represents the LLM's default behavior.

The "Any" action as baseline: The presence of "Any" in the action set is not just a convenience—it is the mechanism that enables the quantitative comparison between solver-guided and unguided LLM behavior. In the CFR experiments, the baseline DLLM is defined as the LLM whose policy always selects the "Any" branch at every decision node. DCFR−LLM, in contrast, selects branches based on the equilibrium distribution computed by CFR. This means DCFR−LLM can still choose "Any" when the equilibrium strategy dictates it, but can also choose more specific instructions when they are strategically advantageous.

Chance nodes model LLM stochasticity: The paper models LLM generation as a chance node in the game tree. When a player selects an instruction action at a decision node, the next node in the tree is a chance node where the LLM stochastically generates a response conditioned on that instruction, the game context, and the dialogue history. This is visible in Figure 1, where decision nodes (squares) alternate with chance nodes (circles). The number of chance outcomes per decision—i.e., how many different possible responses the LLM might generate—is controlled by the parameter num_llm_seeds (set to 2 in the example config). Each seed value produces a potentially different utterance given the same prompt, and these different utterances lead to different subsequent game states.

This chance-node modeling has an important implication: the game tree is not purely deterministic after a player chooses an action. Two players might both select "assertive," but the specific assertive utterances generated could differ due to sampling, leading to different dialogue trajectories and potentially different payoffs. The solver must account for this stochasticity when computing expected values of actions.

Why not make actions be the utterances themselves? If actions were natural language strings, the action space would be infinite (or impractically large), and equilibrium solvers like CFR—which require explicit enumeration of actions per information state—would be computationally intractable. The instruction-based action space is a deliberate abstraction that trades expressivity for tractability. The cost of this tradeoff is that the solver cannot directly choose what the LLM says, only how it says it (tone, style, proposed day). The semantic content of the response is still determined by the LLM conditioned on that instruction. The paper evaluates the fidelity of this conditioning in Section 4 (Table 2 and Appendix B.1), finding that steering accuracy varies significantly: PaLM2-L follows "logos" instructions 96% of the time in debate, but follows "submissive" instructions only 17% of the time in fruit trading.

Procedural Game Generation

The framework is designed to generate many distinct dialogue game instances procedurally, rather than operating on a single hand-crafted game. This is critical for two reasons: it enables the quantitative evaluation over 200 procedurally generated games (Table 1), and it creates the training data for imitation learning of equilibrium policies.

The procedural generation is driven by a configuration dictionary (shown in Appendix F) that specifies templates and parameters for each domain. The key elements that are procedurally varied include:

Player names: An initial list of example names is provided, and the LLM generates additional names (e.g., num_names = 10 in the fruit trading config). Each new game instance randomly draws player names from this generated list.

Private information: The config specifies examples of private information structures—fruit endowments and valuations, meeting day availability and preferences—and the LLM generates additional instances (e.g., num_private_info = (3, 3) meaning 3 new endowments and 3 new valuations). The paper provides example private information in the config:

examples_private_info["fruit_endowment"] = [scenario_trade_fruit.ENDOWMENT_A,
                                             scenario_trade_fruit.ENDOWMENT_B]
examples_private_info["fruit_valuations"] = [scenario_trade_fruit.VALUATION_A,
                                             scenario_trade_fruit.VALUATION_B]

Initial scenarios: Example scenarios define the opening context for a dialogue—an initial email message, the sender and receiver names, and associated private information. The LLM generates new scenarios from these examples.

The payoff computation is also modular: the config specifies a list of payoff objects, each containing an LLM prompt, min and max values, and preprocessing prompts. These can be combined using a user-defined aggregation function, allowing complex multi-component payoffs (though the paper's experiments use single payoffs per domain).

How the game tree is constructed from the config: Given a concrete game instance (specific players, private information, action set, and termination condition), the system builds an extensive-form game tree. The tree has alternating layers of decision nodes and chance nodes. At a decision node for player $i$, the available actions are drawn from the config's given_prompt_actions. At a chance node following an action selection, the number of branches is num_llm_seeds, each representing a different LLM generation sample. The tree depth is bounded by num_max_replies per player (set to 1 in the example config, though the paper's actual experiments likely used larger values given the multi-turn dialogue examples shown). A terminal history is reached when either the depth limit is hit or the LLM termination prompt determines the conversation has ended.

At each terminal history, the payoff for each player is computed by calling RLLM on the full dialogue transcript. The payoff value is then propagated back up the tree by the solver to assign expected values to earlier states and actions.

CFR: Solving Fixed-Action Dialogue Games

Counterfactual Regret Minimization (CFR) is the primary solver used in the paper's quantitative evaluation. CFR is a well-established algorithm for solving imperfect-information extensive-form games by having each player iteratively minimize their counterfactual regret—a measure of how much they would have gained by choosing a different action at a given information state, weighted by the probability of reaching that state under the opponent's strategy.

The paper uses OpenSpiel's (Lanctot et al., 2019) CFR implementation as the solver backend. The text states: "We use OpenSpiel's CFR solver to obtain the equilibrium strategies" and specifies "we use 10 iterations of the CFR solver to approximate an equilibrium for each game" in the imitation learning pipeline. The number of CFR iterations for the main evaluation (Table 1) is not explicitly stated but is implied to be sufficient for convergence to a coarse-correlated equilibrium, given the theoretical guarantees: "An important result of players using regret-minimizing algorithms is that their time average play converges to a coarse-correlated equilibrium (CCE)."

What CFR computes: For each information state in the game tree—each point where a player must choose an instruction given what they know (public dialogue history + private valuations + own past instructions)—CFR outputs a probability distribution over the available actions. This distribution represents the equilibrium strategy: the mix of instructions that minimizes exploitability against a rational opponent.

CFR Gain metric: To evaluate whether solver guidance helps, the paper defines CFR Gain as follows. Consider a baseline scenario where both players use DLLM (the unguided LLM with "Any" at every decision). Now ask: how much would a single player gain by switching to DCFR−LLM (the LLM that selects instructions according to CFR's equilibrium distribution)? CFR Gain is this expected improvement in payoff:

"CFR gain measures gain from switching to DCFR−LLM from DLLM."

A positive CFR Gain means the equilibrium-guided LLM achieves higher payoff against unguided opponents than an unguided LLM does—it exploits the baseline's strategic naivety. The paper reports CFR Gain of +0.106 in debate, +0.037 in meeting scheduling, and +0.038 in fruit trading (Table 1).

NashConv metric: To evaluate whether the equilibrium strategy is stable (not just good against a weak opponent), the paper defines NashConv. Consider a scenario where both players use DCFR−LLM. Now ask: how much would either player gain by deviating to any other strategy (including reverting to DLLM)? NashConv measures this maximum possible gain from deviation:

"NashConv is a pseudo-distance to the Nash equilibrium measuring how much players gain by deviating from DCFR−LLM."

A low NashConv means the equilibrium strategy is self-enforcing—neither player has a strong incentive to switch. The paper reports NashConv of 0.024 in debate, 0.010 in meeting scheduling, and 0.009 in fruit trading (Table 1). These values are small relative to the CFR Gains, indicating that DCFR−LLM forms an approximate Evolutionarily Stable Strategy (ESS)—it performs well both against itself and against alternatives.

How CFR operates with the LLM-in-the-loop: Each iteration of CFR requires evaluating the payoff of different action choices at different states. In a traditional game, this evaluation is cheap—the payoff function is a precomputed table. In this framework, evaluating a specific action sequence requires actually calling the LLM to generate utterances, then calling RLLM to compute payoffs, making each evaluation expensive. The paper does not detail how this computational cost is managed during CFR execution, but the presence of the imitation learning pipeline (Section 4, Appendix E) suggests that CFR is run offline to generate training data—embedding-equilibrium policy pairs—which can then be used to train a fast neural proxy, amortizing the expensive LLM calls over many training examples and enabling rapid equilibrium lookup at inference time.

Relationship between CFR and the config parameters: The CFR solver operates over the game tree constructed from the config. The config parameter params passed to OpenSpiel includes:

params = {"num_distinct_actions": num_players * num_tones,
          "num_llm_seeds": 2,
          "num_players": num_players,
          "min_utility": min([float(p.min) for p in payoffs]),
          "max_utility": max([float(p.max) for p in payoffs]),
          "num_max_replies": 1}

Here, num_distinct_actions accounts for the combinatorics of the action space if the action is multi-dimensional (e.g., recipient × tone), min_utility and max_utility bound the payoff range (needed by CFR for normalization), and num_max_replies = 1 limits the tree depth to a single response per player—though as noted, the paper's actual experimental settings may differ.

PSRO: Expanding the Action Space with Best Responses

Policy-Space Response-Oracles (PSRO) is a meta-algorithm that extends the framework beyond fixed action sets. While CFR computes an equilibrium over a pre-specified set of instructions, PSRO iteratively grows that set by generating new instructions that serve as approximate best responses to the current equilibrium. The paper describes PSRO in Section 3.3 and provides detailed pseudocode in Appendix D.

The standard PSRO algorithm alternates between two steps:

Policy-Space step: Given a set of candidate policies (where each policy is a mapping from information states to actions over the current instruction set), evaluate all pairs of policies in head-to-head matchups. The outcomes are recorded in a normal-form game payoff matrix. Then compute the equilibrium—specifically, the meta-strategy distribution over the candidate policies—of this matrix game. The meta-strategy tells each player what probability to assign to each of their candidate policies when playing against the opponent's meta-strategy.

Response-Oracle step: For each player, compute an approximate best response to the opponent's current meta-strategy. This means finding a new policy (or a new action to add to the action set) that maximizes expected payoff against the equilibrium mixture of opponent policies. The new policy is then added to the candidate set, and the process repeats.

The paper's innovation in PSRO is in how the best response is approximated. Traditional PSRO would use reinforcement learning or gradient-based optimization to train a new policy. The paper instead tasks the LLM itself with generating novel instruction strings, and then evaluates these strings against the current equilibrium. The simplest version—called "Shotgun Approximate Best Response" in Algorithm 1—works as follows:

1. Given the focal player $i$ and the current joint policy $\pi$, set up the existing action set $C$ with their scores under $\pi$.
2. For $t = 1$ to $k$ (where $k$ is the number of shotgun candidates):
   • Prompt an LLM to generate a new candidate instruction string $c_t$ that is not already in $C$.
   • Evaluate this candidate by simulating play against the opponent's policy $\pi_{-i}$ and computing the resulting score $s_t$.
   • Add $(c_t, s_t)$ to $C$.
3. Return the candidate with the maximum score.

This is essentially random search over the space of instruction strings, guided by the LLM's generative capabilities. The paper proposes several more sophisticated variants in Appendix D:

Algorithm 2 (Approximate Better Response): Rather than generating $k$ candidates and picking the best, this uses rejection sampling—keep generating and evaluating new candidates until one is found that achieves a higher score than the focal player's current score under $\pi$.

Algorithm 3 (Trajectory-Aware Approximate Best Response): Instead of generating candidates blindly, provide the generating LLM with the current action set ranked by their scores (ascending order), and ask it to generate $k$ new candidates informed by this fitness landscape. The LLM thus performs a form of guided exploration, using its language understanding to propose instructions that might outperform the current best in the ranked list.

Algorithm 4 (Categorical Approximate Best Response): Extend the search to new categories of actions, not just new instructions within existing categories. For example, if the current action space is defined only over tones, the LLM might generate an entirely new dimension like "argument structure" or "politeness level." This algorithm computes average scores per category, ranks categories, prompts the LLM to generate new candidate categories, then generates and evaluates specific instructions within those new categories.

Why these approximate best response operators are necessary: Training a policy via reinforcement learning to optimize dialogue behavior—the standard PSRO approach—is computationally prohibitive because it requires many LLM calls per training step, and LLM inference is expensive. The paper's approach substitutes learned optimization with LLM-based generation and evaluation, leveraging the LLM's own knowledge of language to propose strategically promising instructions. This is less principled than gradient-based optimization but far more practical given the computational constraints.

PSRO results and interpretation (Figure 3, Figure 4, Figure 5, Figure 6): The paper runs PSRO starting from an initial action set of {calm, assertive, submissive, any} in the fruit trading domain. At iteration $t=0$, the equilibrium (computed via replicator dynamics, an evolutionary game-theory algorithm with connections to regret minimization) places most probability mass on "submissive." The paper interprets this as:

"potentially a tactic to exploit another LLM by eliciting a sense of responsibility or fairness (Park et al., 2023)."

At $t=1$, the approximate best response generates "angry" as a new instruction. At $t=2$, "relaxed" is generated. At $t=3$, "enthusiastic" is generated. The equilibrium distribution shifts at each iteration as the action space expands, eventually settling on a mixture dominated by "assertive," "angry," and "enthusiastic" under the CCE solution concept.

The paper also compares solution concepts. Figure 4 shows the Nash bargaining solution (a cooperative solution concept from two-player bargaining theory) overlaid on the same PSRO run. Under Nash bargaining, "calm" is the final solution, in contrast to the more aggressive tones favored by the competitive CCE. The authors note:

"Both players may extract higher collective value if they remain 'calm' during negotiations."

This comparison highlights that the choice of solution concept—competitive vs. cooperative, CCE vs. Nash bargaining—matters for the strategic behavior that emerges, and the framework is flexible enough to accommodate different solution concepts appropriate to different domains.

In the meeting scheduling domain (Figure 5, Figure 6), PSRO generates "angry" at $t=1$ and "enthusiastic" at $t=2$. The final CCE equilibrium assigns significant mass to "assertive" and "happy," while the Nash bargaining solution roughly agrees with the CCE at the final iteration.

PSRO's role in the broader framework: PSRO serves two purposes. First, it demonstrates that the framework can extend beyond human-specified action sets to discover novel strategic instructions. Second, and more importantly, it addresses the paper's acknowledgment that "the exact binding we present is not unique." PSRO provides a mechanism for automatically refining the binding—discovering which dimensions of variation in language (tones, styles, argument types) are strategically relevant, even if the initial game designer did not anticipate them.

Imitation Learning of Equilibrium Policies

The CFR solver, while principled, is expensive to run at inference time for each new dialogue instance. To address this, the paper trains a neural network to imitate CFR's equilibrium policies, enabling fast equilibrium lookups that can generalize to new, procedurally generated game instances without re-running CFR.

Training data generation (Section 4 and Appendix E): The process works as follows:

1. Generate 200 procedurally generated game instances (e.g., 200 distinct fruit trading scenarios with different players, endowments, and valuations).
2. For each game instance, run 10 iterations of CFR to approximate an equilibrium. This produces, for each information state in each game, a probability distribution over the available actions—the equilibrium policy at that state.
3. For each information state, encode the observable information (dialogue history, game context, private information) as a vector using a T5 (Raffel et al., 2020) text encoder. The paper states these are "length-768 string embeddings."
4. Pair each T5 embedding with its corresponding equilibrium policy distribution over $|A|$ actions.

The result is an imitation dataset: a collection of (embedding vector, target distribution) pairs.

Model architecture and training (Appendix E): The neural equilibrium solver is a small feed-forward network:

"We used a fully connected neural network with two dense hidden layers of size 256 neurons each and a final dense layer trained against the CFR target probabilities to minimize a cross entropy loss."

The loss function is thus:

$\mathcal{L} = -\sum_{a \in A} \pi_{\text{CFR}}(a|s) \log \hat{\pi}(a|s)$

where $\pi_{\text{CFR}}(a|s)$ is the CFR-computed equilibrium probability of action $a$ at information state $s$, and $\hat{\pi}(a|s)$ is the neural network's predicted probability for that action given the T5 embedding of state $s$.

What it computes: For each information state in the training set, this is the categorical cross-entropy between the teacher distribution (CFR's equilibrium) and the student distribution (neural network's softmax output). Minimizing this loss encourages the network to output probability distributions that match CFR's equilibrium distributions.

Why this form: Cross-entropy is the standard loss for distilling a categorical distribution from a teacher model to a student model. It has the property that the global minimum is achieved when $\hat{\pi} = \pi_{\text{CFR}}$ for all states, meaning the student perfectly recovers the teacher's policy. The alternative of using a regression loss (e.g., MSE) on the raw action values would not properly handle the simplicial nature of the output (probabilities must sum to one) and would lose the relative weighting structure that cross-entropy preserves.

Training hyperparameters (verbatim from Appendix E):

"We trained the policy using $10^4$ steps of Adam (Kingma and Ba, 2014) with a batch size of 128 and learning rate of $10^{-3}$."

At inference time, when a new dialogue reaches a decision point, the dialogue history and game context are encoded as a T5 vector, fed into the trained MLP, which outputs a distribution over instructions. A single instruction is sampled from this distribution and passed to the LLM alongside the dialogue history and context.

Evaluation of imitation (Section 4): The paper evaluates DIM−LLM—the LLM guided by the imitation-learned neural equilibrium solver—against the baseline DLLM in a meta-game setup. The meta-game asks: if two players must each choose whether to delegate their dialogue to DIM−LLM or DLLM, what is the equilibrium of this meta-game? In fruit trading, the equilibrium strategy selects DIM−LLM approximately 76% of the time, meaning the imitation policy is preferred over the unguided baseline. However, in meeting scheduling, this relationship reverses—DIM−LLM is chosen only about 44% of the time.

The paper hypothesizes why:

"We suspect that the T5 embedding of the dialogue history contains relatively little useful information for predicting the best day of the week. In contrast, it is plausible that the T5 embedding contains relatively more information about the tone of the dialogue thus far which should be informative in selecting the subsequent 'tone' to deploy in the fruit negotiation."

This reveals an important boundary condition: the imitation learning pipeline works only when the state representation (the T5 embedding) captures information that is predictive of the optimal action. In meeting scheduling, the optimal day to propose depends primarily on the player's own private valuations and the opponent's revealed preferences, but the T5 embedding of conversational history may not adequately encode the semantic content of day proposals and counter-proposals—it may capture surface-level patterns in the text without deeply modeling the strategic implications of which days have been mentioned.

Reward Model Design and Evaluation

The utility function $u: Z \rightarrow \mathbb{R}^n$ is implemented by an LLM-based reward model—RLLM—that evaluates the full dialogue transcript and computes a numerical payoff for each player. This is described in Section 3.1, evaluated in Section 4, and detailed in Appendix C.

Prompt design: RLLM receives the full dialogue transcript, the private information of both players, and a domain-specific prompt that includes few-shot examples and chain-of-thought instructions. For the fruit trading domain, Appendix C shows an example of the intermediate chain-of-thought:

Alina
fruit_endowment:
apple: 2
banana: 1
blueberry: 1
kiwi: 2
fruit_valuations:
apple: 6
banana: 5
blueberry: 1
kiwi: 1
Elroy
fruit_endowment:
apple: 2
banana: 1
blueberry: 1
kiwi: 2
fruit_valuations:
apple: 6
banana: 5
blueberry: 1
kiwi: 1
Player Alina: Receives 1 banana Gives 2 kiwis
Player Elroy: Receives 2 kiwis Gives 1 banana

From this intermediate parsed information, the LLM then computes the payoff as the difference between the value of received items and the value of given items, using each player's private valuations.

Model size selection: The paper compares PaLM2-S and PaLM2-L for the reward model and finds that the larger model significantly reduces error (Table 8). For the fruit trading domain aggregated over all outcome types, PaLM2-L achieves Norm error of 0.06 and Sgn error of 0.02, compared to PaLM2-S with Norm error of 0.55 and Sgn error of 0.46. The paper states:

"We find that using the larger PaLM2-L model significantly reduces RLLM error relative to the smaller PaLM2-S and to an absolute level that is often less than single digit percentage error in each domain. Therefore, we use PaLM2-L for the remainder of our experiments."

Outcome types for reward evaluation: The paper evaluates reward model accuracy across three dialogue outcome categories, using 1000 randomly generated scenarios for each:

• Valid: Dialogues where both players agree to an outcome that is consistent with their private information (e.g., trading fruit they actually possess, scheduling a day they are actually available).
• Rejected: Dialogues where at least one player explicitly rejects reaching an agreement.
• Incomplete: Dialogues that ended before an agreement was reached (due to tree depth limit).

For each outcome type, the paper constructs two-turn dialogue templates with known ground-truth values, allowing direct comparison between the LLM's computed payoff and the correct payoff.

Error metrics: Two metrics are reported in Table 8:

• Norm: "the average absolute reward error divided by the range of possible rewards." This measures the magnitude of error normalized to the payoff scale, so a Norm error of 0.06 means the LLM's payoff estimate is, on average, off by 6% of the total possible payoff range.
• Sgn: "the frequency that the reward model returned a reward whose sign (+/−) did not match ground truth." This measures whether the LLM at least gets the qualitative direction of utility correct (positive vs. negative), even if the magnitude is wrong.

Qualitative failures (Appendix C.2): The paper provides examples of both successful and failed reward calculations. A notable failure case shows an LLM reward model hallucinating a trade where none occurred—even though the LLM's own intermediate reasoning correctly identified that "No agreement has been reached," it nonetheless fabricated a specific trade (Elroy giving John 6 kiwis for 4 blueberries) and proceeded to calculate values based on this hallucination. It then made a second error by using the wrong player's valuation (John's) when calculating the value to Elroy. This highlights that LLM-based reward models, while convenient and flexible, are not perfectly reliable and can compound multiple reasoning errors.

Summary of Design Choices and Their Justifications

• Extensive-form games as the formalism over normal-form or Bayesian games: captures sequential decision-making with information asymmetry, which matches the turn-by-turn, private-information structure of dialogue.
• Instructions as actions over enumerating utterances directly: achieves the finiteness required by solvers while delegating language generation to the LLM; trades expressivity for tractability at the cost of steering fidelity.
• Chance nodes for LLM stochasticity over deterministic modeling: accounts for the inherent nondeterminism in LLM sampling, which means the same strategic choice can lead to different trajectories; this is more realistic but increases the game tree size by a factor of num_llm_seeds per decision.
• LLM-based reward model over hard-coded reward functions: enables flexible, domain-agnostic payoff specification without requiring explicit programming of utility calculations for each domain; sacrifices perfect accuracy for generality and ease of configuration.
• CFR for fixed action sets over simpler solution concepts like iterated best response: provides convergence guarantees to CCE, ensuring the resulting strategies are approximately unexploitable; the computational cost is amortized through imitation learning.
• PSRO for action space expansion over staying with human-specified actions: enables discovery of novel strategic instructions that human designers might not anticipate; the approximate best response operators (shotgun search, trajectory-aware generation) replace expensive RL-based policy optimization with LLM-guided exploration.
• Imitation learning via T5 embeddings + MLP over running CFR at inference time: reduces inference cost from running many CFR iterations to a single forward pass through a small neural network; the tradeoff is that the neural network may not generalize perfectly to states far from the training distribution.
• PaLM2-L for both dialogue generation and reward modeling over smaller models: the paper's ablation shows the larger model significantly improves reward accuracy (Table 8), justifying the increased computational cost for the payoff evaluation component; the dialogue generation model size is not systematically varied or ablated.
• Procedural game generation over fixed scenario testing: enables statistical evaluation over many instances (200 games per domain) and creates the training data volume needed for imitation learning; the config-based specification makes it easy to define new domains without code changes.

4. Key Insights and Innovations

Innovation 1: Dialogue as a Game — Not a Layered System Where Language Sits on Top of a Formal Engine, but Where Language Itself IS the Game Tree

The dominant prior approach to combining game-theoretic reasoning with natural language—exemplified by Cicero in Diplomacy (FAIR et al., 2022)—treats language as a communication channel layered on top of a separately formalized game. In Diplomacy, the game board, unit positions, and move orders constitute the formal game that solvers reason about; language is used to persuade, coordinate, and deceive around that game state, but the strategic reasoning operates on the structured game engine, not on language directly. This architecture implicitly assumes that the strategic core of the interaction can be formalized in advance and that language serves an auxiliary, non-structural role.

The paper fundamentally inverts this relationship. In its framework, there is no separate game engine. The dialogue itself is the extensive-form game. The players, the actions, the information states, the terminal conditions, and the payoffs are all defined directly in terms of the conversational interaction: players are the dialogue participants, actions are instructions that steer the LLM's next utterance, terminal states are reached when the conversation ends (either by turn limit or by LLM-determined completion), and payoffs are computed by an LLM evaluating the dialogue transcript. The game tree is the space of possible conversation trajectories.

This is not an incremental extension of Diplomacy-style architectures. It is a conceptual reframing that opens the door to applying game-theoretic solvers to any domain where strategic interaction happens through natural language, without requiring a pre-existing formal model of the domain's state space. The meeting scheduling domain, for example, has no equivalent of a Diplomacy board—there is no external game state to track beyond what is said in the conversation. Yet the framework can still construct a game tree over this interaction because the binding in Section 2 defines the game elements entirely in terms of the dialogue structure and the LLM's prompts.

The significance of this reframing extends beyond the paper's immediate experimental results. It means that the entire apparatus of extensive-form game solving—CFR, PSRO, replicator dynamics, and the many algorithms developed for poker and other imperfect-information games—can be brought to bear on conversational AI without requiring NLP researchers to hand-craft domain-specific state representations or game formalizations. The binding is a template: provide a set of instruction strings, a procedure for generating private information, a termination condition, and an LLM-based reward prompt, and the framework produces a game that existing solvers can operate on. The paper demonstrates this template across three qualitatively different domains (scheduling, trading, debate) that share no common structure except that they involve two-party dialogue, suggesting the binding is genuinely domain-agnostic rather than tuned to a single task.

The paper's explicit acknowledgment that "the exact binding we present is not unique" reinforces this as a conceptual contribution rather than a claimed optimal solution. The binding is a starting point—a demonstration that the mapping from dialogue to game tree is possible and productive—not a final answer. PSRO's ability to modify and improve the binding by discovering new action dimensions provides a path for iteratively refining this mapping, making the framework self-improving in a way that static layered architectures are not.

Innovation 2: The Instruction-Layer as a Tractability Interface — A Deliberate, Acknowledged Abstraction That Sacrifices Expressivity for Solver Compatibility, and Then Quantifies the Cost

The field has long recognized the tension between the infinite expressivity of natural language and the finite action spaces required by game-theoretic solvers. Prior work resolved this tension in one of two ways: either severely restrict the language to template-based utterances (making the game tractable but losing linguistic richness) or train end-to-end policies that optimize dialogue behavior but sacrifice equilibrium guarantees (Lewis et al., 2017, with its single best-response step and known convergence pathologies). Both approaches implicitly treat the tension as a hard tradeoff—you can have linguistic richness or strategic guarantees, but not both.

The paper's innovation is not that it resolves this tension—it explicitly does not—but that it makes the tradeoff explicit, quantifiable, and improvable through the instruction-layer abstraction. By defining actions as finite instruction strings (tones, argument styles, days of the week) rather than utterances, the framework achieves solver tractability at a measurable cost: the LLM's imperfect ability to follow instructions. This cost is not assumed away or hand-waved; it is empirically measured across domains and model sizes (Tables 2–5 in Appendix B.1). The results reveal a nuanced picture: PaLM2-L follows "logos" instructions 96% of the time in debate but follows "submissive" instructions only 17% of the time in fruit trading. This means the instruction-layer abstraction works well for some strategic dimensions (argument structure) and poorly for others (emotional tone), depending on the LLM's capabilities and the domain.

What makes this an intellectual contribution rather than merely a practical choice is that it reframes the problem. Instead of asking "how do we make equilibrium solvers work with natural language?"—which previous work attempted through increasingly complex integrations—the paper asks "how much strategic value can we extract from a finite instruction set, given that the LLM will only imperfectly execute those instructions?" The answer, from Table 1, is: enough to achieve positive CFR Gain (+0.106 in debate, +0.038 in fruit trading) and low NashConv (0.009–0.024) across all domains, establishing that even imperfect instruction-following is sufficient for meaningful strategic improvement.

This framing also creates a natural research agenda: improving the instruction-following fidelity of the LLM directly improves the quality of the strategic guidance, and the paper's evaluation methodology provides the metrics to measure this. The instruction-layer is not presented as a final solution but as an interface specification—it defines what the solver needs from the LLM (the ability to condition generation on discrete instructions) and what the LLM provides in return (a stochastic mapping from instruction to utterance). Future work can improve either side of this interface independently: better solvers, better instruction sets, or better instruction-following capabilities.

Innovation 3: PSRO as an Action-Space Discovery Mechanism — Not Just a Solver, but a Tool for Finding New Strategic Dimensions of Language

PSRO has been used extensively in game-theoretic research, including in Diplomacy and other complex games, as a method for iteratively expanding policy spaces. The paper's innovation is in what PSRO searches over. In prior work, PSRO expands the set of policies—parameterized strategies for playing the game, typically represented as neural networks trained via RL. The action space itself is fixed; PSRO finds better ways to use existing actions.

In this paper, PSRO expands not policies but actions themselves—specifically, instruction strings. Starting from a human-specified set of four tones {"calm", "assertive", "submissive", "any"} in fruit trading, PSRO discovers "angry," then "relaxed," then "enthusiastic" as instructions that serve as approximate best responses to the current equilibrium. This is fundamentally different from standard PSRO: the algorithm is not just optimizing within a fixed action space but exploring the open-ended universe of possible strategic instructions, using the LLM's own generative capabilities as the search operator. The Shotgun Approximate Best Response (Algorithm 1) prompts an LLM to generate novel instruction strings and evaluates them by simulating play against the current equilibrium—a procedure that has no analog in standard PSRO, where the action space is fixed by the game definition.

The intellectual contribution here is the insight that an LLM's generative knowledge of language can substitute for the expensive policy optimization typically required in PSRO. Rather than training a neural network via RL to discover that "angry" is a good instruction—which would require thousands or millions of costly dialogue simulations—the framework simply asks an LLM to propose candidates. This is an instance of a broader pattern that the paper enables but does not fully explore: using LLMs as search operators over the space of their own prompts, where the search objective is defined by game-theoretic equilibrium rather than by a human designer's intuition.

The comparison of solution concepts in the PSRO results (CCE vs. Nash bargaining, Figures 3–6) adds a further conceptual dimension. Under the competitive CCE, the equilibrium evolves toward aggressive tones ("assertive," "angry," "enthusiastic"). Under the cooperative Nash bargaining solution, "calm" emerges as the final recommendation. This demonstrates that the framework is not just computing optimal strategies—it is surfacing the normative implications of different solution concepts in natural language. A system designer who wants their negotiation agent to maximize individual payoff might deploy the CCE-guided policy; a designer who wants high collective welfare might choose Nash bargaining. The framework makes this choice explicit and quantifiable rather than burying it in opaque model behavior.

Innovation 4: Strategic Stability as a Measurable Property of Dialogue Agents — NashConv and CFR Gain Provide Game-Theoretic Rigor to "Is This LLM Exploitable?"

Prior work on evaluating LLM dialogue quality has focused primarily on fluency, coherence, task completion, and human preference ratings—metrics that assess the surface quality of individual utterances or the overall success rate of isolated interactions. None of these metrics capture whether an LLM's dialogue strategy is exploitable—whether a strategically sophisticated opponent could systematically extract higher payoff by adapting to the LLM's patterns. An LLM that politely agrees to unfavorable meeting times 90% of the time might score well on human preference ratings while being trivially exploitable by any counterpart that learns to propose disadvantageous days.

The paper introduces NashConv and CFR Gain as evaluation metrics specifically designed to measure strategic properties of dialogue agents. NashConv quantifies how close a strategy is to equilibrium—how much any player could gain by unilaterally deviating. CFR Gain quantifies the benefit of switching from an unguided to a guided strategy against an unguided opponent. Together, these metrics characterize two essential properties: whether a strategy is self-enforcing (low NashConv) and whether it meaningfully improves over naive behavior (positive CFR Gain).

What makes this a conceptual contribution rather than just a measurement detail is that it imports a standard from the game-theoretic evaluation of poker and Diplomacy agents into the evaluation of dialogue systems. In competitive game AI, exploitability metrics are standard—they are the primary way of establishing that an agent's strategy is sound rather than merely heuristic. By demonstrating that these same metrics can be computed for dialogue agents, the paper creates a bridge between two evaluation traditions that have operated in isolation. This has implications beyond the paper's specific results: it suggests that future dialogue systems deployed in strategic settings (negotiation assistants, customer service agents, debating systems) should be evaluated not just on how well they complete tasks but on how robust their strategies are against adversarial adaptation.

The finding that DCFR−LLM constitutes an approximate Evolutionarily Stable Strategy—positive CFR Gain combined with low NashConv—across all three domains (Table 1) provides the first quantitative evidence that equilibrium guidance produces dialogue strategies that are both effective and robust. The paper does not claim that the measured gains are large in absolute terms (CFR Gain of +0.038 in fruit trading is modest), but the consistent positive signal across domains supports the framework's validity more convincingly than a large gain in a single domain would.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper evaluates on 200 procedurally generated games per domain for the main CFR experiments (Table 1). These are not drawn from a pre-existing static benchmark; rather, the game configuration specifies templates for private information (fruit endowments/valuations, meeting day availability/preferences, debate topics/stances), player names, and initial scenarios, and an LLM generates novel instances from these templates. The debate topics are drawn from a hand-authored list of 20+ contentious statements (Appendix A.1), ranging from "Breakfast is the most important meal of the day" to "The government should pay for post secondary education." For the reward model evaluation (Section 4, Appendix C), 1000 randomly generated scenarios per outcome type (Valid, Rejected, Incomplete) are used, each producing 2000 reward calculation samples (two players per dialogue). For the imitation learning evaluation (Section 4), the same 200 procedurally generated games are split into training and held-out sets, though the exact split ratio is not specified.

• Base model(s). All experiments use autoregressive models from the PaLM 2 family (Anil et al., 2023). The paper uses two sizes: PaLM2-S and PaLM2-L. The dialogue LLM (DLLM) and the reward model (RLLM) can be different instantiations. For the main experiments, PaLM2-L is selected as the reward model because the ablation in Table 8 shows it significantly reduces reward error compared to PaLM2-S (Norm error of 0.06 vs. 0.55 in fruit trading aggregated across outcome types). The dialogue LLM is PaLM2-S for the steering accuracy evaluation (Table 2, showing 75% aggregate accuracy with definitions) and PaLM2-L for the fruit trading and meeting scheduling evaluations (Tables 4–5). The paper argues these models are "representative of the capabilities of many contemporary LLMs" though this claim is not empirically validated against other model families. The models are used at inference time only—they are frozen, with no fine-tuning for any of the dialogue tasks. All strategic adaptation occurs through the solver's choice of instruction at each turn, conditioning the frozen LLM's generation.

• Metrics. The paper employs multiple evaluation metrics addressing different aspects of the system. NashConv measures the pseudo-distance to Nash equilibrium: in a scenario where both players use DCFR−LLM (the CFR-guided LLM), how much would either player gain by unilaterally deviating to any other strategy, including reverting to the unguided DLLM? Lower values indicate a more stable, self-enforcing equilibrium. It is computed by evaluating the maximum possible gain from deviation across all available strategies. CFR Gain measures the benefit of adopting the equilibrium strategy: in a scenario where both players start as baseline DLLM, how much would a single player gain by switching to DCFR−LLM? A positive value means the guided LLM exploits the unguided baseline's strategic naivety. It is computed as the difference in expected payoff between DCFR−LLM and DLLM against a DLLM opponent. For the imitation learning evaluation, the paper constructs a meta-game where two players must elect to delegate their dialogue to either DIM−LLM (the imitation-policy-guided LLM) or DLLM (the baseline). The equilibrium strategy of this meta-game—specifically, the probability of selecting DIM−LLM—serves as a preference metric. A value above 50% indicates the imitation policy is preferred. Steering accuracy (Appendix B.1, Tables 2–5) measures the fidelity of instruction-following: given an LLM-generated message conditioned on an instruction to take action a, a held-out classifier model P determines whether a is the most likely action in the action set conditioned on the message, i.e., a ∈ arg max_z∈A P(z | message). This measures whether the LLM's generation actually reflects the strategic instruction it was given. Reward model error (Table 8) uses two metrics. Norm reports the average absolute reward error divided by the range of possible rewards, normalized so that 1.0 means the error spans the full payoff range. Sgn reports the frequency with which the reward model's predicted payoff sign (+/−) differs from ground truth, measuring qualitative correctness independent of magnitude. The paper does not report standard task-completion metrics (e.g., deal rate in trading, agreement rate in scheduling, persuasion success in debate) as primary endpoints, focusing instead on the game-theoretic properties of the strategies.

• Baselines. The primary baseline is DLLM — the frozen PaLM-2 LLM prompted with the special token "Any" as its action at every decision node. "Any" provides no specific strategic guidance; the LLM generates utterances using its default behavior given only the game context and dialogue history. This baseline represents the unguided LLM's natural strategic capability (or lack thereof). In the CFR evaluation, DLLM is the strategy that always selects the "Any" branch at every decision node in the game tree. There is no comparison to non-game-theoretic prompting strategies (e.g., chain-of-thought reasoning, role-playing prompts, explicit instruction to "negotiate aggressively"). The paper also does not compare against end-to-end trained negotiation agents (Lewis et al., 2017) as a baseline, though it discusses that work's limitations in Section 5. The comparison is thus strictly between solver-guided and unguided versions of the same frozen LLM, isolating the effect of the equilibrium strategy overlay.

• Generation budget / compute accounting. The paper does not report generation budgets in the conventional sense of "number of LLM calls" or "total tokens generated." The cost of dialogue is implicitly measured by parameters in the game configuration: num_llm_seeds (set to 2 in the example config) determines how many distinct LLM generations are sampled at each chance node, and num_max_replies (set to 1 in the example config, though likely larger in actual experiments given the multi-turn examples shown) limits the tree depth. The game tree size—and thus the total number of LLM calls required to solve it—is a function of these parameters, the size of the action space, and the depth limit. The paper acknowledges that "transitions in the dialogue game we pose are extremely expensive" because "LLM inference is expensive, both computationally and financially and given the large game trees that need to be created, these calculations can very quickly become an intensive resource-consuming operation" (Limitations section), but provides no quantitative accounting of computation cost for any experiment.

• Cross-validation / statistical protocol. The CFR evaluation (Table 1) reports NashConv and CFR Gain averaged over 200 procedurally generated games per domain. The paper does not report confidence intervals, standard deviations, or statistical significance tests for these metrics. For the imitation learning evaluation, the equilibrium policy is trained on CFR outputs from the 200 generated games and evaluated on held-out games, but the train/test split ratio is not specified. The reward model evaluation uses 1000 randomly generated scenarios per outcome type, with each scenario producing 2000 samples (two players), for a total of 6000 scenarios and 12,000 reward calculation samples across the three outcome types. No cross-validation is reported for the PSRO experiments—they appear to be single runs demonstrating the algorithm's progression (Figures 3–6). The paper's "NashConv" and "CFR Gain" are presumably computed via simulation against the specified opponent policies, but the number of simulation rollouts used to estimate expected payoffs is not disclosed.

Main Quantitative Results

Instruction-Following Fidelity (Steering Accuracy)

The paper first establishes whether the LLM can reliably execute the instructions that the solver selects, since the entire framework depends on the LLM's ability to condition its generation on instruction strings. The results reveal substantial heterogeneity across domains, action types, and model sizes.

In the debate domain with PaLM2-S (Table 2), aggregate steering accuracy is 59% without definitions provided in the classifier prompt and 75% with definitions. However, per-action accuracy varies dramatically: "logos" achieves 88% (without definitions) and 98% (with definitions), while "ethos" achieves only 32% and 56%, and "pathos" achieves 47% and 61%. This means constructing a logically-structured argument from an instruction is far more reliable than modulating emotional appeal—the LLM frequently generates pathos-style content when instructed to use ethos, and vice versa, but rarely confuses logos with the other styles.

With PaLM2-L in debate (Table 3), aggregate accuracy drops to 47% (with definitions), and the per-action pattern shifts: "logos" remains strong at 96%, but "ethos" falls to 37% and "pathos" to 32%. The larger model is actually worse at following stylistic instructions in debate, which the paper flags as surprising but does not explain mechanistically. One hypothesis (not stated in the paper) is that larger models have stronger default priors about debate style that dominate the instruction signal—they may default to logos-style argumentation regardless of instruction because that's what their training data predominantly associates with persuasive writing.

In the fruit trading domain with PaLM2-L (Table 4), aggregate accuracy is 43% with definitions. "Calm" achieves 69%, "assertive" achieves 58%, and "submissive" achieves only 17%. The near-total failure of the "submissive" instruction is a critical finding: when the solver recommends a submissive tone as the equilibrium strategy (which it does at PSRO iteration t=0 in Figure 3), the LLM fails to execute it in 83% of cases. This means the equilibrium strategy computed by the solver over the instruction space may not translate to the actual distribution of tones in generated utterances—the effective strategy is not the solver's equilibrium but the solver's equilibrium "pushed forward" through a noisy, asymmetric instruction-following channel.

In the meeting scheduling domain with PaLM2-L (Table 5), aggregate accuracy is 26%, with per-day accuracy ranging from 53% for Monday to 12% for Saturday and 16% for Sunday. The paper's interpretation is that "this likely reflects the fact that the standard work week lies between Monday and Friday and scheduling meetings on weekends is probably a rare occurrence in its training data." This finding has direct implications for the CFR equilibrium: if the solver recommends Saturday or Sunday, the LLM will rarely actually propose those days, making those branches of the game tree effectively inaccessible regardless of their strategic value.

The aggregate steering accuracy across all three domains and action types, weighted by the action-space sizes, is substantially below 100%. The paper treats this as a limitation of the base LLM rather than the framework: "Should this inferior LLM be replaced with one that can always follow the given instruction, we would see LLM generations that are optimal in the strategy taken for that game context." This is the central assumption on which the framework's practical value rests.

Reward Model Accuracy

The reliability of the LLM-based reward model is evaluated using constructed two-turn dialogues with known ground-truth payoffs, across three outcome types and two model sizes (Table 8).

For PaLM2-S, reward errors are substantial:

• Fruit trading (aggregated over all outcome types): Norm error = 0.55, Sgn error = 0.46. This means the reward model's magnitude estimate is off by 55% of the payoff range on average, and it gets the sign wrong (positive vs. negative utility) 46% of the time—essentially coin-flip-level performance for qualitative correctness.
• Meeting scheduling: Norm error = 0.29, Sgn error = 0.42, with particularly high errors for Rejected outcomes (Norm = 0.41, Sgn = 0.63) and Incomplete outcomes (Norm = 0.37, Sgn = 0.57).

For PaLM2-L, errors drop substantially:

• Fruit trading (aggregated): Norm error = 0.06, Sgn error = 0.02. The sign error of 2% means the reward model almost always correctly identifies whether an outcome is beneficial or detrimental, and the magnitude error of 6% of the payoff range is low enough for meaningful equilibrium computation.
• Meeting scheduling (aggregated): Norm error = 0.07, Sgn error = 0.12, with the largest remaining errors on Incomplete outcomes (Norm = 0.09, Sgn = 0.18).

The breakdown by outcome type reveals that PaLM2-L's errors are concentrated in specific categories: in meeting scheduling, Valid outcomes have Norm = 0.11 and Sgn = 0.16 (worse than Rejected outcomes at Norm = 0.02, Sgn = 0.02). The paper does not discuss why Valid outcomes are harder to evaluate correctly than Rejected outcomes, but it may be because Valid outcomes require the reward model to extract and parse specific agreement details from the dialogue, while Rejected outcomes are a uniform zero-payoff case that is easier to recognize.

Based on this ablation, the paper selects PaLM2-L as the reward model for all subsequent experiments, justifying the choice by the "single digit percentage error" achieved in each domain.

CFR-Guided Dialogue: NashConv and CFR Gain

The central quantitative result of the paper is Table 1, which reports NashConv and CFR Gain for DCFR−LLM compared to the DLLM baseline across all three domains, averaged over 200 procedurally generated games.

NashConv values:

• Debate: 0.024
• Meeting Scheduling: 0.010
• Trading Fruit: 0.009

These values are small and positive. A NashConv of 0.009 in fruit trading means that if both players are using the CFR-guided strategy, the maximum expected gain from either player unilaterally switching to any alternative strategy (including the unguided baseline) is 0.009 payoff units. Since the payoff bounds are specified per domain (though the exact ranges are not reported in the main text—the config in Appendix F shows min_utility and max_utility as configurable per game), the interpretation depends on the payoff scale. If the payoff range is, say, [-10, 10] in fruit trading, a NashConv of 0.009 is very small relative to the range. The paper does not normalize NashConv by the payoff range, making cross-domain comparison of absolute values somewhat ambiguous, though the consistent pattern of low values across domains is the key finding.

CFR Gain values:

• Debate: +0.106
• Meeting Scheduling: +0.037
• Trading Fruit: +0.038

These are all positive, meaning DCFR−LLM achieves higher expected payoff against the unguided DLLM baseline than DLLM achieves against itself. Debate shows the largest gain, which the paper does not explicitly explain but which is consistent with the steering accuracy results: in debate, the "logos" instruction achieves 96–98% fidelity, meaning the solver's strategic recommendations are actually executed most of the time, translating equilibrium computation into realized strategic advantage. In fruit trading, where "submissive" achieves only 17% fidelity, the CFR Gain is smaller (+0.038), consistent with the interpretation that instruction-following noise attenuates the strategic benefit.

The paper's interpretation of these combined results is that DCFR−LLM approximately satisfies the conditions for an Evolutionarily Stable Strategy (ESS) : it performs well against itself (low NashConv, meaning no incentive to deviate when both players use it) and it outperforms alternatives when they are present (positive CFR Gain). The paper states: "According to the criteria, DCFR−LLM is an approximate ESS."

Several limitations of this analysis are not discussed in the paper:

• The CFR Gain is measured only against the DLLM baseline (always "Any"), not against other possible alternative strategies (e.g., always "assertive," always "submissive," random action selection, or a strategy that conditions on dialogue history without solver guidance). A positive CFR Gain against the weakest possible baseline is a relatively low bar.
• The NashConv is measured with respect to a limited strategy set (presumably the strategies available in the game tree, though the exact deviation set is not specified). If the deviation set is only DLLM and DCFR−LLM, NashConv of 0.009 means neither player wants to switch to the unguided baseline—but this doesn't establish that the strategy is an equilibrium in the full strategy space.
• The results are reported as averages over 200 games but without any measure of variance. A NashConv of 0.009 could mask substantial variation across game instances—some configurations might be near-perfect equilibrium while others might be far from it.

CFR Gain vs. NashConv Tradeoff

The relationship between CFR Gain and NashConv reveals a pattern across the three domains. In debate, CFR Gain is highest (+0.106) but NashConv is also highest (0.024). In fruit trading, CFR Gain is lower (+0.038) but NashConv is lowest (0.009). This suggests a possible tradeoff: strategies that exploit the baseline most aggressively may also be more exploitable themselves (less stable as equilibria). The paper does not explicitly analyze this tradeoff, but the data in Table 1 is consistent with it. Debate, with its high-fidelity instruction following for "logos," allows the solver to aggressively optimize toward arguments that exploit the unguided LLM's weaknesses, but this optimization may create a strategy that is itself exploitable by a different counter-strategy (though the NashConv remains small enough that the paper treats it as approximately stable).

Impact of Instruction-Following Fidelity on Game-Theoretic Metrics

Combining the steering accuracy results (Tables 2–5) with the game-theoretic metrics (Table 1) suggests a causal relationship: domains where instruction-following is most reliable show the largest strategic gains. In debate, "logos" instructions are followed 96–98% of the time, and CFR Gain is +0.106. In fruit trading, aggregate instruction-following is 43%, and CFR Gain is +0.038. In meeting scheduling, aggregate instruction-following is 26% (with weekend days near 12–16%), and CFR Gain is +0.037. The paper does not compute this correlation explicitly or perform a controlled experiment varying instruction-following fidelity independently, but the pattern is consistent with the hypothesis that the effectiveness of solver guidance is bounded by the LLM's ability to execute the recommended instructions.

This relationship also implies that the measured CFR Gains are underestimates of what the framework could achieve with better instruction-following LLMs. If an LLM with 100% instruction-following fidelity were substituted, the equilibrium strategy computed by CFR over the instruction space would be exactly the strategy realized in the generated utterances, and the strategic benefit would be larger. The paper makes this argument explicitly in the Limitations, but does not quantify the potential gain.

PSRO: Action Space Expansion Results

The PSRO experiments (Figures 3–6, Appendix D) demonstrate that the framework can iteratively expand the action space beyond human specification. The results are presented as descriptive trajectories rather than aggregated quantitative metrics.

In the fruit trading domain (Figure 3, main text), starting from the initial action set {"calm", "assertive", "submissive", "any"}, the CCE equilibrium at iteration t=0 assigns highest probability to "submissive." The paper interprets this as "potentially a tactic to exploit another LLM by eliciting a sense of responsibility or fairness." This interpretation, while plausible, is speculative—the paper does not provide a mechanistic analysis of why the equilibrium solver computes this distribution, only the post-hoc interpretation.

At t=1, the approximate best response generates "angry" as a new instruction. At t=2, "relaxed" is generated. At t=3, "enthusiastic" is generated. The equilibrium distribution shifts at each iteration as the action set expands. By t=3, the CCE equilibrium (shown in white in Figure 3) assigns significant probability to "assertive," "angry," and "enthusiastic," with reduced mass on "submissive" and "calm." The paper characterizes this informally but does not provide quantitative measures of how much the expected payoff improves at each PSRO iteration.

Comparison of solution concepts in PSRO (Figures 4 and 6): The paper compares the CCE equilibrium with the Nash bargaining (NB) solution at each PSRO iteration. In the fruit trading domain (Figure 4), the CCE at t=3 is a mixture of "assertive," "angry," and "enthusiastic," while the NB solution is "calm." The paper states: "Both players may extract higher collective value if they remain 'calm' during negotiations." This comparison demonstrates that different solution concepts produce meaningfully different strategic recommendations—a competitive solver pushes toward aggressive negotiation, while a cooperative solver pushes toward calm, measured exchange. The paper does not, however, empirically demonstrate that the NB solution actually produces higher collective welfare in simulated play—it states this as a conjecture ("may extract higher collective value") without supporting data.

In the meeting scheduling domain (Figure 5), PSRO generates "angry" at t=1 and "enthusiastic" at t=2. The final CCE equilibrium assigns significant mass to "assertive" and "happy" (note: "happy" is presumably generated during the PSRO process, though it is not explicitly listed in the generated actions in the text—Figure 5 shows it in the equilibrium distribution). The NB solution (Figure 6) roughly agrees with the CCE at t=3, in contrast to the fruit trading domain where they diverged.

A significant quantitative omission: the paper does not report how much the expected payoff improves as the action space expands in PSRO. Without this, it is unclear whether the newly discovered instructions ("angry," "relaxed," "enthusiastic") actually provide strategic benefit beyond the initial human-specified set. The equilibrium distributions are reported, but the absolute payoff values under these equilibria are not compared to the initial t=0 equilibrium.

Imitation Learning: Generalization to New Game Instances

The imitation learning experiment evaluates whether the equilibrium policy computed by CFR can be distilled into a neural network that generalizes to novel procedurally generated game instances. The evaluation is conducted as a meta-game between DIM−LLM and DLLM.

In the fruit trading domain, the approximate equilibrium of the meta-game selects DIM−LLM approximately 76% of the time. This means that if two players must each choose which LLM to delegate their negotiation to, the equilibrium strategy is to prefer the imitation-policy-guided LLM over the unguided baseline in 76% of cases. The paper interprets this as evidence that the imitation-learned policy generalizes effectively to new fruit trading scenarios.

In the meeting scheduling domain, the meta-game equilibrium selects DIM−LLM only approximately 44% of the time—meaning the unguided baseline is actually preferred in 56% of cases. This is a negative result: the imitation policy is worse than doing nothing (using "Any" as the instruction). The paper's hypothesis for this reversal is:

"We suspect that the T5 embedding of the dialogue history contains relatively little useful information for predicting the best day of the week. In contrast, it is plausible that the T5 embedding contains relatively more information about the tone of the dialogue thus far which should be informative in selecting the subsequent 'tone' to deploy in the fruit negotiation."

This explanation points to a fundamental limitation of the imitation learning approach: it can only be as good as the state representation (the T5 embedding) allows. If the T5 embedding of the dialogue history does not adequately encode the strategic information that the CFR policy conditions on (e.g., which days have been proposed, which have been rejected, what the implied availability constraints are), then the neural network cannot recover the equilibrium policy regardless of how well it is trained. The meeting scheduling domain's failure case reveals that the T5 embedding, designed as a general-purpose text encoder, may not be suited for tasks requiring precise extraction of semantic content about dates, proposals, and availability from dialogue.

The paper does not report the absolute payoff values in the fruit trading meta-game—only the equilibrium selection probabilities. A 76% preference for DIM−LLM does not directly translate to a statement about how much better it is in payoff terms. A strategy that is only marginally better than the alternative could still be selected 76% of the time in equilibrium if the payoff differences, while small, are consistent.

Ablation Studies and Robustness Checks

Reward model scale: PaLM2-L vs. PaLM2-S (Table 8). Increasing model size from S to L reduces aggregate reward error dramatically in fruit trading (Norm: 0.55 → 0.06, Sgn: 0.46 → 0.02) and substantially in meeting scheduling (Norm: 0.29 → 0.07, Sgn: 0.42 → 0.12). This is the only model-scale ablation in the paper, and it justifies the choice of PaLM2-L for the reward model component. The paper does not ablate the dialogue LLM scale—the experiments use PaLM2-S for some steering evaluations and PaLM2-L for others, but there is no side-by-side comparison of dialogue quality or strategic performance as a function of DLLM model size.

Steering accuracy with vs. without action definitions in prompt (Table 2, debate domain). Providing definitions of argument styles (logos, ethos, pathos) in the classifier prompt increases aggregate steering accuracy from 59% to 75% for PaLM2-S. The improvement is concentrated in "ethos" (32% → 56%) and "pathos" (47% → 61%), while "logos" improves from already-high 88% to 98%. This shows that the LLM's ability to categorize its own generated text according to the instruction it received depends on having clear definitions of the action categories—without them, the classifier confuses ethos and pathos with each other and with the random baseline. The paper does not ablate whether providing definitions in the dialogue generation prompt (rather than just the classifier prompt) would improve instruction-following fidelity.

Outcome type analysis for reward model error (Table 8, breakdown by Valid/Rejected/Incomplete). The error patterns differ by outcome type and domain. In meeting scheduling with PaLM2-L, Valid outcomes have higher Norm error (0.11) than Rejected outcomes (0.02), which is counterintuitive—one might expect valid agreements to be more complex to evaluate. In fruit trading with PaLM2-S, Rejected outcomes have the highest Sgn error (0.65), meaning the smaller model frequently misidentifies the sign of payoff for rejected trades. The paper does not analyze these per-outcome patterns beyond reporting the numbers.

Solution concept comparison (CCE vs. Nash bargaining): Figures 4 and 6 overlay the Nash bargaining solution on the PSRO equilibrium trajectories. In fruit trading at t=3, the solutions diverge (CCE: mixture of aggressive tones, NB: "calm"). In meeting scheduling at t=3, they roughly agree. This is presented as a qualitative comparison rather than an ablation in the strict sense—no quantitative metric is reported for how different solution concepts affect realized payoffs or strategic properties.

Imitation learning across domains: The comparison of DIM−LLM meta-game performance between fruit trading (~76% preference) and meeting scheduling (~44% preference) serves as an implicit ablation of the state representation. The same T5 embedding + MLP architecture succeeds in one domain and fails in another, with the paper attributing the difference to the informational content of the embedding. This suggests that the choice of state encoder is a critical design decision that the paper does not systematically ablate—no alternative encoders (e.g., domain-specific embeddings, instruction-tuned encoders, larger T5 variants) are tested.

Missing ablations: The paper does not ablate several factors that would strengthen confidence in the results. There is no comparison of CFR iteration count to solution quality—the 10-iteration setting for imitation learning data generation is stated without justification or sensitivity analysis. There is no ablation of the number of procedurally generated games (200) to determine whether results are sensitive to this. There is no comparison of different chance-node branching factors (num_llm_seeds)—the config shows 2, but larger values would produce bushier trees with more accurate modeling of LLM stochasticity at higher computational cost. There is no ablation of tree depth (num_max_replies)—longer dialogues might allow more strategic adaptation but cost exponentially more to solve. There is no comparison of the LLM-based termination condition against the simple depth limit to determine whether early termination affects strategic behavior.

Critical Assessment

Does the framework actually produce more strategic dialogue, or does it produce dialogue that is less exploitable against a weak baseline?

The central claim of the paper is that equilibrium solver guidance leads to "more rational and strategic natural language generations" that achieve "higher payoff in the game." The primary evidence is Table 1, showing positive CFR Gain across all three domains. However, CFR Gain is measured specifically as the gain of DCFR−LLM over DLLM when playing against DLLM. The baseline is an LLM that always uses the uninformative "Any" action—it has no strategic adaptation whatsoever. This is a weak baseline. A positive CFR Gain against this baseline demonstrates that the equilibrium-guided LLM can exploit a strategically naive opponent, but it does not demonstrate that the guided LLM plays a genuinely strategic game—only that it plays a less naive one.

What would strengthen the claim: a comparison of DCFR−LLM against a set of heuristic strategies that a practical adversary might employ (e.g., always selecting the same tone, randomly switching tones, conditioning tone on the opponent's previous tone). If DCFR−LLM achieves positive gain against a diverse set of alternative strategies, not just the "Any" baseline, the claim of genuine strategic improvement would be stronger. The NashConv metric partially addresses this by measuring exploitability against any deviation, but the deviation set is limited by the game definition, and NashConv alone does not tell us whether the strategy performs well against specific plausible counter-strategies.

Does the evaluation capture what matters about strategic dialogue, or does it capture properties of the instruction-following interface?

The paper's metrics (NashConv, CFR Gain) measure properties of the strategy over instructions, not properties of the actual utterances generated. The instruction-following evaluation (Tables 2–5) reveals that the mapping from instruction to utterance is noisy and asymmetric—"submissive" instructions succeed only 17% of the time in fruit trading. This means the effective strategy realized in the dialogue is not the strategy the solver computed. The solver might place 40% probability on "submissive" at some information state, but the actual distribution of tones in the generated utterances will be very different because "submissive" instructions rarely produce submissive utterances, while "calm" instructions are more reliable.

None of the game-theoretic metrics account for this noise. CFR Gain and NashConv are computed over the instruction space, assuming that selecting an instruction is equivalent to producing an utterance with that characteristic. The paper acknowledges this limitation—"Should this inferior LLM be replaced with one that can always follow the given instruction, we would see LLM generations that are optimal"—but does not adjust its quantitative claims accordingly. A more accurate statement would be: "The equilibrium strategy over instructions achieves positive CFR Gain under the assumption of perfect instruction-following, and the realized gain in actual dialogue is bounded above by this value and below by a value that depends on the instruction-following fidelity."

Does the PSRO experiment demonstrate genuine strategic discovery, or is it a curated demonstration?

The PSRO results (Figures 3–6) show the algorithm expanding the action set from {"calm", "assertive", "submissive", "any"} to include "angry," "relaxed," and "enthusiastic." This is presented as evidence that PSRO "could be helpful in extending beyond, e.g., a negotiation expert's initial list of suggested tactics to discover new strategic instructions." However, several aspects of this evaluation are weak:

• The paper does not report whether the expanded action set actually improves expected payoff compared to the initial set. Without this, it's unclear whether "angry," "relaxed," and "enthusiastic" are strategically useful discoveries or merely alternative instructions that the equilibrium re-weights without improving outcomes.
• The Shotgun Approximate Best Response (Algorithm 1) generates k random candidates and picks the best. With k undisclosed but presumably small (given computational constraints), this is a very weak optimization procedure. The fact that three emotionally-valenced words are discovered in three iterations may simply reflect that emotional tones are the most salient dimension of variation in the LLM's generative space, not that these are strategically optimal additions.
• Only a single PSRO run per domain is reported. Without multiple runs or statistical characterization, it is impossible to assess whether the progression from "submissive" → "angry" → "relaxed" → "enthusiastic" is a reliable pattern or an artifact of the specific random seeds and LLM samples in that run.

The imitation learning result in meeting scheduling is an important negative finding, but its implications are underanalyzed.

The failure of DIM−LLM in meeting scheduling (~44% preference, meaning worse than the unguided baseline) is the paper's clearest negative result, and it deserves more attention than it receives. The paper attributes it to the T5 embedding containing "relatively little useful information for predicting the best day of the week." This explanation, while plausible, raises questions that the paper does not address:

• If the T5 embedding doesn't capture the strategic information needed for meeting scheduling, does CFR's policy actually use such information? If CFR's policy is largely independent of the dialogue history (e.g., always proposing the day with highest private valuation regardless of what the opponent said), then the failure of imitation learning may reflect that the equilibrium policy is simple enough that the baseline DLLM already approximates it reasonably well. Alternatively, if CFR's policy does condition on the dialogue history in complex ways that T5 cannot capture, this suggests the embedding choice is a serious bottleneck for the entire approach.
• The paper does not attempt alternative state representations for meeting scheduling (e.g., explicitly parsed features indicating which days have been proposed, accepted, or rejected). This would be a natural ablation to determine whether the failure is due to the T5 encoder specifically or due to a more fundamental difficulty in predicting equilibrium actions in this domain.
• The meta-game preference of 44% for DIM−LLM is reported without any measure of statistical significance. With 200 games, the difference between 44% and 50% (no preference) may not be statistically distinguishable from chance. If the true preference is approximately 50%, the result would indicate that imitation learning provides no benefit but also no harm—a different interpretation than "the imitation policy is worse."

The reward model evaluation uses synthetic two-turn dialogues, not organic multi-turn dialogues.

The reward model accuracy results (Table 8) are based on constructed two-turn templates where the outcome (Valid, Rejected, Incomplete) is controlled by design. While this enables clean ground-truth comparison, it does not reflect the complexity of organic multi-turn dialogues where agreement details may be implicit, partially stated, revised mid-conversation, or ambiguous. The qualitative failure case in Appendix C.2—where the reward model hallucinates a trade from an ongoing negotiation where no agreement has been reached—is drawn from an organic dialogue and illustrates failure modes that the synthetic evaluation cannot capture. The true error rate of RLLM on organic multi-turn dialogues may be higher than the single-digit percentages reported in Table 8.

There is no end-to-end evaluation of the full system on a downstream task.

The paper evaluates components separately: instruction-following accuracy, reward model accuracy, NashConv/CFR Gain for the solver-guided LLM, and imitation learning preference. It does not report a single end-to-end metric that captures whether the complete system—game configuration → procedural generation → equilibrium solving → instruction selection → LLM generation → reward evaluation—produces dialogues that are measurably better than those from a simpler baseline. For example, in the fruit trading domain, one could measure: what fraction of dialogues result in a trade agreement? What is the average realized payoff for each player? How does this compare to the unguided baseline? These outcome-level metrics are absent. The paper's focus on game-theoretic properties (exploitability, equilibrium stability) is consistent with its framing, but the practical relevance of the framework depends on whether these properties translate to improved outcomes in concrete dialogue scenarios, and this translation is not demonstrated.

Missing baseline: a simple prompting strategy.

The paper compares DCFR−LLM against DLLM with "Any" as the action. It does not compare against an LLM prompted with a fixed strategy—for example, always using "assertive" tone, or always proposing the day with highest private valuation. Such fixed strategies are simpler than equilibrium computation but might capture a large fraction of the strategic benefit. If "always assertive" achieves CFR Gain of +0.035 in fruit trading compared to +0.038 for the full CFR policy, the marginal benefit of equilibrium computation would be very small. Without this comparison, the paper cannot distinguish between "equilibrium computation provides strategic benefit" and "any moderately sensible strategy provides strategic benefit over random behavior."

Statistical rigor is insufficient for the claims made.

The paper reports means over 200 procedurally generated games for NashConv and CFR Gain (Table 1) but provides no measures of variance, confidence intervals, or statistical tests. For the PSRO experiments, single runs are shown. For the imitation learning meta-game, a single equilibrium selection probability is reported without uncertainty quantification. The steering accuracy evaluation reports percentages with varying sample sizes (80 to 18,165 across tables) but no confidence intervals. The reward model evaluation uses 1000 scenarios per outcome type but reports only point estimates. This lack of statistical characterization makes it difficult to assess whether the reported differences are reliable or could be explained by sampling variation—particularly for the smaller effects (NashConv of 0.009, CFR Gain of 0.037) where the effect sizes are small relative to the plausible variance in LLM-generated dialogues.

The computational cost of the framework is acknowledged but not quantified.

The paper states in the Limitations that "transitions in the dialogue game we pose are extremely expensive" and that "these calculations can very quickly become an intensive resource-consuming operation," but provides no concrete numbers. How many LLM calls does it take to run 10 iterations of CFR on a fruit trading game with action space size 4 and tree depth 2? How many to run one PSRO iteration with k shotgun candidates? Without this information, it is impossible to assess the practical deployability of the approach or to compare its cost-effectiveness against alternatives like simply using a larger LLM with better prompting. This omission is particularly important because the paper's contribution is an inference-time steering mechanism—it adds computation at deployment time, and the cost-benefit ratio of this added computation is never quantified.

In summary, the experiments provide consistent evidence that equilibrium solver guidance produces strategies that are less exploitable (positive CFR Gain, low NashConv) across three dialogue domains, and that the magnitude of this benefit is modest but non-zero. However, the evaluation falls short of demonstrating that this translates to meaningful improvements in the quality of generated dialogues or the outcomes of strategic interactions. The paper establishes that the framework is technically functional—the binding works, solvers compute equilibria, and the resulting strategies differ from unguided behavior in a direction consistent with strategic improvement—but does not establish that the framework is practically valuable at its current level of component reliability (particularly instruction-following fidelity and reward model accuracy) or computational cost.

6. Limitations and Trade-offs

The Instruction-Following Bottleneck: Strategic Recommendations Are Not Executed Reliably

The assumption or constraint. The entire framework depends on the LLM's ability to condition its generated utterances on instruction strings chosen by the equilibrium solver. The paper treats the mapping from instruction to utterance as reliable enough that the solver's equilibrium over instructions translates to strategic behavior in dialogue. However, the paper's own evaluation reveals that this mapping is highly unreliable for many instruction types. In the fruit trading domain with PaLM2-L, "submissive" instructions produce recognizable submissive utterances only 17% of the time (Appendix B.1, Table 4). In meeting scheduling with PaLM2-L, aggregate instruction-following accuracy is 26%, with weekend days falling to 12–16% (Table 5). Only "logos" in debate achieves high fidelity (96% with PaLM2-L, Table 3).

The paper acknowledges this limitation explicitly: "Note that this is a fault of the base LLM's instruction following ability, rather than the pipeline or equilibrium solver inefficacy. Should this inferior LLM be replaced with one that can always follow the given instruction, we would see LLM generations that are optimal in the strategy taken for that game context" (Appendix B.2).

The consequence. The equilibrium strategy computed by CFR or PSRO is a distribution over intended instructions, but the effective strategy realized in generated dialogue is the equilibrium distribution convolved with a noisy, asymmetric instruction-following channel. When the solver places high probability on an instruction the LLM cannot execute (e.g., "submissive" in fruit trading), the actual strategic behavior diverges from the equilibrium. The measured CFR Gain values (Table 1) therefore represent a lower bound on potential benefit and an upper bound on realized benefit—the potential benefit of the equilibrium strategy under perfect execution, but not the realized benefit under the actual noisy channel. The paper never accounts for this gap quantitatively. In the worst case, the solver could systematically recommend instructions the LLM cannot follow, producing behavior that is no better—or potentially worse—than the unguided baseline, while incurring the computational cost of equilibrium solving.

What evidence exists in the paper. Tables 2–5 (Appendix B.1) provide per-action steering accuracy across all three domains and two model sizes. The variation is substantial: from 17% (submissive, PaLM2-L) to 98% (logos, PaLM2-S). The paper does not re-compute NashConv or CFR Gain after adjusting for instruction-following probabilities, meaning all game-theoretic metrics assume perfect execution. The qualitative examples in Tables 6 and 7 (Appendix B.2) show both successful and failed instruction following, confirming that the problem is not merely a measurement artifact but visible in actual generated text.

Mitigation status. The paper treats this as a limitation of the specific LLM used rather than the framework, suggesting that "if replaced with a stronger LLM that has better language reasoning capabilities, [this] can be resolved." No attempt is made to filter or re-weight instructions based on measured fidelity, to train the LLM for better instruction-following on these specific action sets, or to incorporate the instruction-following error rates into the equilibrium computation itself (which would produce a more conservative strategy that avoids unreliable instructions). This is purely deferred to future model improvements.

Computational Cost of Game Tree Construction and Solving Is Unaccounted For and Potentially Prohibitive

The assumption or constraint. The framework requires constructing an extensive-form game tree and running an equilibrium solver (CFR or PSRO) over it. Each node in this tree represents an LLM call for dialogue generation (at chance nodes) or reward evaluation (at terminal nodes). The paper acknowledges this in the Limitations section: "Transitions in the dialogue game we pose are extremely expensive. This is because LLM inference is expensive, both computationally and financially and given the large game trees that need to be created, these calculations can very quickly become an intensive resource-consuming operation."

The cost scales combinatorially: with |A| actions per decision node, num_llm_seeds chance outcomes per action, and depth D per player, the full game tree has approximately (|A| × num_llm_seeds)^D terminal nodes, each requiring an LLM reward evaluation. The example config in Appendix F uses num_llm_seeds = 2 and num_max_replies = 1, producing a tree with only |A| × 2 terminal nodes—a small tree. But for multi-turn dialogues (the examples in Table 6 show up to two turns, and richer strategic interactions would require more), the tree grows exponentially. CFR then requires iterating over this tree multiple times (10 iterations are used for imitation learning data generation; the main experiments do not disclose the iteration count), with each iteration requiring computing counterfactual values at each information state.

The consequence. The paper presents no cost-benefit analysis. The headline result—that DCFR−LLM achieves low NashConv and positive CFR Gain—is reported without quantifying how many LLM calls were required to compute the equilibrium strategy for each game instance. A practitioner considering deployment cannot answer: for a given dialogue domain, is the strategic improvement worth the computational overhead of building and solving the game tree? The situation is worse for PSRO, where each iteration adds new actions and requires re-evaluating the expanded game. The imitation learning pipeline (Section 4, Appendix E) partially addresses this by amortizing CFR's cost over many game instances through offline training of a neural equilibrium solver, but: (a) this requires running CFR on 200 procedurally generated games to create the training set—a large upfront cost; (b) the imitation policy fails to generalize in meeting scheduling (~44% preference, Section 4), meaning the amortization does not always succeed; and (c) the neural solver only replaces the equilibrium lookup, not the game tree construction, which still requires defining the action space, private information structures, termination conditions, and reward prompts per domain.

What evidence exists in the paper. The paper provides no quantitative measurement of computational cost. The number of LLM calls per CFR run, per PSRO iteration, per reward model evaluation, or per dialogue turn is never stated. The config parameters (num_llm_seeds = 2, num_max_replies = 1) in Appendix F suggest small trees were used, but these parameters are from an example config and may not reflect the experiments that produced Table 1. The paper reports results averaged over 200 procedurally generated games, each requiring its own tree construction and solving, but the total compute budget is undisclosed.

Mitigation status. The paper proposes two mitigations but implements neither at scale. First, it suggests that "we will need to push scalable game-theoretic algorithms, e.g., (Burch et al., 2012), to new heights" (Limitations), referencing the need for more efficient solvers. Second, the imitation learning pipeline is presented as a way to avoid per-instance CFR computation, but this only shifts the cost to an offline training phase and fails to generalize in one of two tested domains. No latency or throughput measurements are reported. For a framework that is purely an inference-time intervention (the LLM is frozen, not fine-tuned), the inference-time overhead is the central practical concern, and its absence from the evaluation is a significant gap.

The Binding from Dialogue to Game Theory Requires Domain-Specific Engineering and Expert Judgment

The assumption or constraint. The framework maps dialogue onto an extensive-form game tuple ⟨N, c, A, H, Z, u, τ, S⟩, but instantiating each element requires domain-specific design decisions that are not automated. The action set A must be chosen by a human designer—tones for fruit trading, argument styles for debate, days of the week for meeting scheduling. The granularity of this action set determines the expressivity of the strategic space: too coarse (e.g., only two tones) and the solver has little leverage; too fine (e.g., 50 nuanced emotional descriptors) and the game tree explodes combinatorially. The private information structure (what players know but don't reveal) must be specified. The reward prompt must be constructed with few-shot examples and chain-of-thought reasoning steps (Appendix C). The termination condition must be defined, either as a turn limit or an LLM-based detector of conversational completion. The paper acknowledges this explicitly in Section 2: "The exact binding we present is not unique."

The consequence. The framework is not a turnkey system for making any dialogue agent strategic. Each new domain requires a game designer to make explicit choices about what constitutes a strategic action, what private information exists, and how payoffs should be computed. These choices embed assumptions about the domain that affect the equilibrium: selecting tones as the action space for trading assumes that emotional modulation is the primary strategic lever, while selecting days as the action space for scheduling assumes that proposing specific days is the strategic lever. But what if the optimal strategy in trading involves not just tone but also the sequencing of offers, the timing of concessions, or the disclosure of partial information? The framework can accommodate these only if the designer anticipates them and encodes them in the action set or the private information structure.

This also means that comparing results across domains is difficult because the action spaces represent qualitatively different strategic dimensions. The CFR Gain of +0.106 in debate and +0.038 in fruit trading (Table 1) are not directly comparable—debate's action space (argument styles) may simply afford more strategic leverage than fruit trading's action space (tones), independent of the solver's effectiveness.

Furthermore, the choice of solution concept matters. The PSRO experiments compare CCE and Nash bargaining solutions (Figures 4, 6) and find they diverge in fruit trading (CCE: aggressive tones, Nash bargaining: "calm"). A practitioner must decide which solution concept is appropriate for their domain, and the wrong choice could produce strategies that are overly aggressive (CCE in a cooperative setting) or overly accommodating (Nash bargaining against an adversarial opponent). The paper provides no guidance on how to make this choice.

What evidence exists in the paper. The three domains demonstrate different action space designs, reward model prompts, and private information structures—each hand-crafted by the authors. Appendix F shows the fruit trading config specification, revealing the number of explicit design choices required (action keys, payoff objects, private info generators, termination prompts, scenario templates). The PSRO results (Figure 3) show that starting from a human-chosen action set, the algorithm discovers additional actions ("angry," "relaxed," "enthusiastic"), but these discoveries are still constrained by the initial choice of the tone dimension—PSRO discovers new tones, not new strategic dimensions (e.g., it does not discover that "offer an uneven split" or "threaten to walk away" might be useful actions). The comparison of solution concepts (Figures 4, 6) shows that the equilibrium strategy depends on which concept is selected, but no evaluation of which concept produces better real-world outcomes is attempted.

Mitigation status. The paper positions PSRO as a partial mitigation: "We will explore an approach later, PSRO, that can help to modify and improve an initial selected binding automatically" (Section 2). PSRO can expand the action set within the chosen action category, but it cannot redefine the category itself. Algorithm 4 (Categorical Approximate Best Response) proposes generating entirely new action categories, but this is described in Appendix D as a conceptual extension and is not demonstrated in experiments. The paper does not provide a methodology for validating whether a given binding is "good enough" or for comparing alternative bindings for the same domain. The domain-specific engineering burden remains unaddressed for practitioners who want to apply the framework to new dialogue tasks.

The Evaluation's Baseline Is Weak, and Absolute Performance Improvements Are Not Measured Against Plausible Alternatives

The assumption or constraint. The quantitative evaluation (Table 1) compares DCFR−LLM against DLLM, where DLLM is defined as the LLM that always selects the "Any" branch at every decision node—essentially the frozen PaLM-2 model with no strategic guidance whatsoever. "Any" provides no instruction beyond the game context and dialogue history. This baseline represents the LLM's default behavior in the complete absence of strategic prompting.

The consequence. A positive CFR Gain against this baseline demonstrates that the equilibrium-guided strategy outperforms random or default behavior, but not that it outperforms any plausible heuristic a practitioner might use. Consider alternatives: an LLM prompted with "be assertive" at every turn (always choose the "assertive" instruction), an LLM prompted with chain-of-thought reasoning about what proposal would maximize its payoff given the dialogue history, or an LLM that simply proposes its highest-valued option on every turn. Any of these could achieve positive CFR Gain against the "Any" baseline, potentially matching or exceeding the gain from CFR equilibrium computation at a fraction of the computational cost. The paper provides no comparison against such fixed-strategy baselines, making it impossible to determine whether the equilibrium solver provides value beyond simple rule-based prompting.

The NashConv metric partially addresses this by measuring exploitability against any deviation (including fixed strategies), but NashConv only quantifies stability—how much a player could gain by deviating from DCFR−LLM—not whether DCFR−LLM outperforms those alternative strategies in direct competition. A strategy can have low NashConv (meaning no single deviation is very profitable) while still being worse in expected payoff than a simple heuristic when both face each other or when both face a common opponent. The meta-game evaluation for imitation learning (Section 4) comes closer by comparing DIM−LLM against DLLM in a head-to-head meta-game, but still only against the "Any" baseline, not against alternative guided strategies.

What evidence exists in the paper. Table 1 reports CFR Gain of +0.106 (debate), +0.037 (meeting scheduling), and +0.038 (fruit trading). These are the only comparisons of DCFR−LLM against any alternative strategy. The paper does not report the performance of DCFR−LLM against, for example, an "always assertive" strategy or an "always logos" strategy. The NashConv values (0.009–0.024) suggest that deviations from DCFR−LLM are not highly profitable, but this does not tell us whether a simple fixed strategy would achieve similar NashConv while being computationally free.

Mitigation status. Not addressed. The paper does not discuss this as a limitation, does not propose fixed-strategy baselines as alternatives, and does not suggest that future work should compare against heuristic prompting strategies. The evaluation framework is entirely structured around the comparison between solver-guided and unguided versions of the same LLM, leaving the practical value proposition of the solver overhead unestablished.

No Evidence That the Framework Improves Actual Dialogue Outcomes, as Opposed to Game-Theoretic Metrics Defined Over the Instruction Space

The assumption or constraint. The paper's evaluation metrics—NashConv, CFR Gain, meta-game equilibrium selection probabilities—measure properties of strategies over the instruction space, not properties of the utterances generated or the outcomes achieved in dialogue. NashConv measures how much a player could gain by deviating from DCFR−LLM to another instruction-choosing strategy. CFR Gain measures how much better DCFR−LLM's instruction distribution performs against DLLM's instruction distribution. The imitation learning meta-game measures whether DIM−LLM's instruction policy is preferred over DLLM's. All of these treat the instruction as the action and the solver-computed payoff as the outcome.

The consequence. None of these metrics answer the question a practitioner would most naturally ask: does the solver-guided LLM achieve better negotiation outcomes—more successful deals, higher realized payoffs, more favorable meeting times, more persuasive arguments—than the unguided LLM? The gap between instruction-space metrics and outcome-space reality is created by two factors identified in the paper itself: imperfect instruction-following (the LLM may not execute the recommended instruction) and imperfect reward modeling (the solver uses RLLM's payoff estimates during equilibrium computation, but these estimates have non-zero error as documented in Table 8). The solver optimizes for high RLLM-computed payoff under the assumption of perfect instruction execution, but the actual outcome is produced by imperfect execution evaluated by an imperfect reward model. The extent to which optimization in this loop translates to improved real outcomes is never measured.

The debate domain is particularly telling: CFR Gain of +0.106 is the highest across all domains, but the debate payoff is binary (0 or 1, as described in Section 3.2), meaning the gain represents an increase in the probability of receiving a payoff of 1 rather than 0. If the baseline DLLM wins 50% of debates and DCFR−LLM wins 60.6%, that's a +0.106 gain. But whether this advantage comes from genuinely more persuasive arguments or from exploiting quirks of the RLLM's evaluation criteria is unknown, because the paper provides no external validation of debate quality (e.g., human evaluation of argument persuasiveness).

What evidence exists in the paper. No downstream outcome metrics are reported anywhere in the paper. We do not know: the fraction of fruit trading dialogues that end in agreement, the average realized payoff per player in any domain, the fraction of meeting scheduling dialogues that find a mutually agreeable time, or the win rate in debate as judged by any criterion other than RLLM. The reward model evaluation (Table 8) shows that RLLM has non-zero error, but this error is measured on synthetic two-turn templates (Appendix C), not on organic multi-turn dialogues where the errors may be larger (as the qualitative failure case in Appendix C.2 demonstrates—the reward model hallucinates a trade where none occurred). The instruction-following evaluation (Tables 2–5) shows that the mapping from instruction to utterance is noisy, but this noise is never propagated through to outcome-level metrics.

Mitigation status. Not addressed. The paper's framing emphasizes strategic properties (exploitability, equilibrium stability) over task-completion metrics, which is a defensible research choice—the contribution is about introducing game-theoretic rigor to dialogue, not about maximizing task performance. However, the paper also makes claims about "higher payoff in the game" (Abstract) and "more strategic responses" (Section 1), which imply outcome-level improvement that is never directly demonstrated. The Limitation that comes closest is the paper's acknowledgment that "it is up for debate whether [LLMs'] fidelity is sufficient to draw conclusions on real world interactions" and that "using these models as black-box operators or autoraters for evaluation is often not 100% reliable." But this is framed as a limitation of LLMs in general, not a specific evaluation gap in the paper's own experimental design.

The Framework Offers No Path Forward for Scenarios Where the LLM Has Zero Capability

The assumption or constraint. The framework assumes the LLM can generate utterances that execute the solver's recommended instructions well enough to influence the dialogue outcome. But as the meeting scheduling instruction-following results show (Table 5), there are actions the LLM essentially cannot execute: proposing meetings on Saturday (12% accuracy) or Sunday (16% accuracy). The paper's explanation is that "scheduling meetings on weekends is probably a rare occurrence in its training data" (Appendix B.1), which is plausible but also reveals the deeper issue: the framework provides no mechanism for expanding the LLM's fundamental capabilities. If an action is outside the LLM's behavioral repertoire, the solver can recommend it endlessly without effect.

The consequence. The framework can only redistribute probability mass among actions the LLM can already execute with reasonable fidelity. It cannot teach the LLM to execute new kinds of actions. This is a sharp boundary on the framework's applicability: it works in domains where the strategic lever (tone, argument style, proposed day for weekdays) is within the LLM's existing instruction-following capabilities, and fails where the lever requires behavior the LLM has not learned. This is analogous to the finding from the test-time compute scaling paper that "test-time compute can amplify existing capability but cannot create it"—here, equilibrium solving can redistribute strategic emphasis among existing capabilities but cannot create new ones.

The PSRO algorithm partially addresses this by discovering new instruction strings, but it discovers them from the LLM's generative space—the LLM must be able to produce a plausible candidate instruction. If the concept of "propose Saturday" is so poorly represented in the LLM's training that it cannot follow that instruction, the LLM may also be unable to generate "propose Saturday" as a candidate best response in PSRO, or may generate it but then fail to execute it when selected. PSRO expands the nominal action space but not the effective action space, which is bounded by instruction-following fidelity.

What evidence exists in the paper. The meeting scheduling instruction-following results (Table 5) provide direct evidence: weekend days are proposed only 12–16% of the time when instructed, compared to 53% for Monday. The paper does not test whether these low-fidelity instructions could still be strategically useful (e.g., if proposing Sunday 16% of the time when the solver recommends it is better than never proposing Sunday), nor does it measure whether the equilibrium solver learns to avoid recommending low-fidelity instructions (CFR should, in principle, assign low probability to actions that produce poor outcomes, but only if those poor outcomes are reflected in the payoff evaluations fed back to the solver). The imitation learning failure in meeting scheduling (~44% preference for DIM−LLM, Section 4) may also partly reflect that the equilibrium policy recommends actions the LLM cannot execute, making the imitation policy's recommendations no more effective than the baseline's default behavior.

Mitigation status. The paper does not explicitly address this limitation. The suggestion to use "a stronger LLM that has better language reasoning capabilities" (Appendix B.2) would mitigate it by expanding the set of executable instructions, but this is a model capability improvement outside the framework's scope. The framework itself provides no mechanism for identifying which instructions are executable before including them in the action set, nor for adapting the equilibrium computation to account for differential instruction-following fidelity. A natural mitigation—measuring per-action fidelity during a calibration phase and incorporating those probabilities into the game tree as chance node transition probabilities—is not explored.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a conceptual interface layer that translates between the open-ended space of natural language dialogue and the finite, structured action spaces required by game-theoretic equilibrium solvers. The magnitude of this contribution is not a paradigm shift—it does not disprove prevailing assumptions or introduce a new class of algorithms—but rather a methodological reframing that makes an entire body of existing game-theoretic machinery newly applicable to conversational AI without requiring NLP researchers to hand-craft domain-specific state representations. The key reframing is the recognition that actions in dialogue games can be instructions that steer an LLM rather than the utterances themselves, decoupling strategic decision-making from language generation in a way that preserves solver tractability while retaining linguistic expressivity.

The landscape change operates on two levels. At the practical level, the paper provides a template—the binding from dialogue to the extensive-form game tuple in Section 2—that any researcher can instantiate for a new dialogue domain by specifying an instruction set, a private information structure, a reward prompt, and a termination condition. Appendix F demonstrates this templating concretely with the fruit trading config, and the fact that three qualitatively different domains (scheduling, trading, debate) are implemented within this template suggests the binding is genuinely domain-agnostic. This lowers the barrier to entry for applying game-theoretic reasoning to dialogue tasks: you no longer need expertise in both equilibrium computation and dialogue state tracking to build a strategically-aware conversational agent. You need only to define the game elements in the framework's terms, and existing solvers (CFR, PSRO) handle the rest.

At the conceptual level, the paper resolves a latent tension between two bodies of conflicting evidence. Prior work had demonstrated both that game-theoretic solvers can achieve superhuman performance in complex multi-agent settings (Diplomacy, StarCraft) and that LLMs often fail to exhibit coherent reasoning strategies in interactive settings (Jiang et al., 2020; Turpin et al., 2024). The implicit question was whether the gap existed because game-theoretic methods are fundamentally incompatible with natural language, or because no one had built the right bridge. The paper provides evidence for the latter: by building that bridge—the instruction-layer binding—and showing that LLMs can follow solver recommendations well enough to produce measurable strategic improvement (positive CFR Gain across all three domains, Table 1), it demonstrates that the incompatibility is not fundamental but architectural. The field does not need to choose between linguistic richness and strategic guarantees; it needs to design the interface between them more carefully.

The paper also shifts the conversation around what it means to evaluate dialogue agents. Prior work in conversational AI has focused predominantly on surface metrics—fluency, coherence, task completion rate, human preference scores. None of these metrics captures whether an agent's conversational strategy is exploitable. By importing NashConv and CFR Gain from the game-theoretic evaluation of poker and Diplomacy agents, the paper introduces strategic stability as a measurable property of dialogue systems. This is not merely a new metric; it is a new evaluation criterion that could become standard for dialogue agents deployed in consequential multi-agent settings. An agent that completes 95% of scheduling tasks successfully but is systematically exploitable by any adversary that learns to propose disadvantageous times would score well on conventional metrics but poorly on NashConv. The paper provides the first evidence that these metrics can be meaningfully computed for natural language interactions, establishing that strategic evaluation of dialogue agents is feasible.

The research directions that become more attractive after this work include: (1) improving the interface layer itself—finding instruction sets that are both strategically expressive and reliably executable by current LLMs; (2) developing verifiers or reward models that are accurate enough for equilibrium computation to produce strategies that transfer to real outcomes, not just to the reward model's internal evaluations; (3) exploring the space of possible bindings—the paper explicitly acknowledges its binding is "not unique," and alternative bindings (finer action granularity, different information structures, different solution concepts) may produce different strategic behaviors; (4) scaling the game-theoretic algorithms to handle the large trees that realistic multi-turn dialogue requires. The directions that become less attractive include: (1) attempting to make LLMs strategically sophisticated through prompting alone, without any external strategic reasoning module—the paper's positive CFR Gain demonstrates that equilibrium computation provides benefit beyond what the LLM's default behavior achieves; (2) treating language as merely a communication layer on top of a pre-formalized game engine—the paper shows that dialogue itself can be the game, making separately engineered game states unnecessary for domains where the interaction is purely conversational.

However, the paper also surfaces a sobering boundary that constrains the optimism: the framework's effectiveness is fundamentally bounded by instruction-following fidelity. The finding that PaLM2-L follows "submissive" instructions only 17% of the time in fruit trading (Table 4) means that the equilibrium strategy computed by the solver may diverge substantially from the strategy actually realized in dialogue. The solver may correctly identify that a submissive tone is optimal at some game state, but if the LLM cannot execute that instruction, the strategic benefit evaporates. This is not a flaw of the framework—it cleanly separates the strategic reasoning from the execution, and the execution failure is the LLM's—but it means that the framework's practical value is directly tied to the instruction-following capabilities of the underlying model. As LLMs improve in their ability to condition generation on fine-grained instructions, the framework's potential grows. But until then, the effective action space is not the nominal action set but the subset of instructions the LLM can reliably execute, and the paper provides no mechanism for identifying this subset or adapting the equilibrium computation to account for it.

Follow-Up Research This Work Enables

Calibrating the instruction-following channel and incorporating it into equilibrium computation. The paper's steering accuracy results (Tables 2–5) reveal that the mapping from intended instruction to realized utterance is a noisy, asymmetric channel—some instructions are executed with high fidelity (logos: 96%), others with near-zero fidelity (submissive: 17%). Yet the CFR equilibrium computation in Table 1 treats this channel as transparent, assuming that selecting an instruction is equivalent to producing an utterance with that characteristic. A natural extension is to measure the full instruction-following confusion matrix for a given LLM on a given action set—for each intended instruction $a_i$, empirically estimate the probability $P(\text{produced } a_j \mid \text{intended } a_i)$ using the held-out classifier P described in Appendix B.1—and then incorporate these probabilities into the game tree as the transition distribution at chance nodes. This would transform the solver's optimization problem from "choose the instruction that maximizes expected payoff assuming perfect execution" to "choose the instruction that maximizes expected payoff under the actual noisy channel." The hypothesis is that the equilibrium strategy would shift probability mass away from low-fidelity instructions (like "submissive") and toward high-fidelity alternatives (like "calm" at 69%), producing a strategy that is less optimal in the idealized instruction space but more effective in realized dialogue. A strong experiment would compare the realized dialogue payoffs of CFR policies computed with and without the calibrated channel model, using an LLM-as-judge to evaluate actual utterance quality independently of the intended instruction, to determine whether channel-aware equilibrium computation closes the gap between nominal and realized strategic benefit.

Testing the framework with systematically varied instruction granularity to map the expressivity-tractability-fidelity tradeoff. The paper instantiates three domains with three different action space designs—argument styles for debate, tones for fruit trading, days of the week for scheduling—but provides no systematic analysis of how action space granularity affects strategic outcomes. An experiment would take a single domain (e.g., fruit trading) and vary the action set along a spectrum: a coarse set (e.g., {"cooperative", "competitive"} with 2 actions), the paper's set ({"calm", "assertive", "submissive", "any"} with 4 actions), a finer-grained set (adding "enthusiastic", "angry", "relaxed", "firm", "apologetic" to make 9 actions), and a very fine-grained set (20+ emotional descriptors). For each granularity level, the experiment would measure three quantities: (1) CFR Gain over the "Any" baseline (strategic benefit), (2) aggregate instruction-following accuracy (execution fidelity), and (3) the computational cost of solving the game tree (which grows with $|A|$). The prediction—motivated by the paper's finding that PSM2-L achieves only 43% aggregate accuracy on a 4-action set (Table 4)—is that instruction-following fidelity degrades as the action space becomes more finely differentiated, creating a tradeoff: more actions allow finer strategic control but each action is executed less reliably. This experiment would identify the Pareto-optimal granularity for a given LLM on a given domain, providing practitioners with guidance on how to design the binding.

Running PSRO for enough iterations (and with enough candidates per iteration) to determine whether action-space expansion asymptotes or continues to yield strategic improvement. The paper's PSRO experiments (Figures 3–6) demonstrate 3 iterations of action-space expansion starting from a 4-action set, but stop without characterizing the trajectory. Does each added instruction improve expected payoff, or does the benefit plateau after the first few discoveries? With PSRO's Shotgun Approximate Best Response (Algorithm 1) generating $k$ candidates per iteration (presumably small), the algorithm may be finding locally optimal instructions that are easy to generate rather than globally strategic ones. An extended experiment would run PSRO for 20–50 iterations on a single domain (fruit trading is the natural choice given the existing results), using a larger $k$ (e.g., 100 candidates per iteration) and reporting the expected payoff of the equilibrium at each iteration. The key question is whether PSRO discovers instructions that are qualitatively different from the initial human-specified set—not just more tone words, but instructions that open new strategic dimensions (e.g., "propose an asymmetric trade," "reveal partial information about your valuation," "threaten to walk away"). If the discovered instructions remain within the initial semantic category (emotional tone) even after many iterations, this would suggest that PSRO is limited by the LLM's generative biases—it can only propose instructions that are near neighbors in the LLM's representation space, not genuinely novel strategic concepts. If, however, the LLM eventually proposes cross-category instructions, this would validate PSRO as a genuine strategic discovery mechanism. The comparison of CCE vs. Nash bargaining solutions at each iteration (already shown in Figures 4 and 6) should be extended to include realized payoffs under both solution concepts, testing the paper's conjecture that Nash bargaining "may extract higher collective value."

Replicating the framework with LLMs that have been fine-tuned for instruction-following on the specific action sets, to isolate whether the framework's ceiling is the concept or the model. The paper's instruction-following results are measured on frozen PaLM-2 models that have received no task-specific training. This conflates two sources of limitation: the fundamental difficulty of mapping discrete instructions to nuanced linguistic behavior, and the specific PaLM-2 models' (lack of) training on these particular instruction types. A strong follow-up would take a base LLM, fine-tune it on a dataset of (instruction, game_context, dialogue_history, target_utterance) examples for a single domain (e.g., fruit trading), where the target utterances are generated by a larger model or written by humans to exemplify each tone, and then re-run the full CFR and PSRO pipelines with the fine-tuned dialogue LLM. If the fine-tuned model achieves, say, 85%+ instruction-following accuracy (vs. PaLM2-L's 43% aggregate in fruit trading), the hypothesis is that CFR Gain would increase substantially—the equilibrium strategy would actually be executed, translating strategic optimization into realized payoff improvement. If CFR Gain remains modest even with near-perfect instruction-following, this would suggest a deeper limitation: that the action space itself (tone modulation) provides limited strategic leverage in negotiation, and that the binding needs to incorporate richer action dimensions (offer structure, information revelation) to achieve substantial strategic benefit. This experiment would disambiguate whether the framework's modest gains are due to execution failure or expressive limitation.

Evaluating the framework against human negotiators to determine whether equilibrium-guided LLM strategies transfer beyond LLM-vs-LLM interactions. All experiments in the paper involve LLM-LLM dialogue—the solver computes equilibria over strategies played against another LLM, and the payoffs are evaluated by yet another LLM. This creates a closed LLM ecosystem that may not reflect strategic interactions with humans. Humans have different behavioral patterns, different susceptibilities to emotional tone, and different reward structures than LLM-based reward models. An experiment would deploy DCFR−LLM and DLLM (baseline) against human participants recruited through a crowdsourcing platform, in a controlled negotiation task (fruit trading is the most natural domain, given its structured payoffs). Human participants would negotiate with one of the two agents (randomly assigned, between-subjects), and the primary metric would be the human's realized payoff vs. the agent's realized payoff, compared to the game-theoretic predictions. A secondary metric would be human perception of the agent's negotiation style (rated on scales of assertiveness, fairness, cooperativeness). The key question is whether the equilibrium-guided agent actually extracts higher payoff from human counterparts, or whether the strategies that work against LLMs (which the paper shows are biased toward certain tones and argument styles) fail to generalize to human interaction. A negative result—DCFR−LLM achieves no advantage over DLLM against humans—would not invalidate the framework but would sharply circumscribe its applicability to LLM-LLM settings (e.g., multi-agent simulations, synthetic data generation), whereas a positive result would demonstrate real-world strategic value.

Stress-testing the framework on a domain where the strategic lever is not in the surface form of language but in the information structure—e.g., a negotiation where the optimal strategy involves when and how much private information to reveal. All three domains in the paper use action spaces defined over how something is said (tone, argument style, which day to propose) rather than what information is conveyed. But a central topic in game-theoretic negotiation is strategic information revelation—when to disclose your private valuation, when to bluff, when to make commitments. The current binding, with its instruction-based action space, can in principle accommodate this: the action set could include instructions like "reveal your true valuation," "bluff about your valuation," "remain silent about your valuation," or "ask about their valuation." An experiment would design such an action set for a negotiation domain (fruit trading naturally supports this, since players have private valuations), run CFR to compute the equilibrium policy over information-revelation strategies, and measure whether the resulting dialogue exhibits sophisticated information management (e.g., revealing information when it induces a favorable counter-offer, concealing information when it would weaken bargaining position). This would test whether the framework can capture strategic depth beyond stylistic modulation, and whether current LLMs can execute information-revelation instructions reliably (which is likely harder than executing tone instructions, since it requires the LLM to reason about what information to reveal or conceal in context).

Practical Applications and Downstream Use Cases

Multi-agent simulation for social science and policy modeling. The paper's framework enables constructing populations of LLM-based agents that interact strategically according to equilibrium policies, and the procedural game generation capability (Section 3.2, Appendix F) means these simulations can be instantiated over thousands of varied scenarios. A social scientist studying the emergence of cooperation norms in bargaining could configure the fruit trading domain with different solution concepts—CCE for competitive populations, Nash bargaining for cooperative ones—and observe whether the equilibrium strategies discovered by PSRO (Figure 3) lead to different population-level outcomes (average welfare, inequality of payoffs, rate of agreement). The paper's finding that solution concept choice matters (CCE pushes toward aggressive tones, Nash bargaining toward "calm" in Figure 4) provides preliminary evidence that the framework can surface how different normative assumptions percolate into behavioral patterns. The key advantage over traditional agent-based models is that the agents produce natural language, not abstract action tokens, making the simulation outputs interpretable as transcripts that could be shown to human subjects or analyzed with NLP tools. The limitation—and the reason this is not yet production-ready—is the reward model's non-zero error rate (6% Norm error for PaLM2-L in fruit trading, Table 8), which means the strategic behavior is optimized for an imperfect proxy of true payoffs.

Generating strategic training data for negotiation and debate coaches. The framework can produce large volumes of varied, strategically-sophisticated dialogue by procedurally generating game instances and having the equilibrium-guided LLMs play them out. For a company building a negotiation training simulator, the config-based approach (Appendix F) means they can specify new negotiation scenarios (different goods, different valuation structures, different constraints) by modifying configuration parameters without rewriting prompts or retraining models. The PSRO mechanism (Figure 3) further means the system can discover strategic variations (different tones, different argumentative approaches) that the training designers might not have anticipated, enriching the training data with non-obvious strategies. The CFR Gain results (Table 1) establish that the generated dialogue is more strategic than unguided LLM output, and the low NashConv values suggest the strategies are relatively stable—they won't collapse under modest adversarial pressure. The practical bottleneck is computational cost: each game tree requires many LLM calls to solve, as the paper acknowledges, and generating thousands of training dialogues would require either accepting significant latency or amortizing through the imitation learning pipeline (Section 4, Appendix E), which currently fails to generalize in the meeting scheduling domain (44% preference for DIM−LLM).

Strategic routing in conversational AI systems with variable task criticality. An organization deploying conversational agents for customer service, scheduling, or procurement could use the framework's distinction between unguided (DLLM) and guided (DCFR−LLM) behavior to implement a tiered system. Low-stakes interactions (informational queries, routine scheduling with flexible constraints) use the baseline DLLM with "Any" instruction, incurring no solver overhead. High-stakes interactions (negotiating a contract renewal, resolving a dispute over resource allocation, scheduling with tight constraints and asymmetric preferences) invoke the equilibrium solver to compute a strategic policy given the specific game parameters (private valuations, action set, reward structure). The paper's evidence that DCFR−LLM is an approximate ESS (positive CFR Gain, low NashConv across all domains, Table 1) means that in high-stakes settings, the guided agent is both more effective against naive counterparts and less exploitable by strategic ones. The difficulty estimation problem—determining when an interaction is "high-stakes enough" to justify the solver overhead—is not addressed by the paper but is analogous to the difficulty estimation problem in the test-time compute scaling literature: estimate the strategic complexity of the situation before deciding how much inference-time computation to invest.

When to Prefer This Method

The paper does not position its framework against named alternative methods with explicit tradeoff criteria. It compares against an internal baseline (DLLM with "Any" action) but does not, for example, compare equilibrium-guided LLMs to end-to-end trained negotiation agents (Lewis et al., 2017), to LLMs with fixed prompting strategies, or to LLMs with chain-of-thought reasoning about strategy. The paper's Limitations section primarily discusses constraints of the framework itself rather than conditions under which a practitioner should choose it over alternatives. The "prefer this method" framing would therefore require imposing a tradeoff structure that the paper itself does not articulate, which falls outside the scope of analyzing the provided paper content.