Modeling Multiple Support Strategies within a Single Turn for Emotional Support Conversations

🎯 Pitch

Nearly one in five supportive utterances naturally combine multiple strategies—like self-disclosure, affirmation, and a question—yet all prior systems forced a single strategy per turn. By letting models generate multi-strategy utterances with iterative reasoning and RL, this work doubles dialogue success rates while shortening conversations, proving that rigid one-strategy-per-turn modeling fundamentally cripples emotional support.

1. Executive Summary

This paper revisits emotional support conversation (ESC) by reformulating it as a multi-strategy utterance generation task, where each supporter turn may contain one or more strategy-response pairs rather than the conventional single-strategy assumption. Using the ESConv benchmark with LLaMA-3.1-8B-Instruct as the backbone, the authors propose two generation methods—All-in-One (generating all strategy-response pairs in a single decoding step) and One-by-One (iteratively predicting strategy-response pairs until a termination flag signals completion)—both enhanced with structured cognitive reasoning distilled from multiple large language models and further optimized via Group Relative Policy Optimization (GRPO) reinforcement learning. The best configuration, One-by-One with reasoning and RL, achieves a dialogue-level success rate of 40.00% compared to 13.85% for the single-strategy baseline, while also reducing average dialogue turns from 9.56 to 8.46, establishing that modeling multiple strategies per utterance is both feasible and beneficial for emotional support conversations only when the base task is properly augmented with explicit reasoning and reinforcement-driven strategy selection.

2. Context and Motivation

The Core Problem: Real Supportive Conversations Don't Follow a One-Strategy-per-Turn Script

The fundamental disconnect this paper addresses is a mismatch between how emotional support conversations are modeled in NLP systems and how they naturally occur between humans. Since Liu et al. (2021) introduced the ESConv dataset and formalized the ESC task, the research community has almost universally adopted a simplifying assumption: each utterance from a supporter corresponds to exactly one support strategy. Under this formulation, a model trained on ESConv predicts — given the dialogue history — a single strategy label (e.g., "Question," "Reflection of feelings," "Providing Suggestions") and generates a response aligned with that strategy.

This assumption is analytically convenient. It decomposes the generation problem into two clean sub-tasks: strategy prediction (essentially a classification problem over eight strategy types) and strategy-conditioned response generation (sequence-to-sequence generation with a known intent). It also aligns naturally with standard supervised learning pipelines — each training instance maps one context to one strategy-response pair.

But it is demonstrably false for the very dataset that anchors the field. As the paper shows in Table 1, across the 15,325 supporter utterances in ESConv, 17.7% contain two or more strategies. Specifically: 2,427 utterances (15.8%) use two strategies, 256 utterances (1.7%) use three, and 34 utterances (0.2%) use four or more. These are not rare edge cases — they represent nearly one in every five turns of supportive dialogue.

The consequence of the one-strategy-per-turn assumption is that existing models are trained on a distorted representation of supportive communication. When a training example contains a single strategy label but the original utterance actually employed two strategies (say, the supporter first validated the seeker's feelings with "Affirmation and Reassurance" and then asked a "Question" to explore further), the model never learns to compose multiple supportive functions within a single turn. At inference time, it produces one strategy per utterance, which the paper shows leads to dialogues that are less efficient (requiring more turns to achieve the same emotional progress) and less effective (achieving lower success rates).

The problem extends beyond just generation accuracy. It's about dialogue efficiency and naturalness. A human supporter who delivers emotional validation and a practical suggestion in the same turn is both more effective and more conversational than someone who rigidly separates those functions across two turns. The one-strategy assumption forces models into a stilted interaction pattern that departs from how skilled supporters actually communicate.

Why This Problem Matters: Real-World Impact and Theoretical Significance

Real-world impact. Emotional support conversation systems sit at a sensitive intersection of NLP and mental well-being. Unlike task-oriented dialogue (booking a flight, checking a bank balance) where efficiency is measured in slot-filling accuracy, ESC success depends on perceived empathy, emotional attunement, and the ability to make the seeker feel understood. Systems that can naturally weave together multiple supportive strategies — validating feelings while simultaneously offering perspective, or sharing a personal experience while gently probing for more information — are more likely to create the kind of conversational flow that builds trust and facilitates emotional relief.

The paper's dialogue-level results quantify this impact concretely. A single-strategy model achieves a 13.85% success rate on simulated emotional support dialogues. The best multi-strategy model reaches 40.00%. That's not an incremental improvement — it's nearly a tripling of task success. For practical deployment of ESC systems (whether as standalone supportive chatbots, components of mental health applications, or assistive tools for human counselors), this gap represents the difference between a system that rarely helps and one that succeeds in a substantial fraction of interactions.

There's also a resource efficiency argument. The multi-strategy approach achieves higher success rates while using fewer dialogue turns on average (8.46 vs. 9.56 for the single-strategy baseline). In real deployments where API costs or latency matter, delivering more effective support in fewer turns translates directly to better user experience and lower operational costs.

Theoretical significance. Beyond the practical gains, the paper addresses a deeper question about human communication modeling: should dialogue systems treat utterance-level structure as an emergent property of the task, or should they explicitly model the compositional nature of real utterances? The ESC literature to date has implicitly chosen the former — assuming that one strategy per turn is a reasonable approximation. This paper provides the first systematic empirical evidence that this approximation is harmful rather than neutral, and that modeling the compositional structure of utterances (where multiple discourse functions can co-occur) yields measurable improvements in dialogue quality.

This connects to broader interests in NLP around structured generation and compositional sequence modeling. The All-in-One and One-by-One methods represent two different architectural choices for handling variable-length structured outputs (flat generation vs. iterative prediction with a learned stopping criterion), and the paper's comparison of these approaches provides evidence about which decomposition works better for this specific structure. The finding that One-by-One outperforms All-in-One (33.53 vs. 29.97 EMR with reasoning + RL) suggests that breaking the problem into sequential sub-decisions — each with the full dialogue context and previously generated strategies — enables better strategy selection and ordering than predicting everything at once.

Where Prior Approaches Fall Short

The paper situates its contribution against several threads of prior work, each of which addresses parts of the ESC challenge but leaves the multi-strategy problem unexamined.

The ESConv baseline and its lineage (Liu et al., 2021). The foundational ESC paper established the task as generating a single strategy-conditioned response per turn. This formulation enabled rapid progress — subsequent work could focus on improving strategy prediction accuracy or response quality without questioning the one-strategy assumption. But it also locked in a modeling constraint that the paper now shows is suboptimal. The baseline models that Liu et al. (2021) and most follow-up work evaluate against are all single-strategy systems, meaning the field has been optimizing for a task formulation that underspecifies what real supporters actually do.

Knowledge-enhanced and persona-aware approaches. Several lines of work aim to improve ESC by incorporating external knowledge about the seeker's emotional state, needs, or persona (Tu et al., 2022; Peng et al., 2022; Cheng et al., 2023; Hao and Kong, 2025). These methods use COMET for commonsense reasoning, graph networks for integrating seeker information, or persona extraction to better tailor responses. While they improve single-strategy response quality, none of them address the structural question of whether a turn should contain one or multiple strategies. They could, in principle, be combined with the multi-strategy framework — using better seeker understanding to inform which strategies to compose and how to order them — but as standalone contributions they inherit the one-strategy assumption.

Strategy planning across turns. Some work models strategy sequences at the dialogue level rather than the turn level (Cheng et al., 2022; Zhao et al., 2023). These approaches use lookahead planning or turn-level state transition modeling to decide which strategy to use next given the dialogue trajectory so far. This is a form of structural modeling — it captures dependencies between strategies across multiple turns — but it still assumes exactly one strategy per turn. The paper's contribution is orthogonal: it asks what happens when each turn can itself be structurally complex, with multiple strategies. In principle, dialogue-level strategy planning could be combined with turn-level multi-strategy generation, creating a hierarchical structure where a high-level planner selects a set of strategies for an upcoming turn, and a low-level generator composes them into a coherent utterance.

Reasoning-augmented ESC (Chen et al., 2025; Zhu et al., 2025). The most directly relevant prior work is the line of research that augments ESC models with explicit reasoning mechanisms. Chen et al. (2025) propose SocialSim, which incorporates chain-of-thought reasoning for socialized simulation of ESC. Zhu et al. (2025) introduce CARE, which uses a four-node cognitive reasoning chain (Context, Cognition, Emotion, Support Plan) and reinforcement learning to improve strategy selection and response generation. The current paper adopts the CARE reasoning framework directly — the four-node structure, the distillation from multiple LLMs, and the GRPO-based RL optimization are all inherited from Zhu et al. (2025). The novel contribution is applying this reasoning framework specifically to the multi-strategy generation setting and comparing two different generation architectures for handling the compositional output structure.

This is a crucial point of positioning: the paper's primary novelty is not the reasoning mechanism or the RL setup (both prior work), but rather the demonstration that these techniques are particularly valuable when the model must coordinate multiple strategies within a single turn, and that modeling multi-strategy utterances is itself beneficial when properly supported by reasoning and RL.

The only prior multi-strategy work: Bai et al. (2025). To the authors' knowledge, Bai et al. (2025) is the only prior work that explicitly investigates multi-strategy generation within a single turn for ESC. The current paper acknowledges this as the closest related contribution and frames its own work as addressing the question Bai et al. raised but left unresolved: "whether allowing multiple strategies in emotional support conversations is beneficial remains unclear." The current paper positions itself as providing the first systematic empirical evidence on this question, using two complementary evaluation settings (utterance-level and dialogue-level) and showing that the answer depends on having adequate reasoning and RL support.

How This Paper Positions Itself

The paper makes a clear argument about what it contributes and what it inherits:

The novel contribution is the multi-strategy formulation and its evaluation. The paper does not claim to invent cognitive reasoning for ESC (that's from Zhu et al., 2025), nor does it claim to invent the All-in-One and One-by-One generation paradigms (structured output generation has a long history in NLP). What it claims is: (1) reformulating ESC to allow multiple strategies per utterance, (2) comparing two generation approaches for this formulation, (3) showing that reasoning and RL are necessary to make multi-strategy generation work well, and (4) providing the first dialogue-level evidence that multi-strategy utterances improve ESC outcomes.

The paper positions multi-strategy generation as improving efficiency, not just accuracy. The authors are careful to frame the benefit of multi-strategy generation in terms of dialogue efficiency: fewer turns to achieve emotional progress, not just better n-gram overlap with references. The utterance-level evaluation (Table 2) establishes that the models can generate multi-strategy utterances that match references in surface form, but the dialogue-level evaluation (Table 6) makes the stronger case — that these utterances lead to more successful conversations. This is important because surface-level automatic metrics can be misleading for dialogue (a model could achieve high BLEU by reproducing common supportive phrases without actually helping the seeker). The dialogue-level results, triangulated with human evaluation (Table 7), provide evidence that the multi-strategy approach improves the actual task objective.

The paper is candid about remaining gaps. In the limitations section, the authors acknowledge that the proportion of multi-strategy utterances generated by their models (8.4% for All-in-One, 7.7% for One-by-One with reasoning + RL) is still substantially lower than the 18.9% in the ESConv reference data. This honesty is informative: it tells us that even with reasoning and RL, the models are conservative about using multiple strategies, likely because the training data is dominated by single-strategy instances (82.3% of turns). This suggests a direction for future work — data balancing, explicit multi-strategy encouragement in the reward function, or architectural biases toward strategy composition — and the paper's clear quantification of the gap makes it easier for subsequent work to measure progress.

The paper connects to broader questions about structured generation. While the paper is focused on ESC specifically, the All-in-One vs. One-by-One comparison touches on a fundamental design question in sequence generation: when the output has compositional structure (here, a variable-length sequence of strategy-response pairs), should you generate the whole structure at once or break it into sequential decisions? The paper's finding that One-by-One outperforms All-in-One (particularly in EMR: 33.53 vs. 29.97) provides empirical evidence favoring iterative generation for this type of structure, which may generalize to other tasks where the output is a variable-length sequence of typed segments.

3. Technical Approach

3.1 Reader Orientation

The system is a fine-tuned language model (LLaMA-3.1-8B-Instruct) that acts as an emotional support conversation agent, generating supportive utterances in response to a help-seeker's statements. It solves the problem that real human supporters often use multiple strategies (e.g., validating feelings and offering suggestions) within a single conversational turn, but prior ESC systems were constrained to generating exactly one strategy per utterance, making their responses less natural and less efficient. The solution's shape is a model that can produce a variable-length sequence of (strategy, response) pairs within a single turn, enhanced with explicit reasoning about the seeker's emotional state and optimized through reinforcement learning to select and sequence strategies effectively.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components organized in a training-then-inference pipeline:

Base Language Model (LLaMA-3.1-8B-Instruct): The pretrained transformer that serves as the foundation for all generation. It is fine-tuned via LoRA for efficiency.
Multi-Strategy Output Formatter: Defines how strategy-response pairs are structured in the model's output. Two variants exist: All-in-One, which concatenates all pairs into a flat sequence [strategy₁, response₁, strategy₂, response₂, ...] and generates them in a single autoregressive pass; and One-by-One, which iteratively generates individual (strategy, response, continue_flag) triples, using a learned binary flag to decide when the utterance is complete.
Cognitive Reasoning Module: A four-node reasoning chain (Context, Cognition, Emotion, Support Plan) that the model generates before producing the supportive utterance. This chain is supervised through distillation from four large teacher LLMs (DeepSeek-R1, Qwen3-235B, GPT-5, Gemini-2.5-Flash) and is included in both SFT training data and RL generation.
Group Relative Policy Optimization (GRPO) Trainer: A reinforcement learning stage that optimizes the SFT model using task-specific reward functions. The reward evaluates both format validity (whether the output follows the required structural template) and strategy accuracy (Levenshtein Ratio between predicted and reference strategy sequences). For One-by-One, an additional reward term encourages correct termination flag prediction.
Seeker Simulator and Critic (for dialogue-level evaluation): GPT-5 acts as the seeker in self-play dialogues, generating responses to the supporter's utterances. A separate GPT-5 instance serves as a critic, evaluating after each turn whether the seeker's emotional distress has been alleviated and producing a scalar progress score.

Information flows as follows: Dialogue context (seeker and supporter utterances) enters the system → the model generates a cognitive reasoning chain (Context → Cognition → Emotion → Support Plan) → based on this reasoning, the model generates one or more strategy-response pairs (either all at once via All-in-One or iteratively via One-by-One) → the complete utterance is returned as the supporter's turn → at the dialogue level, the seeker simulator responds, and the critic evaluates progress.

3.3 Roadmap for the Deep Dive

First, the formal task definition and output representation (Section 2.2 material, revisited in technical detail), because understanding what the model predicts (variable-length strategy-response sequences) is prerequisite to understanding how it predicts it.
Second, the All-in-One generation method, since it is the conceptually simpler approach that generates the full utterance in one pass, establishing a baseline for multi-strategy generation.
Third, the One-by-One generation method, which decomposes the problem into iterative sub-decisions and introduces the termination prediction mechanism that distinguishes it from All-in-One.
Fourth, the cognitive reasoning framework and distillation procedure, since this is the shared enhancement applied to both generation methods and is critical to their performance.
Fifth, the reinforcement learning setup with GRPO, including the reward functions for both methods, the format constraint, and the data balancing strategy — since RL drives the largest performance gains and is where the two methods diverge most in their optimization objectives.
Sixth, the training pipeline (SFT → RL) and hyperparameters, including LoRA configuration, data sizes, and hardware setup, to provide a complete implementation picture.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a methods and empirical evaluation paper whose core idea is that allowing multiple support strategies per utterance improves emotional support conversation quality, but only when the model is equipped with structured reasoning and reinforcement learning to coordinate which strategies to use and in what order.

Task Formalization and Output Representation

The paper defines the ESC task with a multi-strategy output structure. Let the dialogue context be $U = \{u_{1}^{\text{usr}}, u_{1}^{\text{sys}}, u_{2}^{\text{usr}}, u_{2}^{\text{sys}}, \ldots, u_{t}^{\text{usr}}\}$ , where $u_i^{\text{usr}}$ is the seeker's utterance at turn $i$ and $u_i^{\text{sys}}$ is the supporter's utterance at turn $i$ . The current turn is $t$ , meaning the seeker has just spoken ( $u_t^{\text{usr}}$ ) and the model must now generate the supporter's response $u_t^{\text{sys}}$ .

The critical departure from prior work is the decomposition of the supporter's utterance:

$u_t^{\text{sys}} = \{r_{t,1}^{\text{sys}}, \ldots, r_{t,N_t}^{\text{sys}}\}$

where $N_t$ is the number of distinct response segments within the utterance, and each segment $r_{t,i}^{\text{sys}}$ is associated with exactly one support strategy $s_{t,i}^{\text{sys}}$ drawn from the eight types defined in ESConv: Question, Restatement or Paraphrasing, Reflection of feelings, Self-disclosure, Affirmation and Reassurance, Providing Suggestions, Information, and Others.

What this decomposition represents: Each supportive utterance is not a monolithic block of text but a composition of one or more discourse functions, each implemented by a strategy-aligned text segment. For example, a supporter might first validate the seeker's feelings (Affirmation and Reassurance), then share a related personal experience (Self-disclosure), and finally ask an exploratory question (Question) — all within a single turn. The utterance is the concatenation of these segments in their natural order.

Why this formulation: The one-strategy-per-turn formulation used in prior work corresponds to the special case where $N_t = 1$ for all $t$ . By allowing $N_t \geq 1$ , the model can produce utterances that match the 17.7% of ESConv turns that naturally contain multiple strategies. This is not merely a representational convenience — it changes what the model must learn. Rather than selecting a single strategy and generating text conditioned on it, the model must now decide how many strategies to use, which ones, in what order, and how to compose their associated text segments into a coherent utterance. These are interdependent decisions that the single-strategy formulation collapses into one choice.

The training data for each instance includes the dialogue context $c^{(i)}$ (all preceding seeker and supporter turns) and the target output $y^{(i)}$ . The structure of $y^{(i)}$ differs between the two generation methods, as described below.

All-in-One Generation Method

The All-in-One method treats the entire supportive utterance — all strategy-response pairs — as a single flat sequence to be generated in one autoregressive decoding pass.

Output format. The target sequence is constructed by concatenating each strategy tag with its corresponding response text, in order:

$y_t^{\text{sys}} = \text{concat}\left(s_{t,1}^{\text{sys}}, r_{t,1}^{\text{sys}}, \ldots, s_{t,N_t}^{\text{sys}}, r_{t,N_t}^{\text{sys}}\right)$

For example, if the supporter uses two strategies, the target might be: [Affirmation and Reassurance] I understand being angry about that. [Providing Suggestions] Maybe having a professional review your resume would help. The strategy tags serve dual roles: they are both prediction targets (the model must decide which strategy to use) and structural delimiters (they mark boundaries between response segments in the flat output).

Training objective. Given a training set $\mathcal{D} = \{(c^{(i)}, y^{(i)})\}_{i=1}^{N}$ where $c^{(i)}$ is the dialogue context and $y^{(i)}$ is the concatenated strategy-response sequence, the model is trained with standard autoregressive language modeling:

$\mathcal{L}(\mathcal{D}) = -\frac{1}{N} \sum_{i=1}^{N} \sum_{j=1}^{|y^{(i)}|} \log P\left(y_j^{(i)} \mid p_1, y_{<j}^{(i)}, c^{(i)}\right)$

where $p_1$ is the prompt template for All-in-One (shown in Appendix E.2, Figure 6 without reasoning and Figure 7 with reasoning), $y_j^{(i)}$ is the $j$ -th token of the target sequence, and $y_{<j}^{(i)}$ are all preceding tokens.

What it computes: For each training instance, the model processes the dialogue context (formatted according to the prompt template $p_1$ ) and predicts the entire strategy-response sequence token by token, left to right. The loss is the average negative log-likelihood across all tokens in all training instances — the standard cross-entropy objective for sequence generation. The model learns to model $P(\text{strategy}_1, \text{response}_1, \text{strategy}_2, \text{response}_2, \ldots \mid \text{context})$ directly as a joint distribution over the flat token sequence.

Why this form: This is the simplest possible adaptation of standard language model training to the multi-strategy setting — it requires no architectural changes, no special decoding logic, and no auxiliary prediction heads. The model treats the structured output as ordinary text and learns to reproduce it from supervised examples. The implicit assumption is that the model's autoregressive attention mechanism can capture the dependencies between strategy choices and response generation across multiple segments within a single forward pass. The downside is that the model must predict the entire sequence length (how many strategies plus all their associated text) in one shot, with no opportunity to condition later strategy choices on the text already generated for earlier ones — the autoregressive factorization handles token-by-token conditioning, but the high-level decision of "how many strategies and in what order" is not explicitly decomposed.

Inference. At test time, given the dialogue context, the model generates the complete sequence exactly as during training, with no iterative or multi-step procedure. The structure of the output (how many strategy-response pairs) emerges from the model's token-by-token predictions, just as paragraph length emerges from standard text generation.

One-by-One Generation Method

The One-by-One method decomposes multi-strategy utterance generation into an iterative process: at each step, the model predicts exactly one strategy, its associated response, and a binary termination flag that indicates whether to continue or stop.

Output format. For the $i$ -th step of generation on turn $t$ , the model produces a triple:

$y_{t,i}^{\text{sys}} = \text{concat}\left(s_{t,i}^{\text{sys}}, r_{t,i}^{\text{sys}}, f_{t,i}^{\text{sys}}\right)$

where $f_{t,i}^{\text{sys}} \in \{\text{true}, \text{false}\}$ is a binary termination indicator. If $f_{t,i}^{\text{sys}} = \text{false}$ , the model will generate another strategy-response pair ( $i \rightarrow i+1$ ). If $f_{t,i}^{\text{sys}} = \text{true}$ and $i = N_t$ , the utterance is complete. At inference time, generation stops when either the flag signals completion or a predefined maximum number of steps $K = 3$ is reached. The choice of $K = 3$ is based on the observation that "nearly all utterances in the validation set contain at most three strategies."

Training objective. Analogous to All-in-One, but the training set $\mathcal{E} = \{(c^{(i)}, y^{(i)})\}_{i=1}^{M}$ now contains the concatenated strategy-response-flag sequences:

$\mathcal{L}(\mathcal{E}) = -\frac{1}{M} \sum_{i=1}^{M} \sum_{j=1}^{|y^{(i)}|} \log P\left(y_j^{(i)} \mid p_2, y_{<j}^{(i)}, c^{(i)}\right)$

where $p_2$ is the prompt template for One-by-One (shown in Appendix E.3, Figure 9 without reasoning and Figure 10 with reasoning).

What it computes: The training objective is superficially identical to All-in-One — autoregressive token prediction — but the target sequences $y^{(i)}$ are constructed differently. Instead of containing all strategy-response pairs for a turn, each training instance contains exactly one (strategy, response, continue_reply) triple. This means that for a turn with $N_t$ strategies, there are $N_t$ separate training instances, each with the same dialogue context but with the previously generated strategies (from earlier steps) included in the context. The termination flag $f$ is false for all steps except the last, where it is true.

Why this decomposition matters: By training on individual steps, the model learns a conditional distribution $P(\text{strategy}_i, \text{response}_i, \text{continue}_i \mid \text{context}, \text{previous strategies and responses})$ . At inference time, this enables adaptive termination: the model can decide after each strategy-response pair whether the utterance feels complete or whether another strategy is needed. This is fundamentally different from All-in-One, where the number of strategies is determined before any text is generated. The iterative approach also means that later strategies are explicitly conditioned on both the dialogue context and the content of earlier strategies within the same turn, which the All-in-One model can only capture implicitly through its internal representations.

Inference procedure. The model repeatedly:

Takes the dialogue context plus any previously generated (strategy, response) pairs for this turn.
Generates one (strategy, response, continue_reply) triple.
If continue_reply is false (or $K = 3$ steps have been reached), stops and returns the concatenation of all generated responses.
Otherwise, appends the new pair to the context and returns to step 1.

Instance counts. The paper reports that SFT training without reasoning uses 10,679 instances for All-in-One (the original ESConv training utterances, one per turn) but 12,759 instances for One-by-One (Table 8). The increase comes from the decomposition: a single three-strategy turn becomes three training instances for One-by-One. With reasoning, both methods use substantially more instances (42,222 for All-in-One, 51,035 for One-by-One) because each reasoning-augmented utterance is a separate training example, and the reasoning is distilled from four teacher LLMs, further multiplying the data.

Why the One-by-One approach is expected to help: By forcing the model to make explicit, step-by-step decisions about whether to continue, it introduces a learned stopping criterion that the model can calibrate based on the dialogue context and what it has already said. This prevents both premature termination (generating one strategy when two would be better) and excessive continuation (generating redundant strategies). The paper's results confirm this expectation: One-by-One achieves higher EMR (33.53 vs. 29.97) and higher dialogue success rates (40.00% vs. 34.62%) compared to All-in-One in the best configuration (reasoning + RL), suggesting that the iterative decomposition provides a better inductive bias for multi-strategy composition.

Cognitive Reasoning Framework

Both generation methods are enhanced with a structured cognitive reasoning chain that the model generates before producing the supportive utterance. The reasoning chain is adopted directly from Zhu et al. (2025)'s CARE framework and consists of four nodes:

Context Node: Models the external situation and salient emotional cues from the seeker's most recent message. For example: "The seeker describes academic pressure and fears of not meeting expectations." This grounds the reasoning in observable facts.
Cognition Node: Captures the seeker's internal interpretations, beliefs, or self-assessments. For example: "The seeker believes their worth depends on academic achievement." This infers the cognitive appraisals underlying the emotional response.
Emotion Node: Represents the emotional states emerging from the cognition. For example: "The seeker feels anxiety about potential failure and shame about perceived inadequacy." This explicitly names the affective states the supporter must address.
Support Plan Node: Specifies the intended strategies and their sequential purpose. For example: "Step 1: Using [Affirmation and Reassurance] to validate the seeker's effort. Step 2: Using [Providing Suggestions] to offer concrete coping strategies." This bridges understanding to action.

Integration into the output format. The original model output $y^{(i)}$ is extended to include the reasoning chain before the supportive content:

$y^{(i)\prime} = \text{thinking } r^{(i)} \text{ response } \langle\text{answer}\rangle y^{(i)} \langle/\text{answer}\rangle$

where $r^{(i)} = \langle r^{(i)}_{\text{ctx}}, r^{(i)}_{\text{cog}}, r^{(i)}_{\text{emo}}, r^{(i)}_{\text{plan}} \rangle$ is the four-node reasoning chain, and $y^{(i)}$ is the strategy-response output as defined for All-in-One or One-by-One.

What this augmentation achieves: The model must now learn to generate a structured reasoning trace before generating the supportive utterance. At inference time, this means the model explicitly considers the seeker's situation, interpretations, emotions, and the rationale for strategy choice before producing any text that the seeker sees. The reasoning trace is enclosed in thinking tags and the actual response in <answer> tags, making the separation explicit in the training data and learnable by the model.

Why this structure: The four-node decomposition maps onto established psychological frameworks for helping conversations (Hill, 2019), where effective support involves understanding the situation (Context), identifying cognitive patterns (Cognition), attuning to emotions (Emotion), and planning interventions (Support Plan). By supervising the model to generate this structure, the authors aim to make the strategy selection process more deliberate and interpretable — rather than the model implicitly guessing which strategy fits, it must articulate why each strategy is appropriate. The reasoning chain also provides a form of regularization: the model cannot simply memorize strategy-context co-occurrences from the training data; it must learn the mediating cognitive steps that justify a particular strategy choice.

Distillation from multiple teacher LLMs. The reasoning chains are not human-annotated. Instead, for each training instance, the authors obtain reasoning annotations from four large language models: DeepSeek-R1 (Guo et al., 2025), Qwen3-235B-A22B-Instruct-2507 (Yang et al., 2025), GPT-5 (OpenAI, 2025), and Gemini-2.5-Flash (Google, 2024). The distillation prompt (Appendix E.2, Figure 8 for All-in-One; Appendix E.3, Figure 11 for One-by-One) provides each teacher with the dialogue context and the gold supporter reply (with strategy tags) and asks it to generate the reasoning chain as a JSON object with exactly four fields.

The rationale for using multiple teachers is explicitly stated: "Distilling from multiple teacher models allows us to capture diverse reasoning styles and complementary strengths, reducing the bias or idiosyncrasies of any single source and improving robustness and generalization of the student model." The paper validates this claim empirically in Table 5: no single teacher dominates across all metrics (GPT-5 is best on BLEU-4, Gemini on ROUGE-L, DeepSeek-R1 on BERTScore and EMR), and combining all four yields the strongest overall performance (EMR 29.72 vs. 23.61–28.46 for individual teachers). This pattern suggests that the teachers have genuinely different reasoning styles that capture complementary aspects of supportive dialogue, and the student model benefits from seeing this diversity.

Training data expansion. The distillation from four teachers multiplies the training data. Table 8 shows: SFT without reasoning uses 10,679 instances for All-in-One; SFT with reasoning uses 42,222 instances — approximately a $4\times$ increase, consistent with each training utterance getting reasoning from all four teachers. For One-by-One, the increase is from 12,759 to 51,035 instances, again approximately $4\times$ . This data expansion is a significant factor in the performance gains observed with reasoning: the model sees each dialogue context multiple times with different reasoning chains, potentially learning more robust strategy-response mappings.

Reinforcement Learning with GRPO

After supervised fine-tuning, both methods are further optimized using Group Relative Policy Optimization (GRPO), a reinforcement learning algorithm introduced by Shao et al. (2024). The RL stage uses task-specific reward functions that evaluate the quality of the model's generated outputs against structural and strategic criteria.

GRPO overview. GRPO is a policy gradient method that, for each prompt, samples a group of multiple outputs from the current policy (the model being trained), scores each with a reward function, and updates the policy to increase the probability of high-reward outputs relative to the group average. The key advantage over standard PPO is that GRPO does not require a separate value function (critic) model; it uses the average reward within each group as a baseline, reducing memory and computational requirements.

Format reward (shared across methods). Every generated output $y$ is first checked for structural validity using a format reward:

$r_{\text{fmt}}(y) = \begin{cases} 1 & \text{if } y \text{ follows all required formatting constraints} \\ 0 & \text{otherwise} \end{cases}$

What this checks: The output must be structurally compliant — for All-in-One, it must be a valid JSON array of (strategy, text) objects; for One-by-One, it must be a valid JSON object with strategy, text, and continue_reply fields. If the model produces unparseable output, the format reward is zero, and all other reward components are also zeroed out (since strategy extraction fails).

Why this form: The binary format reward acts as a hard constraint that gates all other rewards. This ensures the RL optimization does not sacrifice output structure for strategy accuracy — a model that learns to produce high-quality strategies but in an unparseable format would be useless at inference time. The binary nature (rather than a graded score) makes the constraint clear: either the output is structurally valid and receives strategy-based rewards, or it is not and receives zero total reward.

All-in-One reward function. For All-in-One, the strategy-level reward compares the predicted strategy sequence $s_y$ (extracted from the generated output $y$ ) against the reference strategy sequence $s_r$ using the Levenshtein Ratio:

$LR(s_y, s_r) = 1 - \frac{\text{Levenshtein Distance}(s_y, s_r)}{\max(\text{len}(s_y), \text{len}(s_r))}$

The final reward is:

$r_1(y) = \begin{cases} LR(s_y, s_r) & \text{if } r_{\text{fmt}}(y) = 1 \\ 0 & \text{otherwise} \end{cases}$

What Levenshtein Ratio computes: The Levenshtein Distance is the minimum number of single-element edits (insertions, deletions, substitutions) required to transform the predicted strategy sequence into the reference sequence. The Levenshtein Ratio normalizes this distance by the length of the longer sequence, mapping it to $[0, 1]$ where $1$ means exact match and $0$ means completely different. For example, if the reference is [Affirmation, Question] and the prediction is [Question, Affirmation], the distance is 2 (one deletion + one insertion, or two substitutions), and the ratio is $1 - 2/2 = 0$ — order matters. If the prediction is [Affirmation] (missing one strategy), the distance is 1 (one deletion) and the ratio is $1 - 1/2 = 0.5$ — partial credit.

Why Levenshtein Ratio over exact match: Exact match would give zero reward unless the predicted sequence is identical to the reference in both content and order. This is too strict for RL — the model would rarely receive positive reinforcement, making learning inefficient. The Levenshtein Ratio provides a graded signal: getting the right strategies but in the wrong order is better than getting completely different strategies, and getting a subset of the right strategies is better than getting none. This is appropriate because strategy ordering matters but is not the only thing that matters; an utterance that uses the correct strategies in suboptimal order is still partially helpful.

Data balancing for RL. Because multi-strategy utterances are rarer in the training data (only 17.7% of turns), the reward distribution would be dominated by single-strategy instances where the exact match rate is naturally higher. The paper states: "Since multi-strategy instances are fewer, we down-sample single-strategy instances to balance the reward distribution." The specific down-sampling ratio is not given, but the intent is to ensure that the RL optimization sees roughly equal numbers of single- and multi-strategy instances, preventing the policy from collapsing to always generating one strategy (which would maximize reward on the majority of the data but fail on multi-strategy cases).

One-by-One reward function. For One-by-One, the reward adds a term for correct termination flag prediction:

$r_2(y) = \begin{cases} LR(s_y, s_r) + r_{\text{flag}}(y) & \text{if } r_{\text{fmt}}(y) = 1 \\ 0 & \text{otherwise} \end{cases}$

where $r_{\text{flag}}(y) = 1$ if the predicted continue_reply flag matches the reference and $0$ otherwise.

What the flag reward adds: The termination flag is a binary prediction that was not present in the All-in-One method. The extra reward term encourages the model to learn when to stop generating strategies — a critical capability for the iterative approach. If the model never learns to stop (always predicting continue_reply = true), it would generate strategy-response pairs until the hard limit $K = 3$ , producing redundant or forced strategies. If it always stops after one strategy (predicting continue_reply = false immediately), it would degenerate to the single-strategy baseline. The flag reward explicitly penalizes both types of error.

Why the reward functions differ: The All-in-One method has no termination flag, so the reward focuses entirely on strategy sequence accuracy. The One-by-One method must additionally learn the stopping behavior, which motivates the extra reward component. The sum (rather than, say, a weighted combination) treats strategy accuracy and flag accuracy as equally important, which is a design choice — it assumes that getting the right strategies is neither more nor less important than knowing when to stop using them.

RL hyperparameters. From Appendix B: RL is run for 7 epochs using the VERL framework (Sheng et al., 2025) with the GRPO algorithm, on 4 NVIDIA A100 80GB GPUs. The batch size is 1024, rollout size is 16 (each prompt generates 16 candidate outputs for group-relative comparison), learning rate is $1 \times 10^{-6}$ , and KL-penalty coefficient is fixed at $0.01$ . The KL penalty discourages the policy from deviating too far from the SFT starting point, which is standard practice to prevent reward hacking (where the model finds outputs that score highly under the reward function but are nonsensical or unhelpful).

RL training instances. Table 8 shows: All-in-One uses 3,696 RL instances; One-by-One uses 12,759 RL instances. The large difference is notable — One-by-One has roughly $3.5\times$ more RL data. This is because the One-by-One method generates one training instance per strategy-response pair (so a turn with three strategies yields three RL instances), while All-in-One generates one instance per turn. The paper does not discuss whether this data advantage contributes to One-by-One's superior performance (33.53 vs. 29.97 EMR), but it is a plausible contributing factor.

Training Pipeline and Hyperparameters

Supervised Fine-Tuning (SFT) stage. All models use LLaMA-3.1-8B-Instruct as the backbone. Fine-tuning is performed with LoRA (Low-Rank Adaptation) using the LLaMA-Factory framework (Zheng et al., 2024). For All-in-One: LoRA rank $r = 8$ , LoRA scaling factor $\alpha = 16$ . For One-by-One: LoRA rank $r = 16$ , LoRA scaling factor $\alpha = 32$ . The doubled LoRA dimensions for One-by-One suggests the authors found the iterative method benefits from more adapter capacity, possibly because it must learn both the generation task and the termination decision, or because the training data is larger (12,759 vs. 10,679 instances without reasoning; 51,035 vs. 42,222 with reasoning).

SFT uses: per-device batch size 4, gradient accumulation over 2 steps (effective batch size 8), learning rate $3 \times 10^{-5}$ , training for 5 epochs, on 4 NVIDIA A100 80GB GPUs. The learning rate of $3 \times 10^{-5}$ is a standard choice for LoRA fine-tuning of 8B-parameter models.

Data sizes. The instance counts in Table 8 reveal the scale of data used:

SFT without reasoning: 10,679 instances for All-in-One (the original ESConv training turns, one per supporter utterance); 12,759 for One-by-One (decomposed into per-step instances).
SFT with reasoning: 42,222 for All-in-One; 51,035 for One-by-One. These are approximately $4\times$ the no-reasoning counts, consistent with distillation from four teacher LLMs (each original turn gets reasoning chains from all four teachers, producing four reasoning-augmented training instances).
RL: 3,696 for All-in-One (after down-sampling single-strategy instances); 12,759 for One-by-One (maintaining the per-step granularity).

Hardware and cost. Training uses 4 NVIDIA A100 80GB GPUs. The paper reports an approximate total API cost of 800 USD for data distillation, self-play evaluation, and other processes involving advanced model API calls (GPT-5, Gemini, DeepSeek-R1, Qwen3).

Why LoRA: Full fine-tuning of an 8B-parameter model would require substantially more GPU memory and compute. LoRA keeps the base model frozen and adds trainable low-rank matrices to attention layers, dramatically reducing the number of trainable parameters while preserving most of the model's expressive capacity. This is a practical choice that makes the approach reproducible with modest academic computing resources (4 A100s).

Why 5 SFT epochs: The paper does not discuss epoch tuning or early stopping. Five epochs is a common default for LoRA fine-tuning that balances sufficient training with overfitting prevention. The relatively small dataset (10–51K instances) and the relatively large model (8B parameters) mean that overfitting is a concern, but the LoRA parameter efficiency likely provides implicit regularization.

Why GRPO over PPO: The paper does not explicitly justify the choice of GRPO, but the primary advantage cited in the GRPO paper (Shao et al., 2024) is that it eliminates the need for a separate critic model. For an 8B-parameter model, training a separate critic of comparable size would roughly double the memory requirements. GRPO's group-relative baseline (using the average reward of the rollout group) achieves similar variance reduction without additional parameters.

KL penalty of 0.01. The KL divergence between the RL policy and the SFT policy is penalized with coefficient $0.01$ in the GRPO objective. This is a standard value that allows the policy to deviate enough to improve significantly while preventing the catastrophic forgetting and reward hacking that can occur with unconstrained RL. The paper does not ablate this value, so its sensitivity is unknown.

Inference details. For One-by-One, the maximum number of iterative steps is $K = 3$ . The paper does not explicitly describe how the All-in-One method handles the maximum number of strategies at inference time — it presumably relies on the model's learned distribution and sequence length to naturally produce 1–3 strategies. The inference prompts are provided in Appendix E (Figures 6, 7 for All-in-One; Figures 9, 10 for One-by-One), and include role descriptions, strategy definitions, dialogue context, and output format specifications.

4. Key Insights and Innovations

Innovation 1: The Multi-Strategy Formulation as an Empirical Hypothesis — Not Just a Modeling Choice

Most papers that relax a simplifying assumption frame the relaxation as inherently desirable — "real data has property X, therefore our model should accommodate X." This paper does something more subtle and intellectually honest: it treats multi-strategy utterance generation as an empirical hypothesis to be tested, not an obvious improvement to be assumed.

The dominant assumption in ESC research since Liu et al. (2021) has been that each supporter turn corresponds to exactly one strategy. This assumption is demonstrably false for the ESConv dataset — 17.7% of utterances contain two or more strategies (Table 1) — but being false doesn't automatically make it harmful. Many successful modeling paradigms use simplifying assumptions that are technically incorrect but practically neutral (e.g., bag-of-words models ignoring word order, or mean-field approximations in variational inference). The field could reasonably have argued: "Yes, humans sometimes use multiple strategies per turn, but modeling single strategies is a good enough approximation — the extra complexity of multi-strategy generation isn't worth the engineering cost."

The paper's key conceptual move is to refuse to accept this defense without evidence. Instead of stating "we propose multi-strategy generation because it's more realistic," the authors structure their investigation around the open question raised by Bai et al. (2025): "whether allowing multiple strategies in emotional support conversations is beneficial remains unclear." The paper then designs experiments that could, in principle, have produced a negative result — showing that multi-strategy generation introduces noise, degrades single-strategy performance, and fails to improve dialogue outcomes. The fact that the experiments instead show substantial gains (dialogue success rate rising from 13.85% to 40.00%) transforms the finding from a modeling preference into an empirical discovery: the one-strategy assumption is not merely inaccurate but actively harmful, and relaxing it yields measurable improvements on the actual task objective.

This matters beyond ESC because it establishes a template for how to approach simplifying assumptions in dialogue systems: measure the cost of the approximation at the task level, not just the representational level. The paper doesn't just show that multi-strategy utterances are more common in the data (that's a representational argument); it shows that dialogue agents constrained to one strategy per turn are less effective at actually helping seekers (that's a task argument). The distinction is crucial. Many NLP papers introduce structured outputs because the data has structure; fewer demonstrate that failing to model that structure degrades downstream task performance. This paper does.

The evidence is clearest in the dialogue-level evaluation (Table 6), where the single-strategy baseline achieves 13.85% success rate versus 17.69% for All-in-One without reasoning or RL — a modest but real gain from the formulation change alone. The gain compounds dramatically when reasoning and RL are added (34.62% for All-in-One, 40.00% for One-by-One), indicating that the multi-strategy formulation is a necessary but not sufficient condition for strong ESC performance. The formulation opens the door; reasoning and RL walk through it.

The paper also quantifies what the single-strategy assumption costs on multi-strategy utterances specifically (Table 3): on utterances with multiple strategies in the reference, the single-strategy baseline achieves 0.00 EMR — it literally cannot match the reference strategy sequence because it only predicts one strategy. The BLEU-4 score on these instances is 1.32 versus 2.00–2.93 for the multi-strategy methods with reasoning and RL. The single-strategy assumption doesn't just reduce accuracy on these turns; it makes them fundamentally impossible to model correctly.

This is a fundamental reframing, not an incremental improvement. Prior work treated the one-strategy-per-turn constraint as a practical convenience; this paper demonstrates it is a performance-limiting bottleneck. The distinction is between "we didn't bother modeling X" and "not modeling X is why the system fails on 17.7% of turns."

Innovation 2: Reasoning and RL as Necessary Enablers, Not Optional Enhancements

The paper's second conceptual contribution is a specific, empirically-grounded claim about when multi-strategy generation works and why. The claim, which runs counter to what one might naively expect, is that multi-strategy generation without explicit reasoning and reinforcement learning provides only marginal benefits — and that reasoning and RL are therefore not optional niceties but enabling conditions for the approach.

This matters because it resolves a tension in the results that could otherwise be misinterpreted. Looking at the base models without reasoning or RL: the Single-Strategy baseline achieves 25.21 EMR, while All-in-One achieves 23.61 and One-by-One achieves 24.99 (Table 2). A superficial reading might conclude that multi-strategy generation is slightly worse at strategy prediction than the simpler single-strategy approach, and that the reported gains are entirely attributable to reasoning and RL rather than the multi-strategy formulation. The paper anticipates and addresses this interpretation: it shows that the base multi-strategy models already outperform the single-strategy baseline on utterance quality metrics (BLEU, ROUGE, BERTScore) and dialogue success rate (17.69% vs. 13.85%), even with lower EMR, because EMR penalizes multi-strategy predictions when the reference has only one strategy — a measurement artifact, not a performance deficit.

The deeper insight is about task difficulty. Predicting a single strategy from eight options given dialogue context is a relatively straightforward classification-like problem. Predicting a sequence of 1–3 strategies in correct order, along with coherent responses for each, is a substantially harder structured prediction problem. The paper's results show that standard supervised fine-tuning on the ESConv data is insufficient to learn this harder task — the model defaults to single-strategy predictions (only 2.7% of All-in-One outputs contain multiple strategies without reasoning, versus 18.9% in the reference data; Table 4) and achieves only modest dialogue-level gains.

What reasoning and RL provide is the additional supervision signal needed to overcome this difficulty. The cognitive reasoning chain (Section 3.3) gives the model explicit intermediate targets — it must articulate why a particular strategy sequence is appropriate before generating it. The RL reward function (Equations 4–6) provides direct optimization pressure toward correct strategy sequences, including graded credit for partial matches via the Levenshtein Ratio. Together, these mechanisms supply the learning signal that the raw SFT data lacks for the harder multi-strategy task.

The evidence for this claim is the magnitude of the reasoning+RL effect, which is far larger than what would be expected if these were merely helpful refinements. From the base All-in-One to All-in-One + Reasoning + RL: EMR improves from 23.61 to 29.97 (a 27% relative gain), dialogue success rate improves from 17.69% to 34.62% (a 96% relative gain), and the proportion of multi-strategy utterances increases from 2.7% to 8.4% (a ~3× increase). For One-by-One: EMR goes from 24.99 to 33.53 (34% relative gain), success rate from 16.15% to 40.00% (148% relative gain), and multi-strategy proportion from 1.1% to 7.7% (7× increase). These are not incremental improvements from better hyperparameters — they are qualitative changes in model behavior driven by the combination of reasoning supervision and RL optimization.

This finding is a diagnostic contribution: it tells the field that modeling multiple strategies per turn is feasible but not trivial, and that simply changing the output format during SFT is insufficient. Future work on multi-strategy ESC (or similar structured-output dialogue tasks) should plan to invest in reasoning mechanisms and/or RL-based optimization as part of the core approach, not as optional add-ons. The paper also implicitly warns against evaluating structured-output methods using only surface-level metrics: the base One-by-One model looks competitive with the single-strategy baseline on EMR and generation metrics, but its dialogue-level performance reveals it is not yet using the multi-strategy capability effectively (16.15% success rate vs. 13.85%, a marginal gain).

Innovation 3: The One-by-One vs. All-in-One Comparison as an Inductive Bias Experiment

The paper's comparison of All-in-One (flat generation) and One-by-One (iterative generation) is more than an architectural ablation — it represents a controlled experiment on inductive bias for structured sequence generation. Both methods produce the same output (a variable-length sequence of strategy-response pairs) from the same input (dialogue context) using the same backbone model. The only difference is how the output structure is presented to the model during training and inference: as a flat concatenation or as an iteratively composed sequence with an explicit termination decision.

This comparison matters beyond ESC because it addresses a question that arises whenever NLP systems generate compositional structured outputs: should the model learn the structure implicitly through flat autoregressive generation, or should it be given an explicit decomposition that matches the compositional structure of the data? The flat approach (All-in-One) is simpler to implement — it requires no special training data construction, no special decoding logic, and no auxiliary prediction targets. The iterative approach (One-by-One) adds complexity but provides a stronger inductive bias: the model's architecture and training procedure mirror the compositional structure of the target output (each step produces one atomic unit, with a learned decision about when the composition is complete).

The paper's results provide evidence that the stronger inductive bias is beneficial for this task, but with important nuance. At the base SFT level (no reasoning, no RL), One-by-One and All-in-One perform similarly: EMR of 24.99 vs. 23.61, dialogue success rate of 16.15% vs. 17.69% (Tables 2, 6). The inductive bias alone doesn't help much. But when reasoning and RL are added, One-by-One pulls ahead substantially: EMR 33.53 vs. 29.97, success rate 40.00% vs. 34.62%. The iterative structure amplifies the benefits of reasoning and RL — the model is better able to leverage the explicit reasoning signal and the RL reward when the output is decomposed into individual strategy-response decisions.

Why the inductive bias helps: The One-by-One method forces the model to make three decisions sequentially for each strategy: (1) which strategy to use, (2) what response text to generate for that strategy, (3) whether another strategy is needed. At step i, decisions (1) and (2) are conditioned on the full dialogue context plus all previously generated strategy-response pairs for this turn. This means the model can adjust its strategy choice and response content based on what it has already said. In All-in-One, by contrast, all strategies and responses are generated in one forward pass — the later strategies depend on the earlier ones only through the model's internal hidden states, which may not provide as explicit or reliable a conditioning signal. The RL reward for One-by-One (Equation 6) also rewards correct termination decisions directly, giving the model a clear signal about when to stop composing strategies — a signal that has no analog in the All-in-One reward (Equation 5).

The termination flag is a particularly elegant mechanism. It converts the problem of "how many strategies should this utterance have?" from an implicit decision distributed across all tokens in the flat generation (where the model could "decide" to stop after one response simply by generating an end-of-sequence token) into an explicit binary prediction at each step. This makes the decision learnable through direct supervision (the RL flag reward $r_{\text{flag}}$ ) rather than emergent from the complex dynamics of autoregressive token prediction. The paper's data shows this pays off: One-by-One with reasoning + RL achieves both the highest EMR and the lowest average dialogue turns (8.46 vs. 9.56 for the single-strategy baseline; Table 6), indicating it is both more accurate in strategy selection and more efficient in knowing when to stop.

This finding is a methodological contribution with implications beyond ESC. For any task where the output is a variable-length sequence of typed segments (e.g., generating a sequence of dialogue acts, a recipe as a sequence of steps with types, a multi-paragraph argument with explicit rhetorical moves), the One-by-One decomposition provides a general template: train a model to generate one atomic unit plus a continuation flag, iterate until termination. The paper provides empirical evidence that this decomposition outperforms flat generation when the task requires coordinating multiple decisions (strategy selection, response generation, stopping behavior) that benefit from explicit step-by-step conditioning.

Innovation 4: Dialogue-Level Evaluation as the True Test of Multi-Strategy Utility

The paper's decision to evaluate at both the utterance level and the dialogue level is not merely thorough methodology — it represents a specific epistemological claim about what counts as evidence for multi-strategy generation. The utterance-level evaluation (Table 2) shows that the models can produce multi-strategy utterances that match reference outputs in surface form. But surface form matching is a weak signal for ESC: a model could achieve high BLEU scores by generating plausible-sounding supportive phrases without actually helping the seeker. The dialogue-level evaluation (Table 6) asks the deeper question: do multi-strategy utterances lead to more successful conversations?

The distinction is crucial because it's possible — and indeed common in dialogue research — for a method to improve automatic metrics while degrading actual task performance. The paper's own results contain hints of this tension: One-by-One with reasoning + RL achieves slightly lower BLEU-2 and BLEU-4 scores than One-by-One with only reasoning (Table 2, rows 9 vs. 10: BLEU-2 8.28 → 8.14, BLEU-4 3.88 → 3.71), yet achieves substantially higher dialogue success rate (26.15% → 40.00%). The RL optimization prioritizes strategy accuracy and termination correctness over n-gram overlap with references — and the dialogue-level results suggest this is the right tradeoff.

The dialogue-level evaluation design is also conceptually careful. The seeker is simulated by GPT-5 (a stronger model than the LLaMA-3.1-8B supporter), which is important because a weak seeker simulation would not provide a realistic test of supportive quality. The critic is a separate GPT-5 instance that evaluates emotional progress after each turn, categorizing the seeker's state into four levels and mapping them to scalar rewards. The success criterion — the final-turn score exceeding a predefined threshold — operationalizes "the seeker's emotional issue has been sufficiently addressed" as a measurable, reproducible quantity. This is substantially more meaningful than the common practice of evaluating dialogue agents using only turn-level reference-based metrics.

The paper's human evaluation (Table 7) provides a partial triangulation: both multi-strategy methods with reasoning + RL are consistently ranked better than the single-strategy baseline across all four dimensions (Identification, Comforting, Suggestion, Overall), with average rankings of 1.62–2.00 for the multi-strategy methods versus 2.18–2.40 for the baseline. The human evaluation samples are relatively small (50 dialogues, three annotators), and the paper acknowledges that the multi-strategy methods' advantage is "partly influenced by cognitive reasoning and reinforcement learning" — meaning the human evaluation doesn't isolate the multi-strategy formulation's contribution from the reasoning and RL gains. But it does establish that the full pipeline (multi-strategy + reasoning + RL) produces dialogues that humans prefer, which is the practical bottom line.

This contribution is a methodological standard-setting move. By demonstrating that utterance-level metrics and dialogue-level success can diverge (the RL model improving one while slightly degrading the other), the paper implicitly argues that dialogue-level evaluation with simulated interaction should be the primary evaluation paradigm for ESC systems — and that utterance-level metrics should be treated as diagnostic tools, not success criteria. This aligns with broader trends in dialogue evaluation (toward interactive, task-oriented assessment) but applies it specifically to the ESC domain with a concrete, reproducible protocol.

A nuance worth highlighting: the dialogue-level evaluation uses GPT-5 as both seeker and critic, which introduces a potential evaluator-model confound. If GPT-5 has specific preferences or biases about what constitutes good emotional support, those preferences will be reflected in both the seeker's responses and the critic's judgments. The paper's human evaluation partially mitigates this concern by showing human preferences align with the critic's rankings, but the alignment is imperfect (the human rankings are close between the two multi-strategy methods: 1.66 vs. 2.00 Overall, while the dialogue-level success rates differ more: 34.62% vs. 40.00%). This suggests the LLM critic and human evaluators may weight different aspects of supportive quality, and the dialogue-level numbers should be interpreted as relative rankings (method A > method B) rather than absolute quality estimates.

5. Experimental Analysis

Evaluation Methodology

Dataset. All experiments use the ESConv dataset (Liu et al., 2021), which contains 1,300 emotional support dialogues annotated with eight support strategy types: Question, Restatement or Paraphrasing, Reflection of feelings, Self-disclosure, Affirmation and Reassurance, Providing Suggestions, Information, and Others. The standard split is 1,040 dialogues for training, 130 for validation, and 130 for testing, yielding 10,679 training utterances, 2,257 validation utterances, and 2,389 test utterances (Table 1). Each dialogue is additionally annotated with a problem type, an emotion type, and a situation description, which are used for self-play dialogue evaluation.
Base model(s). All fine-tuned models use LLaMA-3.1-8B-Instruct as the backbone, an 8-billion-parameter instruction-tuned language model. The authors argue this model represents a reasonable balance between capability and accessibility — strong enough to learn the ESC task but small enough to fine-tune with LoRA on 4 A100 GPUs. For instruction-following LLM baselines, the paper evaluates GPT-5 (gpt-5-2025-08-07), DeepSeek-R1 (Guo et al., 2025), and Qwen3-235B-A22B-Instruct-2507 (Yang et al., 2025) — all substantially larger models used in zero-shot or few-shot settings without task-specific fine-tuning.
Metrics:
- Utterance-level automatic metrics (Table 2): The paper evaluates both strategy prediction and response generation quality. For strategy prediction: Exact Match Rate (EMR) requires the predicted strategy sequence to match the reference sequence exactly in both content and order, giving zero credit for partial matches; Levenshtein Ratio (LR) computes $1 - \text{LevenshteinDistance}(s_y, s_r) / \max(\text{len}(s_y), \text{len}(s_r))$ , providing graded credit for partial correctness; Average Length Difference (ALD) measures how utterance lengths compare to references, capturing whether models systematically under- or over-generate. For generation quality: BLEU-1/2/4 (Papineni et al., 2002), ROUGE-1/2/L (Lin, 2004), and BERTScore (Zhang et al., 2020) — all standard n-gram and embedding-based text generation metrics comparing predicted utterances against reference utterances. The paper reports BERTScore using the * variant (exact version not specified, but based on the scale of ~18–21, it likely uses the F1 variant with rescaling).
- Dialogue-level evaluation metrics (Table 6): Following Deng et al. (2024), the paper reports Average Turn (AT) — the mean number of supporter turns required across dialogues to reach the success criterion, with lower numbers indicating more efficient support — and Success Rate (SR) — the fraction of dialogues where the seeker's emotional state reaches a predefined improvement threshold within a maximum of 10 turns. A novel metric introduced by the paper is Average Strategy (AS) — the mean number of support strategies used per dialogue — which tracks whether multi-strategy methods are actually deploying more strategies or simply achieving better outcomes with similar strategy counts.
- Human evaluation (Table 7): On a 50-dialogue subset, three professional annotators independently rank three systems (Single-Strategy baseline, All-in-One + Rea. + RL, One-by-One + Rea. + RL) on a 3-point scale (1 best) across four dimensions: Identification (depth of problem exploration and perspective expansion), Comforting (empathy display and emotional relief), Suggestion (targetedness and feasibility of advice), and Overall (comprehensive supportive experience). Rankings are averaged across annotators and dialogues.
Baselines:
- Single-Strategy baseline: A fine-tuned LLaMA-3.1-8B-Instruct model trained to predict exactly one strategy and its corresponding response per turn, mirroring the conventional ESC formulation. This is the primary comparison point — it uses the same backbone model, same SFT training procedure, and same LoRA configuration, differing only in the output structure (single strategy-response pair vs. variable-length sequence). The prompt is provided in Appendix E.1, Figure 5.
- Instruction-following LLMs: GPT-5, DeepSeek-R1, and Qwen3-235B-Instruct, evaluated using the One-by-One method (prompt from Appendix E.3) without any task-specific fine-tuning. These serve as upper-bound references for what very large models can achieve with instruction following alone, without learning from ESConv training data. The paper also reports All-in-One performance for these models in Appendix C (Table 9).
- No reasoning and no RL ablations: Both All-in-One and One-by-One are evaluated in their base SFT form (without cognitive reasoning and without RL), serving as ablations that isolate the contribution of the multi-strategy formulation from the reasoning and RL enhancements.
Generation budget / compute accounting. The paper does not use a unified generation budget metric across methods, since all methods generate exactly one supporter turn per inference call. The relevant compute dimension is training data scale and RL sample efficiency rather than inference-time budget. Training is reported in terms of instances used (Table 8): SFT without reasoning uses 10,679 (All-in-One) or 12,759 (One-by-One) instances; SFT with reasoning uses 42,222 or 51,035 instances (approximately 4× the no-reasoning counts due to distillation from four teacher LLMs); RL uses 3,696 (All-in-One, after down-sampling single-strategy instances) or 12,759 (One-by-One) instances. All models are trained on 4 NVIDIA A100 80GB GPUs, with SFT for 5 epochs and RL for 7 epochs. The paper reports an approximate total API cost of 800 USD for data distillation and evaluation.
Cross-validation / statistical protocol. The paper does not report cross-validation for model selection — the test set is a fixed held-out split (130 dialogues, 2,389 utterances). This is standard practice for ESConv, where the train/validation/test splits are predefined by Liu et al. (2021) and widely used across the literature. The paper does not report confidence intervals, statistical significance tests, or multiple runs with different random seeds. Human evaluation uses three annotators and reports average ranks, but inter-annotator agreement metrics (e.g., Krippendorff's alpha) are not reported. The RL training curve (Figure 3) shows performance at individual training steps (0, 4, 8, 12, 16), providing some indication of training stability, but these appear to be single-run values without error bars.

Main Quantitative Results

Utterance-Level Evaluation: Strategy Prediction and Generation Quality

Table 2 presents the core utterance-level comparison across 10 model configurations on the 2,389-utterance ESC test set. The headline finding is that multi-strategy generation with reasoning and RL achieves substantially higher strategy prediction accuracy and generation quality than the single-strategy baseline, but only when both reasoning and RL are present — the base multi-strategy models without these enhancements show mixed results.

Fine-tuned models vs. instruction-following LLMs (rows 1–3 vs. 4–10): The instruction-following LLMs perform poorly across all metrics. GPT-5 achieves an EMR of only 0.67% — effectively failing to predict the correct strategy sequence on almost all test utterances. DeepSeek-R1 (12.85 EMR) and Qwen3-235B-Instruct (15.15 EMR) perform better but still far below any fine-tuned model (lowest fine-tuned EMR: 23.61). This confirms that emotional support conversation requires task-specific fine-tuning — large instruction-following models, even when prompted with the full ESConv strategy definitions and output format, cannot reliably produce correct multi-strategy utterances. The BLEU and ROUGE scores for instruction-following LLMs are correspondingly low (BLEU-4: 1.06–1.57, ROUGE-L: 13.51–15.29), indicating their generated responses are substantially different from reference utterances in both content and style.

Single-Strategy baseline vs. base multi-strategy models (rows 4, 5, 8): The Single-Strategy baseline achieves EMR of 25.21, which is higher than both base multi-strategy models (All-in-One: 23.61; One-by-One: 24.99). However, the generation quality metrics tell a different story: All-in-One surpasses the baseline on BLEU-1 (17.04 vs. 16.54), BLEU-2 (7.19 vs. 6.98), BLEU-4 (3.06 vs. 3.01), ROUGE-1 (23.33 vs. 22.95), ROUGE-2 (4.74 vs. 4.63), ROUGE-L (18.27 vs. 18.06), and BERTScore (18.17 vs. 18.16). One-by-One similarly exceeds the baseline on all generation metrics except BLEU-1 (16.97 vs. 16.54, a smaller margin). The key pattern: multi-strategy models generate higher-quality text even when their strategy prediction accuracy is slightly lower, suggesting that EMR — which requires exact match of the full strategy sequence — penalizes the multi-strategy models for predicting multiple strategies on turns that happen to have single-strategy references. The ALD values are comparable across methods (12.50–12.70), indicating that multi-strategy models are not simply generating longer utterances to achieve higher BLEU/ROUGE scores.

Effect of cognitive reasoning (rows 5 vs. 6, 8 vs. 9): Adding cognitive reasoning during training produces consistent and substantial improvements across all metrics. For All-in-One: EMR improves from 23.61 to 29.72 (+6.11 points, 26% relative gain), BLEU-4 from 3.06 to 3.98 (+30%), ROUGE-L from 18.27 to 20.10 (+10%), BERTScore from 18.17 to 19.98 (+10%). For One-by-One: EMR improves from 24.99 to 29.55 (+4.56 points, 18% relative), BLEU-4 from 3.30 to 3.88 (+18%), ROUGE-L from 19.26 to 20.23 (+5%), BERTScore from 18.95 to 20.72 (+9%). The magnitude and consistency of these gains across both methods and all metrics is strong evidence that the four-node cognitive reasoning chain provides genuinely useful supervisory signal — it is not merely an architectural decoration but a meaningful augmentation of the training objective.

Effect of reinforcement learning (rows 6 vs. 7, 9 vs. 10): RL provides additional but more nuanced gains. For All-in-One + Reasoning + RL vs. All-in-One + Reasoning: EMR improves from 29.72 to 29.97 (+0.25 points), BLEU-4 from 3.98 to 4.07 (+2%), ROUGE-L from 20.10 to 20.71 (+3%), BERTScore from 19.98 to 20.68 (+4%). The gains are smaller than those from adding reasoning to the base model, suggesting that SFT with reasoning already captures most of the achievable improvement and RL provides fine-tuning rather than a qualitative shift. For One-by-One + Reasoning + RL vs. One-by-One + Reasoning: EMR improves from 29.55 to 33.53 (+3.98 points, 13% relative) — a substantially larger gain than for All-in-One — but BLEU scores slightly decrease (BLEU-2: 8.28 → 8.14; BLEU-4: 3.88 → 3.71), while ROUGE-L (20.23 → 20.38) and BERTScore (20.72 → 21.11) modestly improve. This pattern — RL improves strategy prediction at the cost of slight surface-form degradation — is consistent with the RL reward function prioritizing strategy accuracy (Levenshtein Ratio) over exact lexical matching with references.

Best overall configuration: The One-by-One method with reasoning and RL (row 10) achieves the highest EMR (33.53), highest BERTScore (21.11), and lowest ALD (12.13), while All-in-One with reasoning and RL (row 7) achieves the highest BLEU-1 (18.75), BLEU-2 (8.62), BLEU-4 (4.07), and ROUGE scores (R-1: 25.68, R-2: 6.26, R-L: 20.71). Neither method strictly dominates across all metrics, indicating a tradeoff: One-by-One is better at strategy prediction and semantic similarity (BERTScore), while All-in-One is better at lexical overlap with references (BLEU, ROUGE). This tradeoff is expected given the methods' different architectures — One-by-One's iterative decomposition may encourage more deliberate strategy selection (hence higher EMR) but at the cost of less fluent global coherence (lower BLEU), while All-in-One's flat generation may produce more naturally flowing text at the expense of precise strategy sequencing.

Performance Breakdown by Utterance Type: Single-Strategy vs. Multi-Strategy Utterances

Table 3 separates test utterances into those with a single strategy in the reference (1,937 utterances, 81.1% of the test set) and those with multiple strategies (452 utterances, 18.9%). This breakdown reveals where the multi-strategy methods gain and lose relative to the baseline, and the results are asymmetric.

On single-strategy utterances: The Single-Strategy baseline achieves EMR of 31.09 and strong generation scores (B-4: 3.71, R-L: 19.69, BERTScore: 19.74). Both multi-strategy base models underperform the baseline across all metrics: All-in-One achieves EMR 28.76 (vs. 31.09), B-4 3.34 (vs. 3.71), R-L 18.72 (vs. 19.69), BERTScore 18.87 (vs. 19.74); One-by-One achieves EMR 30.72 (vs. 31.09), B-4 3.61 (vs. 3.71), R-L 19.70 (vs. 19.69), BERTScore 19.57 (vs. 19.74). The degradation is modest — 1–2 EMR points, 0.1–0.4 BLEU-4 points — but consistent, confirming the authors' hypothesis that "modeling multiple strategies can introduce noise for these simpler cases." However, with reasoning and RL, both multi-strategy methods surpass the single-strategy baseline even on single-strategy utterances: All-in-One + Rea. + RL achieves EMR 34.42, B-4 4.47, R-L 21.38, BERTScore 21.48; One-by-One + Rea. + RL achieves EMR 38.24, B-4 4.05, R-L 21.03, BERTScore 21.86. This is a non-trivial finding: reasoning and RL not only enable multi-strategy generation but also improve single-strategy performance, likely because the cognitive reasoning chain provides better strategy selection even when the output contains only one strategy.

On multi-strategy utterances: The Single-Strategy baseline fails catastrophically — EMR of 0.00, since it can never match a reference containing two or more strategies by predicting only one. Its generation scores are also poor: B-4 of 1.32 (vs. 2.93 for All-in-One + Rea. + RL), R-L of 14.19 (vs. 18.17 for One-by-One + Rea. + RL), BERTScore of 13.90 (vs. 17.92 for One-by-One + Rea. + RL). The base multi-strategy models provide some improvement — All-in-One achieves EMR 1.55, B-4 1.86, R-L 16.34, BERTScore 15.15; One-by-One achieves EMR 0.44, B-4 2.00, R-L 17.41, BERTScore 16.31 — but the EMR values are still near zero, indicating that even when multi-strategy generation is allowed, the base models rarely predict the correct multi-strategy sequence. This is where reasoning and RL have their largest impact: All-in-One + Rea. + RL achieves EMR 10.88 (a ~7× increase over the base All-in-One's 1.55), and One-by-One + Rea. + RL achieves EMR 13.36 (a ~30× increase over the base One-by-One's 0.44). The generation metrics improve correspondingly: B-4 reaches 2.93 for All-in-One and 2.75 for One-by-One (vs. 1.32 for the baseline), and BERTScore reaches 17.86–17.92 (vs. 13.90 for the baseline).

The asymmetry is instructive: On single-strategy utterances, the baseline holds a small advantage that reasoning and RL can overcome. On multi-strategy utterances, the baseline is fundamentally incapable and the gap is enormous. Since multi-strategy utterances represent 18.9% of the test set, the overall performance of multi-strategy methods reflects a weighted average: modest losses on 81.1% of turns, enormous gains on 18.9% of turns, net positive. This explains why the aggregate EMR for base All-in-One (23.61) is slightly below the baseline (25.21) — the small degradation on single-strategy utterances outweighs the near-zero EMR improvement on multi-strategy utterances at the base level. With reasoning and RL, the model improves sufficiently on both utterance types (and especially on multi-strategy utterances) to surpass the baseline in aggregate.

Multi-Strategy Utterance Generation Frequency

Table 4 quantifies how often each method actually produces multi-strategy utterances at inference time. The reference data contains 18.9% multi-strategy utterances (two or more strategies per turn). The models' behavior does not match this distribution:

The Single-Strategy baseline, by construction, produces 0.0% multi-strategy utterances.
Base All-in-One produces only 2.7% multi-strategy utterances. Base One-by-One produces 1.1%. Both are far below the 18.9% reference rate, indicating that the models are highly conservative — they default to single-strategy generation even when trained to allow multiple strategies.
Adding reasoning increases multi-strategy generation to 6.3% for All-in-One and 3.6% for One-by-One — still below the reference rate but substantially higher than the base models.
Adding RL further increases these rates to 8.4% (All-in-One) and 7.7% (One-by-One).

A critical observation: the best models still produce multi-strategy utterances less than half as often as the reference data (8.4% vs. 18.9% for All-in-One; 7.7% vs. 18.9% for One-by-One). This means that even with reasoning and RL, the models are under-utilizing the multi-strategy capability. The paper does not investigate why — possible explanations include: (1) the training data imbalance (82.3% single-strategy) biases the model toward conservative predictions; (2) the RL reward function does not explicitly incentivize multi-strategy generation (the Levenshtein Ratio provides partial credit for single-strategy predictions on multi-strategy references, so the model can achieve non-zero reward without predicting multiple strategies); (3) the cognitive reasoning chains, being distilled from LLMs that were shown the gold multi-strategy references, may not provide sufficiently strong signals to overcome the prior toward single-strategy outputs.

Teacher Model Distillation Analysis

Table 5 provides an ablation of which teacher models contribute to the reasoning distillation, using the All-in-One method. The key findings:

All single-teacher settings outperform no reasoning: EMR ranges from 26.66 (GPT-5) to 28.46 (DeepSeek-R1), versus 23.61 for no reasoning. This confirms that cognitive reasoning from any strong LLM is beneficial, regardless of the specific teacher.
No single teacher dominates: GPT-5 yields the highest BLEU-4 (3.72), Gemini yields the highest ROUGE-L (19.97), and DeepSeek-R1 yields the highest EMR (28.46) and BERTScore (19.32). The pattern suggests that different teachers emphasize different aspects of supportive generation — some are better at producing responses that lexically match references, others are better at teaching strategy selection — and these strengths are complementary.
Combining all four teachers yields the best overall performance: EMR 29.72 (+1.26 over the best single teacher, DeepSeek-R1 at 28.46), BLEU-4 3.98 (+0.26 over the best single teacher, GPT-5 at 3.72), ROUGE-L 20.10 (+0.13 over the best single teacher, Gemini at 19.97), BERTScore 19.98 (+0.66 over the best single teacher, DeepSeek-R1 at 19.32). The improvements from combining teachers are modest in absolute terms but consistent across all metrics. This validates the paper's claim that "aggregating heterogeneous reasoning signals" improves robustness, though the effect size suggests the primary benefit of using multiple teachers is coverage (ensuring at least one teacher provides a good reasoning chain for each instance) rather than synergy (the combination being better than the sum of its parts).

A limitation of this analysis is that it reports only All-in-One results. It would be informative to see whether the same teacher ranking holds for One-by-One, particularly since One-by-One achieves higher overall performance and might benefit differently from different reasoning styles. The paper does not provide this comparison.

Dialogue-Level Self-Play Evaluation

Table 6 reports dialogue-level performance where GPT-5 simulates the seeker and a separate GPT-5 instance serves as the critic. The maximum dialogue length is 10 turns; dialogues that do not reach the success threshold within 10 turns are counted as failures.

Instruction-following LLMs (rows 1–3): GPT-5, DeepSeek-R1, and Qwen3-235B-Instruct all fail to resolve any dialogue within 10 turns, achieving 0.00% success rate across all 130 test dialogues. Their average turns (AT) are at the maximum of 10.00, and average strategies per dialogue (AS) range from 10.20 to 10.96 — they simply generate strategies for 10 turns without achieving the success criterion. This is a striking result: even a model as capable as GPT-5, when used without task-specific fine-tuning, cannot successfully navigate a 10-turn emotional support conversation under the evaluation protocol. This reinforces the utterance-level finding that ESC requires learning from task-specific data, not just general instruction-following ability.

Single-Strategy baseline (row 4): Achieves SR of 13.85%, AT of 9.56, and AS of 9.56. The success rate is non-trivial but low — the baseline resolves roughly one in seven dialogues. The near-equality of AT and AS (both 9.56) is expected since the baseline always uses exactly one strategy per turn.

Base multi-strategy models (rows 5, 8): All-in-One achieves SR 17.69% (+3.84 points over baseline), AT 9.50, AS 9.63. One-by-One achieves SR 16.15% (+2.30 points), AT 9.72, AS 9.74. Both methods improve success rate while using comparable numbers of turns and total strategies — the improvement comes from more effective strategy deployment within each turn, not from using more strategies overall. The AS values are only marginally higher than the baseline (9.63 and 9.74 vs. 9.56), consistent with the utterance-level finding that base multi-strategy models rarely generate multiple strategies per turn (Table 4: only 1.1–2.7% of utterances). The modest SR gains (3–4 percentage points) suggest that even occasional multi-strategy generation provides dialogue-level benefits.

Effect of reasoning (rows 5 vs. 6, 8 vs. 9): Adding cognitive reasoning dramatically improves dialogue outcomes. All-in-One + Reasoning achieves SR 32.31% (+14.62 points over base All-in-One), AT 8.80, AS 9.44. One-by-One + Reasoning achieves SR 26.15% (+10.00 points over base One-by-One), AT 9.08, AS 9.32. Both methods achieve these large SR gains while actually reducing average turns (8.80 vs. 9.50 for All-in-One; 9.08 vs. 9.72 for One-by-One) and with stable or slightly lower strategy usage (AS: 9.44 vs. 9.63; 9.32 vs. 9.74). This is the efficiency argument in action: reasoning enables the model to achieve more emotional progress per turn, resolving dialogues faster without using more strategies. The fact that AS is stable despite higher multi-strategy generation rates (Table 4: 6.3% for All-in-One with reasoning) means that when the model does use multiple strategies, it's using them on turns where they genuinely help, allowing the dialogue to conclude in fewer total turns.

Effect of RL (rows 6 vs. 7, 9 vs. 10): RL provides further gains, but asymmetrically. All-in-One + Reasoning + RL achieves SR 34.62% (+2.31 over reasoning-only), AT 8.80 (unchanged), AS 9.90 (+0.46). One-by-One + Reasoning + RL achieves SR 40.00% (+13.85 over reasoning-only), AT 8.46 (−0.62), AS 10.29 (+0.97). For One-by-One, the RL gains are large and accompanied by both fewer turns and more strategies — the model is using more multi-strategy utterances (Table 4: 7.7%) to resolve dialogues more efficiently. For All-in-One, the RL gains are modest and come with unchanged AT and slightly more strategies. The asymmetry suggests that One-by-One's iterative architecture is better able to leverage RL optimization — the explicit termination flag reward $r_{\text{flag}}$ in Equation 6 provides a direct learning signal for efficient strategy deployment that has no analog in the All-in-One reward (Equation 5).

Best dialogue-level configuration: One-by-One with reasoning and RL (row 10) achieves the highest SR (40.00%), lowest AT (8.46), and highest AS (10.29). This combination — highest success rate, fewest turns, most strategies — suggests the model is deploying strategies more densely (multiple strategies per turn on key turns) and achieving faster emotional resolution. Compared to the Single-Strategy baseline: SR improves from 13.85% to 40.00% (a 2.9× relative increase), AT decreases from 9.56 to 8.46 (1.1 fewer turns on average), and AS increases from 9.56 to 10.29 (0.73 more strategies per dialogue, despite fewer turns). The efficiency gain is clear: the multi-strategy model achieves substantially better outcomes with fewer turns, though it uses slightly more total strategies (concentrated in those fewer turns).

Human Evaluation

Table 7 reports human evaluation on 50 sampled dialogues, comparing the Single-Strategy baseline against the two best multi-strategy configurations (All-in-One + Rea. + RL and One-by-One + Rea. + RL). Three professional annotators independently rank the three systems on a 1–3 scale (1 = best) across four dimensions. Average ranks are reported.

Identification: All-in-One ranks 1.90, One-by-One ranks 1.92, Single-Strategy baseline ranks 2.18. Both multi-strategy methods are preferred, with All-in-One having a slight edge.
Comforting: All-in-One ranks 1.84, One-by-One ranks 1.92, baseline ranks 2.24. Again, both multi-strategy methods preferred.
Suggestion: All-in-One ranks 1.62, One-by-One ranks 1.98, baseline ranks 2.40. All-in-One shows the strongest advantage here.
Overall: All-in-One ranks 1.66, One-by-One ranks 2.00, baseline ranks 2.34. All-in-One achieves the best average rank.

Several patterns are notable. First, both multi-strategy methods are consistently preferred over the baseline across all dimensions, with average ranks of 1.62–2.00 versus 2.18–2.40. The gap is clearest on Suggestion (1.62/1.98 vs. 2.40), suggesting that multi-strategy utterances are particularly effective for delivering actionable advice — possibly because they can combine suggestion with emotional validation or personal disclosure within a single turn, making the advice feel more integrated and less directive.

Second, All-in-One + Rea. + RL is consistently ranked slightly better than One-by-One + Rea. + RL across all dimensions (1.62–1.90 vs. 1.92–2.00). This is the opposite of the dialogue-level automatic evaluation, where One-by-One + Rea. + RL achieved the highest success rate (40.00% vs. 34.62%). The discrepancy could reflect differences in what human annotators value versus what the GPT-5 critic evaluates — humans may prefer the more fluent, globally coherent utterances produced by All-in-One (which achieved higher BLEU and ROUGE scores in Table 2), while the critic may reward the more precise strategy sequencing of One-by-One (higher EMR in Table 2). The paper does not investigate this divergence further, but it highlights the importance of multi-faceted evaluation — neither automatic dialogue-level metrics nor human judgments alone provide a complete picture of system quality.

Third, the human evaluation ranks are fairly close between the two multi-strategy methods (differing by 0.02–0.36 points on a 1–3 scale), suggesting that the practical difference in perceived quality between All-in-One and One-by-One may be smaller than the dialogue-level success rate difference (34.62% vs. 40.00%) would imply. The sample size (50 dialogues) and the coarse ranking methodology (1–3 scale rather than direct comparison or Likert ratings) limit the precision of these estimates.

Reinforcement Learning Training Dynamics

Figure 3 shows the learning curve for All-in-One under RL training, tracking two quantities across 16 training steps: dialogue-level Success Rate (SR) and the percentage of generated utterances containing 2 or more strategies. The dynamics reveal a non-monotonic relationship between multi-strategy usage and success rate:

Steps 0–4 (initial phase): The percentage of multi-strategy utterances decreases from approximately 7.05% at step 0 to about 5% at step 4. During the same interval, SR decreases from 32.31% to roughly 28%. This is the "exploration dip" — the model initially reduces its use of multiple strategies (perhaps because the RL policy is uncertain and defaults to the safer single-strategy prediction) and dialogue performance suffers.
Steps 4–16 (improvement phase): The percentage of multi-strategy utterances steadily increases from ~5% at step 4 to approximately 12.50% at step 16. SR also increases from ~28% to 34.62%. The model learns that using more strategies (when appropriate) leads to higher reward, and dialogue outcomes improve in tandem.
Final state (step 16): Multi-strategy usage (12.50%) is substantially higher than at step 0 (7.05%) but still below the reference rate (18.9%). SR (34.62%) is higher than at step 0 (32.31%), confirming that RL training produces a net improvement, though the gain is modest (+2.31 percentage points in SR).

The paper presents this curve as evidence that "reinforcement learning eventually encourages more effective multi-strategy usage." This interpretation is reasonable but requires caution: the correlation between multi-strategy usage and SR is suggestive but not causal — other aspects of policy improvement during RL (better response quality, more appropriate single-strategy selections) could also drive the SR improvement. The curve does demonstrate that RL does not cause the model to abandon multi-strategy generation (the initial dip is temporary) and that the final policy uses more multi-strategy utterances than the SFT starting point.

The paper only shows this curve for All-in-One. A comparable curve for One-by-One would be informative, particularly since One-by-One's RL gains are much larger (SR from 26.15% to 40.00% vs. 32.31% to 34.62% for All-in-One). The One-by-One curve might show whether the termination flag reward $r_{\text{flag}}$ drives different learning dynamics — for example, faster improvement in multi-strategy usage or earlier stabilization of the stopping behavior.

Ablation Studies and Robustness Checks

Format of multi-strategy output (All-in-One vs. One-by-One): This is the central architectural comparison, evaluated at all stages of enhancement. At the base SFT level (Table 2, rows 5 vs. 8): One-by-One achieves slightly higher EMR (24.99 vs. 23.61) and comparable generation metrics. With reasoning added (rows 6 vs. 9): the methods are essentially tied on most metrics (EMR: 29.72 vs. 29.55; B-4: 3.98 vs. 3.88; R-L: 20.10 vs. 20.23; BERTScore: 19.98 vs. 20.72). With reasoning and RL (rows 7 vs. 10): One-by-One pulls ahead on EMR (33.53 vs. 29.97) and BERTScore (21.11 vs. 20.68), while All-in-One leads on BLEU and ROUGE. In dialogue-level evaluation (Table 6), One-by-One with reasoning + RL achieves substantially higher SR (40.00% vs. 34.62%) and lower AT (8.46 vs. 8.80). The consistent pattern is that the methods are comparable without RL but diverge with RL, with One-by-One achieving better dialogue outcomes. This interaction suggests that the iterative decomposition is particularly valuable when combined with optimization pressure — the explicit stopping decision and the per-step conditioning enable more effective policy improvement during RL.

Cognitive reasoning (with vs. without): Evaluated for both methods at the SFT level. For All-in-One (Table 2, rows 5 vs. 6): reasoning improves EMR by 6.11 points (23.61 → 29.72), B-4 by 0.92 (3.06 → 3.98), BERTScore by 1.81 (18.17 → 19.98). For One-by-One (rows 8 vs. 9): EMR improves by 4.56 points (24.99 → 29.55), B-4 by 0.58 (3.30 → 3.88), BERTScore by 1.77 (18.95 → 20.72). The gains are consistent, substantial, and similar in magnitude across both methods (~20–30% relative improvement on most metrics). This ablation establishes that cognitive reasoning is the single largest contributor to performance improvement among all enhancements studied — larger than the effect of switching from All-in-One to One-by-One, and larger than the marginal effect of RL on top of reasoning. The paper does not ablate individual reasoning nodes (Context, Cognition, Emotion, Support Plan) to determine which components of the reasoning chain are most important — this would be a valuable follow-up experiment.

Reinforcement learning (with vs. without, given reasoning): For All-in-One (Table 2, rows 6 vs. 7): RL improves EMR by 0.25 (29.72 → 29.97), B-4 by 0.09 (3.98 → 4.07), BERTScore by 0.70 (19.98 → 20.68). For One-by-One (rows 9 vs. 10): RL improves EMR by 3.98 (29.55 → 33.53), BERTScore by 0.39 (20.72 → 21.11), but B-4 decreases by 0.17 (3.88 → 3.71). The RL effect is method-dependent: large for One-by-One's strategy prediction, small-to-mixed for All-in-One. In dialogue-level evaluation (Table 6), RL improves One-by-One's SR from 26.15% to 40.00% (+13.85) but All-in-One's SR from 32.31% to 34.62% (+2.31). This interaction suggests that RL is particularly synergistic with the One-by-One architecture — the iterative decomposition provides a more tractable optimization landscape for the GRPO algorithm, possibly because the reward function more directly supervises the model's per-step decisions (strategy choice, response generation, continuation) rather than expecting the model to internalize all these decisions in a single generation.

Teacher model for reasoning distillation (single vs. ensemble): Evaluated for All-in-One (Table 5). Individual teachers yield EMR of 26.66–28.46 (vs. 23.61 for no reasoning). The combined ensemble yields EMR 29.72. The gain from ensemble over the best single teacher (DeepSeek-R1: 28.46 → 29.72) is +1.26 EMR points — modest but positive. The ranking of teachers varies by metric: GPT-5 leads on BLEU-4 (3.72), Gemini on ROUGE-L (19.97), DeepSeek-R1 on EMR (28.46) and BERTScore (19.32). This heterogeneity supports the paper's rationale for using multiple teachers — no single teacher is universally best, and the ensemble captures complementary strengths. The ablation does not report dialogue-level metrics for each teacher, which would reveal whether EMR improvements from better teachers translate to better dialogue outcomes.

Multi-strategy utterance proportion (Table 4): While not a controlled ablation, this analysis shows that the proportion of multi-strategy utterances produced by each model configuration varies substantially: 0.0% (Single-Strategy baseline), 1.1–2.7% (base multi-strategy models), 3.6–6.3% (with reasoning), 7.7–8.4% (with reasoning + RL). All models significantly under-produce multi-strategy utterances relative to the reference distribution (18.9%). This analysis serves as a diagnostic rather than an ablation per se — it identifies a remaining gap (the models are still conservative about using multiple strategies) and shows that each enhancement (reasoning, RL) partially closes the gap. The paper does not experiment with explicit mechanisms to encourage higher multi-strategy generation rates, such as adjusting the RL reward to penalize under-use of multiple strategies or balancing the training data to oversample multi-strategy instances. These would be natural follow-up experiments.

Down-sampling single-strategy instances for RL (Equation 5 description): The paper states that for All-in-One RL training, "we down-sample single-strategy instances to balance the reward distribution" but does not report results without this down-sampling. This is an implicit ablation that is not empirically validated — we do not know whether the down-sampling is necessary or beneficial, or by how much. Given that the All-in-One RL gains are modest (+0.25 EMR), one might ask whether different down-sampling ratios would yield larger improvements, or whether the down-sampling introduces its own biases.

Maximum iterative steps K for One-by-One: The paper sets K = 3 based on the observation that "nearly all utterances in the validation set contain at most three strategies." No ablation is reported for K = 2 or K = 4. Given that the best One-by-One model produces multi-strategy utterances 7.7% of the time (Table 4), the K = 3 limit is unlikely to be binding for the current model, but it could become relevant if future improvements increase the multi-strategy generation rate.

LoRA rank (8 vs. 16): The paper uses different LoRA configurations for All-in-One (rank 8, alpha 16) and One-by-One (rank 16, alpha 32). This is a confound in the direct comparison of the two methods — One-by-One has twice the adapter capacity, which could contribute to its superior performance. The paper does not ablate LoRA rank or justify the choice with evidence that One-by-One benefits from more capacity. As a result, we cannot determine whether One-by-One's advantages come from the architectural decomposition or simply from having more trainable parameters. The paper would be strengthened by a controlled comparison where both methods use the same LoRA configuration, or by showing that All-in-One does not benefit from increased LoRA rank.

Seeker model for dialogue evaluation: The dialogue-level evaluation uses GPT-5 as both seeker and critic. The paper does not ablate this choice — we do not know how results would change with a different seeker model (e.g., DeepSeek-R1, a fine-tuned LLaMA, or human seekers) or a different critic. The instruction-following LLM results (Table 6, rows 1–3: all 0.00% SR) suggest that the dialogue evaluation is difficult enough to discriminate between models, but the specific success rates (13.85%, 34.62%, 40.00%) may depend on the GPT-5 seeker's behavior. A robustness check with an alternative seeker model would strengthen confidence in the SR rankings.

Critical Assessment

Does the evidence support the claim that multi-strategy generation is beneficial for ESC?

The paper's central claim is that "modeling multiple strategies within an utterance is feasible and beneficial, consistently outperforming the Single-Strategy baseline" (Section 6, conclusion). The evidence supports this claim with the important qualification that reasoning and RL are necessary to realize the benefit.

The base multi-strategy models without reasoning or RL provide only weak evidence: dialogue-level SR improves from 13.85% (Single-Strategy) to 17.69% (All-in-One) and 16.15% (One-by-One) — gains of 2–4 percentage points. These are positive but modest, and the utterance-level EMR is actually lower (23.61 and 24.99 vs. 25.21). A skeptical reader could interpret the base results as showing that the multi-strategy formulation introduces complexity without clear benefit.

The strong evidence comes from the reasoning + RL configurations: All-in-One + Rea. + RL achieves 34.62% SR (2.5× the baseline); One-by-One + Rea. + RL achieves 40.00% SR (2.9× the baseline). These are large and practically meaningful gains. The human evaluation (Table 7) corroborates that humans prefer the multi-strategy systems. The utterance-level breakdown (Table 3) confirms that the gains are concentrated on — but not limited to — multi-strategy utterances, where the baseline achieves 0.00 EMR.

The conditional nature of the finding is critical: multi-strategy generation without reasoning and RL is marginal; with reasoning and RL it is clearly beneficial. The paper does not fully disentangle how much of the reasoning + RL gain comes from better single-strategy selection (which would benefit any model, multi-strategy or not) versus better multi-strategy coordination specifically. The fact that reasoning + RL improves single-strategy utterance performance (Table 3, top section: EMR 34.42–38.24 for multi-strategy methods vs. 31.09 for the baseline) suggests that a portion of the dialogue-level gain is attributable to generally better strategy selection and response generation, not to multi-strategy coordination per se.

What was not tested: The paper does not include a baseline that uses the same reasoning and RL enhancements but with the single-strategy output format. This would be the cleanest test of whether the multi-strategy formulation adds value beyond what reasoning and RL provide. If a Single-Strategy + Reasoning + RL model achieved, say, 35% SR, then the multi-strategy formulation would be adding only ~5 percentage points to SR rather than the apparent 27 percentage point gap (13.85% vs. 40.00%). The absence of this baseline means we cannot precisely quantify the unique contribution of the multi-strategy formulation when all other enhancements are held constant.

Does the evidence support the claim that the One-by-One method outperforms All-in-One?

The evidence is mixed and depends on the evaluation setting. In utterance-level evaluation (Table 2), One-by-One + Rea. + RL leads on EMR (33.53 vs. 29.97) and BERTScore (21.11 vs. 20.68) while All-in-One + Rea. + RL leads on all BLEU and ROUGE scores. In dialogue-level evaluation (Table 6), One-by-One + Rea. + RL achieves substantially higher SR (40.00% vs. 34.62%). But in human evaluation (Table 7), All-in-One + Rea. + RL is consistently ranked slightly better than One-by-One + Rea. + RL across all four dimensions.

This pattern suggests that One-by-One optimizes for what the automatic critic rewards (strategy accuracy and efficient termination, reflected in higher EMR and higher dialogue SR), while All-in-One optimizes for what humans perceive as fluent and coherent support (reflected in higher BLEU/ROUGE and better human rankings). The paper does not address this tension directly, but it implies that the "best" method depends on the deployment context: if the goal is to maximize the probability of resolving the seeker's issue (as measured by the GPT-5 critic), One-by-One is better; if the goal is to produce support that humans find most natural and helpful, All-in-One may be preferable.

The LoRA configuration confound (rank 8 for All-in-One, rank 16 for One-by-One) weakens the evidence for One-by-One's superiority. One-by-One has double the adapter capacity, and it's not possible to determine whether its dialogue-level advantage comes from the architectural decomposition or simply from having more trainable parameters. A fair comparison would use the same LoRA rank for both methods.

Does the evidence support the claim that cognitive reasoning and RL are necessary enablers?

Strongly supported for reasoning; moderately supported for RL. The ablation of reasoning (Table 2, rows 5 vs. 6 and 8 vs. 9) shows consistent, substantial gains across all metrics for both methods. The ablation of the reasoning source (Table 5) shows that reasoning from any teacher helps and that combining teachers helps more. The evidence for reasoning as an enabler is robust.

The evidence for RL is more nuanced. For One-by-One, RL provides large gains in dialogue-level SR (26.15% → 40.00%) and utterance-level EMR (29.55 → 33.53), with only minor degradation in BLEU scores. For All-in-One, the RL gains are small in both utterance-level EMR (29.72 → 29.97) and dialogue-level SR (32.31% → 34.62%). This interaction — RL helps One-by-One substantially more than All-in-One — is interesting but the paper does not investigate it deeply. One hypothesis: the One-by-One reward function (Equation 6) includes the flag reward $r_{\text{flag}}$ which provides a direct learning signal for efficient termination; this signal has no analog in the All-in-One reward (Equation 5), so One-by-One gets more optimization leverage from the same RL algorithm.

What was not tested: The paper does not report performance of models trained with RL but without reasoning. This would reveal whether RL alone (without the reasoning supervision) can achieve benefits comparable to reasoning, or whether reasoning is a prerequisite for RL to be effective. The paper also does not ablate the specific RL reward components — would a reward based only on the Levenshtein Ratio (without the flag reward) perform similarly for One-by-One? Would a simpler exact-match reward work as well as the graded Levenshtein Ratio? These ablations would clarify which aspects of the RL setup are driving the observed gains.

Does the evidence support the claim that the approach generalizes beyond the specific experimental setup?

Not tested. All experiments use ESConv, a single dataset with 130 test dialogues. The paper does not evaluate on any other emotional support dataset, general empathetic dialogue corpus, or out-of-domain transfer scenario. The base model is a single architecture (LLaMA-3.1-8B-Instruct); the paper does not test whether the findings hold for other model families (e.g., Qwen, Mistral) or model scales. The teacher LLMs for reasoning distillation are all very large proprietary or open-weight models; the paper does not test whether smaller or less capable teachers would also be effective, which matters for reproducibility.

The human evaluation (Table 7) uses the same ESConv test set situations and the same GPT-5-based seeker simulation — it does not involve real help-seekers in genuine distress, nor does it test the system with users whose emotional needs differ from the ESConv distribution. The paper is appropriately cautious about this in the limitations section, noting that "our evaluation relies on simulated seekers for dialogue-level assessment; while this enables large-scale testing, it may not fully capture the nuances of real human emotional interactions." This limitation is substantive: the dialogue-level success rate (40.00%) measures performance against a specific GPT-5 instantiation of seeker behavior and a specific GPT-5 critic, and we do not know how these numbers would correlate with outcomes in real supportive conversations.

Specific methodological weaknesses

No statistical significance reporting. The paper does not report confidence intervals, standard deviations, p-values, or any other measure of statistical reliability. The test set contains 2,389 utterances but only 130 dialogues; the dialogue-level success rates (13.85%, 34.62%, 40.00%) are based on 130 binary outcomes and could have wide confidence intervals. Without statistical testing, we cannot determine whether differences of a few percentage points (e.g., All-in-One vs. Single-Strategy at the base level: 17.69% vs. 13.85%) are statistically significant or could arise from random variation.

Small human evaluation sample. The human evaluation uses 50 dialogues and 3 annotators, ranking 3 systems on a 3-point scale. This yields 150 rankings total (50 × 3). The paper reports average ranks but no inter-annotator agreement metrics, making it impossible to assess whether the annotators' judgments are consistent. The close rankings between the two multi-strategy methods (1.66 vs. 2.00 Overall) could easily reflect annotator noise rather than a genuine quality difference.

Confounded comparison between All-in-One and One-by-One. Beyond the LoRA rank difference mentioned above, the two methods also differ in their RL data quantities (Table 8: 3,696 All-in-One RL instances vs. 12,759 One-by-One RL instances) due to the per-step decomposition of One-by-One. The paper does not control for this — One-by-One sees approximately 3.5× more RL training instances, which could contribute to its superior RL performance independently of the architectural differences.

No oracle or upper-bound analysis. The paper does not report what the maximum achievable performance would be if the model had perfect strategy prediction. For example: what is the dialogue success rate when using gold strategies with a strong response generator? This would help contextualize the 40.00% SR — is this approaching the ceiling for the ESConv task with simulated seekers, or is there substantial room for improvement? Similarly, the paper does not report the proportion of reference utterances with multiple strategies that are genuinely multi-strategy (as opposed to cases where the annotators split what could be considered a single strategy into two labels), which affects the interpretation of the 18.9% figure.

No latency or inference cost analysis. While the paper focuses on effectiveness, the One-by-One method requires multiple autoregressive generation steps per turn (up to K = 3), while All-in-One generates the full turn in one pass. The paper does not report the actual inference time or computational cost difference between these approaches. For real-time dialogue applications, the 2–3× inference cost of One-by-One could be a practical concern that offsets its dialogue-level SR advantage.

Limited exploration of the difficulty spectrum. The paper does not analyze performance as a function of dialogue characteristics — for example, whether multi-strategy generation helps more on certain emotion types (anxiety vs. depression vs. grief) or problem types (relationships vs. work vs. health), or whether the benefits are concentrated in early vs. late dialogue turns. Such analysis could provide practical guidance about when to deploy multi-strategy versus single-strategy generation.

6. Limitations and Trade-offs

6.1 The Approach Has Not Been Evaluated on Genuine Human-Seeker Interactions

The assumption or constraint. All dialogue-level evaluation in this paper uses simulated seekers — specifically, GPT-5 (gpt-5-2025-08-07) role-playing as a help-seeker with a predefined emotional problem. The critic that determines whether emotional support has succeeded is also GPT-5, using a four-level ordinal scale mapped to scalar rewards. The human evaluation (Section 4.3.2, Table 7) asks annotators to rank complete dialogues, but these dialogues were themselves generated through interaction with the GPT-5 seeker simulator, not with real humans in distress. The paper acknowledges this limitation explicitly in the Limitations section:

"our evaluation relies on simulated seekers for dialogue-level assessment; while this enables large-scale testing, it may not fully capture the nuances of real human emotional interactions"

The consequence. The headline dialogue-level success rates (13.85% for the single-strategy baseline, 40.00% for One-by-One + Rea. + RL) quantify performance against a specific LLM's simulation of seeker behavior, and we have no calibration of how this correlates with outcomes in real supportive conversations with actual help-seekers. Several failure modes are plausible and untested. The GPT-5 seeker may respond more predictably or more forgivingly than a real person in distress — for example, it may produce coherent, on-topic responses consistently, whereas a real user might be incoherent, hostile, or withdraw from the conversation. The GPT-5 critic may reward surface-level features of supportive language (phrases like "I understand how you feel") that correlate with strategy usage in the training data but do not reflect genuine emotional relief. Conversely, the critic may penalize supportive behaviors that are genuinely helpful but outside its training distribution — for instance, challenging a seeker's maladaptive beliefs directly rather than first offering validation, which can be therapeutically appropriate but may appear "less supportive" to a critic trained on polite, agreement-heavy interactions. The 40.00% SR number should therefore be interpreted as a relative ranking (One-by-One + Rea. + RL > Single-Strategy) rather than an absolute estimate of how often the system would succeed with real users.

What evidence exists in the paper. The human evaluation (Table 7) provides partial triangulation: human annotators preferred the multi-strategy systems over the single-strategy baseline across all four dimensions, with average ranks of 1.62–2.00 versus 2.18–2.40. However, the annotators were evaluating dialogues generated against the same GPT-5 seeker simulation, not against real human seekers. So the human evaluation validates that people find the multi-strategy responses more appropriate when reading the GPT-5 interactions — it does not validate that the system would perform well in a live conversation with a person experiencing genuine distress. The paper does not report any study with human seekers, any comparison of simulator-based success rates to human-baseline success rates, or any analysis of how the critic's judgments correlate with human assessments of emotional progress. The sample size is also modest (50 dialogues, 3 annotators), and inter-annotator agreement metrics are not reported.

Mitigation status. The paper does not attempt to address this beyond acknowledging it. The authors do not propose any method for bridging the sim-to-real gap, calibrating the critic against human judgments, or validating the approach with real seekers. This is a standard limitation of simulation-based dialogue evaluation, but it is particularly consequential for ESC because the task involves sensitive emotional content where model failures could cause genuine harm. The paper's ethical considerations section notes that "the model should not be viewed as a replacement for professional psychological or medical assistance," which appropriately scopes the contribution but does not resolve the evaluation gap.

6.2 The Systems Produce Far Fewer Multi-Strategy Utterances Than the Reference Data — And It's Not Clear Why

The assumption or constraint. The paper demonstrates that allowing multi-strategy generation improves dialogue outcomes, but the models consistently under-produce multi-strategy utterances relative to the reference distribution. As shown in Table 4, the ESConv reference data contains 18.9% utterances with two or more strategies. The best model (All-in-One + Rea. + RL) produces only 8.4% multi-strategy utterances; the best One-by-One configuration produces 7.7%. Even after reasoning and RL, the models use multiple strategies less than half as often as the human supporters in the training data. This gap means the systems are still fundamentally more conservative than the conversational behavior they are trained to emulate.

The consequence. The under-generation of multi-strategy utterances means the dialogue-level SR gains (13.85% → 40.00%) are being achieved with a partial implementation of the multi-strategy capability. The systems are improving outcomes primarily by being better at single-strategy selection and response generation (visible in Table 3: on single-strategy utterances, All-in-One + Rea. + RL achieves EMR 34.42 vs. 31.09 for the single-strategy baseline) and by using occasional multi-strategy turns on high-impact moments. But they are not realizing the full potential of multi-strategy generation — if increasing multi-strategy usage from 0% to ~8% yields a 3× improvement in SR, what would happen if the systems could match the 18.9% human rate? The paper cannot answer this, and the under-generation may indicate a fundamental limitation in how the models learn to compose strategies. Specifically, it may reflect that the models are risk-averse in strategy composition: predicting one safe strategy (e.g., Affirmation and Reassurance) is always rewarded in training because most turns are single-strategy; predicting two strategies risks getting the order wrong or generating an incoherent response, and the reward gradient from the Levenshtein Ratio may not provide sufficient incentive to take that risk.

What evidence exists in the paper. Table 4 provides the direct quantification. Figure 3 shows the RL learning curve for All-in-One: multi-strategy usage starts at ~7.05% at step 0, dips to ~5% at step 4, then rises to ~12.50% at step 16 — still below 18.9%. The paper does not investigate why the models under-generate. Potential causes that are suggested by the paper's own data but not explored: (1) the training data imbalance (82.3% single-strategy utterances) creates a strong prior toward single-strategy outputs that reasoning and RL only partially overcome; (2) the Levenshtein Ratio reward provides partial credit for single-strategy predictions on multi-strategy references — a model predicting [Affirmation] on a reference [Affirmation, Question] gets LR = 0.5, which may be "good enough" from the RL optimizer's perspective; (3) the cognitive reasoning chains, which are distilled from LLMs shown the gold multi-strategy references, may not provide strong enough signals about when to compose multiple strategies as opposed to which strategies to use.

Mitigation status. The paper identifies this as a limitation in Section Limitations: "the proportion of utterances with multiple strategies generated by our models is still lower than in the ESConv dataset, which may limit the diversity and richness of multi-strategy supportive responses." However, it does not propose or test any mechanism to address it — no data balancing experiments (e.g., oversampling multi-strategy instances during SFT or RL), no reward function modifications to explicitly incentivize higher multi-strategy rates, and no analysis of whether the gap is concentrated in specific dialogue contexts or strategy combinations. The paper suggests this as future work implicitly but does not provide a concrete direction.

6.3 No Ablation Exists to Isolate the Unique Contribution of the Multi-Strategy Formulation from Reasoning and RL Gains

The assumption or constraint. The paper's central claim is that multi-strategy generation is beneficial for ESC. The evidence for this claim comes from comparing a Single-Strategy baseline (no reasoning, no RL) against multi-strategy models that include reasoning and RL (Table 6: 13.85% SR vs. 34.62–40.00%). But the paper never evaluates a Single-Strategy model with the same reasoning and RL enhancements. The enhancements that drive the largest performance gains (cognitive reasoning and GRPO-based RL) are added to the multi-strategy models but not to the single-strategy baseline. This means the experimental design cannot separate two effects: (1) the benefit of allowing multiple strategies per utterance and (2) the benefit of reasoning-supervised training and RL optimization.

The consequence. We cannot determine how much of the 27-percentage-point SR improvement (13.85% to 40.00%) comes from the multi-strategy formulation versus from generally better strategy selection and response generation that would benefit any model, regardless of output structure. Evidence from the paper suggests that a substantial portion of the gain may be structure-independent: on single-strategy utterances (Table 3), the multi-strategy models with reasoning and RL achieve higher EMR than the single-strategy baseline (38.24 for One-by-One + Rea. + RL vs. 31.09), even though both are generating exactly one strategy. This indicates that reasoning and RL improve strategy selection even when the output format is the same. If a hypothetical Single-Strategy + Rea. + RL model achieved, say, 30% SR, then the unique contribution of the multi-strategy formulation would be ~10 percentage points, not ~27. This would still be meaningful but substantially smaller than what the current comparison implies. The paper cannot answer this counterfactual.

What evidence exists in the paper. The ablation structure in Tables 2 and 6 shows: base multi-strategy models without reasoning/RL achieve only modest gains over the single-strategy baseline (SR 16.15–17.69% vs. 13.85%). This suggests that the multi-strategy formulation alone provides a small benefit. But we do not know what a single-strategy model would achieve with reasoning and RL, so we cannot partition the total gain. The human evaluation (Table 7) partially addresses this by comparing the best multi-strategy configurations against the single-strategy baseline, but it does not include a Single-Strategy + Rea. + RL configuration either. The paper's discussion in Section 4.2.2 notes that "cognitive reasoning substantially improves performance across all metrics" for single-strategy utterances (Table 3), confirming that reasoning helps regardless of output structure — making the missing ablation particularly salient.

Mitigation status. Not addressed. The paper does not acknowledge this as a limitation, and the experimental design does not include the necessary control condition. The strong claim in the abstract — "modeling multiple strategies within an utterance is feasible and beneficial" — must be interpreted with the understanding that "beneficial" has only been demonstrated in combination with reasoning and RL, not in isolation. A reader deploying this approach would not know whether to invest in the multi-strategy formulation specifically, or whether simply applying reasoning and RL to their existing single-strategy system would yield comparable gains at lower implementation complexity.

6.4 The Approach Has Only Been Validated on ESConv with a Single Base Model Architecture

The assumption or constraint. All experiments — utterance-level, dialogue-level, human evaluation — use exactly one dataset (ESConv) and one base model architecture (LLaMA-3.1-8B-Instruct) for all fine-tuned systems. The paper does not evaluate on any other emotional support corpus (e.g., motivational interviewing datasets, counseling transcripts, peer support forums), any general empathetic dialogue benchmark, or any out-of-domain transfer scenario. The authors state: "Further evaluation on other emotional support conversation datasets or real-world deployment is needed to assess generalizability" (Limitations section).

The consequence. We do not know whether the findings — that multi-strategy generation improves SR from ~14% to ~40%, that One-by-One outperforms All-in-One in dialogue evaluation, that reasoning from multiple LLMs is better than from any single LLM — are specific to the ESConv dataset's properties or generalize to emotional support conversation more broadly. ESConv has specific characteristics that could influence the results: (1) the eight strategy types are well-defined and annotated with high agreement, making strategy prediction a relatively clean supervised task; other corpora might have fuzzier strategy boundaries. (2) The dialogues are relatively short (average 11.8 turns) and focused on everyday emotional challenges (relationships, work stress, health concerns), not clinical or crisis scenarios; the strategy composition patterns (which strategies tend to co-occur, in what order) may be dataset-specific. (3) The data was collected using crowdworkers role-playing as seekers, which may produce different conversational dynamics than genuine help-seeking interactions — and the 18.9% multi-strategy rate in ESConv may not reflect the rate in other supportive dialogue contexts. (4) The seeker simulation for dialogue evaluation is GPT-5 prompted with ESConv problem types and emotion types; whether GPT-5's seeker behavior generalizes to seekers with problems outside the ESConv distribution is unknown. Similarly, the base model architecture (LLaMA-3.1-8B-Instruct) may have specific properties — instruction-tuning style, dialogue capabilities, sensitivity to LoRA fine-tuning — that influence the results. The paper does not test with other model families (e.g., Qwen, Mistral, Gemma) or scales, so we cannot assess whether the multi-strategy benefit is model-dependent.

What evidence exists in the paper. None beyond ESConv and LLaMA-3.1-8B. The instruction-following LLM baselines (GPT-5, DeepSeek-R1, Qwen3-235B) are evaluated zero-shot on ESConv but achieve near-zero success rates (Table 6: 0.00% SR), which tells us that the task is difficult for non-fine-tuned models on this dataset but does not test generalization of the fine-tuned approach to other datasets.

Mitigation status. The paper acknowledges this limitation but offers no mitigation — no secondary dataset, no cross-domain transfer experiment, no multi-model comparison. The generalizability question is deferred entirely to future work. For a practitioner, this means the demonstrated 40% SR should be treated as an ESConv-specific result until replicated. A system built on this approach for a different domain (e.g., peer support for addiction recovery, customer service empathy, or mental health triage) would need its own validation, and there is no evidence in the paper about how performance would transfer.

6.5 The LoRA Configuration and RL Data Quantity Differences Confound the All-in-One vs. One-by-One Comparison

The assumption or constraint. The paper compares All-in-One and One-by-One as two architectural alternatives for multi-strategy generation, concluding that One-by-One + Rea. + RL achieves the best dialogue-level results (40.00% SR vs. 34.62%). However, the two methods differ in three confounded dimensions beyond the generation architecture: (1) LoRA capacity: All-in-One uses LoRA rank r = 8 with scaling factor α = 16, while One-by-One uses r = 16 with α = 32 — double the adapter parameters (Section 4.1, Appendix B). (2) RL training instances: All-in-One RL training uses 3,696 instances, while One-by-One RL training uses 12,759 instances (Table 8) — a ~3.5× difference arising from One-by-One's per-step decomposition. (3) RL reward function: One-by-One receives an additional flag reward component r_flag (Equation 6) that has no analog in the All-in-One reward (Equation 5). None of these differences is controlled for in the experimental design.

The consequence. The observed performance gap between One-by-One and All-in-One (particularly in dialogue SR: 40.00% vs. 34.62%) may be partially or entirely attributable to One-by-One having more trainable parameters, more RL optimization data, and/or a richer reward signal — not to the iterative generation architecture. The paper cannot isolate the contribution of the architectural decomposition. For a practitioner deciding which method to implement, this matters: if One-by-One's advantage comes primarily from seeing more RL data (a byproduct of the per-step decomposition that could be replicated for All-in-One by simply generating more training instances), then the choice of architecture is less important than the choice of data scale. If the advantage comes from the flag reward, then All-in-One could potentially be improved by adding a termination prediction to its flat output format. Without controlled ablations, neither hypothesis can be tested.

What evidence exists in the paper. The base SFT comparison (Table 2, rows 5 vs. 8) provides the cleanest head-to-head, since both methods use their respective LoRA configurations but neither has RL. Here, One-by-One achieves EMR 24.99 vs. All-in-One 23.61 — a 1.38-point gap that could reflect the architecture, the doubled LoRA capacity, the larger SFT data (12,759 vs. 10,679 instances), or some combination. The gap is small enough that architectural differences are unlikely to be dominant at this stage. When reasoning is added (rows 6 vs. 9), the gap narrows further (EMR 29.72 vs. 29.55), with the methods essentially tied. Only when RL is added (rows 7 vs. 10) does a substantial gap emerge (EMR 33.53 vs. 29.97), coinciding with the larger RL data difference and the additional flag reward. This pattern is consistent with the confound — the RL stage amplifies differences that could be driven by data quantity or reward design as much as architecture.

Mitigation status. The paper does not acknowledge or address this confound. The LoRA configuration differences are stated in Appendix B without justification. The RL instance count difference is visible in Table 8 but not discussed. The reward function difference is described in Section 3.3 but not ablated against a shared reward. A minimal set of controls would include: (1) evaluating both methods with the same LoRA rank, (2) reporting All-in-One performance with increased RL data (e.g., by generating variants or training for more epochs to match One-by-One's effective sample count), and (3) testing whether adding a termination signal to the All-in-One output format improves its RL performance.

6.6 The Dialogue-Level Success Rate Metric Depends on Arbitrary Thresholds and a Single Critic Model

The assumption or constraint. The dialogue-level evaluation protocol (Section 4.3) defines "success" as: after each turn, a GPT-5 critic evaluates the seeker's emotional state on a four-level ordinal scale (A: "feels worse," B: "feels the same," C: "feels better," D: "issue solved"), mapped to scalar rewards. A dialogue is successful if the final-turn score exceeds a predefined threshold. The paper does not specify this threshold value, but it is implied to be the boundary between level C and level D (since the prompt in Appendix E.4, Figure 13 states that D is the "solved" state and the success criterion requires the seeker's issue to be "sufficiently addressed"). This operationalization embeds several assumptions: (1) the four-level scale is a valid ordinal measure of emotional progress, (2) the mapping from levels to scalar rewards preserves the structure of the task, (3) GPT-5 can reliably distinguish among these levels for diverse emotional problems, and (4) the threshold for "success" correctly captures when a supportive conversation has been effective.

The consequence. The SR numbers are sensitive to choices that the paper does not explore. If the threshold were set more strictly (requiring unequivocal resolution rather than "feels better"), SRs would drop across all methods, potentially changing their relative ranking if different methods tend to achieve different levels of improvement (e.g., one method might consistently reach level C but rarely D, while another reaches D less often but sometimes fails entirely). If a different critic model were used (e.g., DeepSeek-R1, a fine-tuned classifier, or a different GPT-5 version), the SRs could shift because different models may apply different standards for what constitutes "the issue has been solved." The paper's choice of GPT-5 as the critic is motivated by Kim et al. (2025)'s finding that "stronger evaluators provide more reliable judgments," but strength is not calibrated against any ground truth — we do not know GPT-5's accuracy at assessing emotional progress relative to human expert judgments. The paper also averages 10 critic evaluations per turn to "produce a continuous final-turn score," introducing additional hyperparameters (the number of samples, the aggregation method) that are not ablated.

What evidence exists in the paper. The human evaluation (Table 7) provides some evidence that the critic's relative ranking aligns with human preferences — both the GPT-5 critic (via SR) and human annotators (via average rank) prefer the multi-strategy systems over the single-strategy baseline. However, the alignment is imperfect: the critic ranks One-by-One + Rea. + RL substantially above All-in-One + Rea. + RL (SR 40.00% vs. 34.62%), while human annotators consistently rank All-in-One + Rea. + RL slightly better (Overall: 1.66 vs. 2.00). This divergence suggests the critic and humans weight different aspects of dialogue quality, and the "true" ranking between these two configurations is unresolved. The paper does not report correlation between critic judgments and human annotator scores at the dialogue level, critic calibration metrics, or sensitivity of SR to the success threshold.

Mitigation status. Not addressed. The paper treats the success threshold and critic model as fixed components of the evaluation protocol without ablation or sensitivity analysis. The human evaluation partially addresses the concern about critic validity — by showing humans also prefer the multi-strategy systems — but does not validate the specific SR numbers or the ranking between the two multi-strategy methods. A practitioner deploying these systems would want to know: if I tune my model to maximize this specific SR metric, will it produce dialogues that humans actually find more helpful, or will it overfit to the GPT-5 critic's idiosyncratic preferences? The paper cannot answer this without more extensive human evaluation or critic calibration.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper does not introduce a fundamentally new algorithm or architecture — its contributions are a reformulation, a diagnostic, and a specific empirical finding about what is necessary to make that reformulation work. The magnitude is best understood as a reframing with sharp boundary conditions rather than a paradigm shift. Prior ESC research operated under an assumption (one strategy per turn) that was known to be factually inaccurate but whose practical cost had never been measured. This paper measures it, and the measurement is damning: a single-strategy system achieves 13.85% dialogue success rate, while a multi-strategy system with appropriate enhancements achieves 40.00%. The assumption is not merely inaccurate — it is actively harmful, tripling the failure rate on the task the field cares about.

The primary conceptual shift is from "structure follows evaluation convenience" to "structure should match task phenomenology." The one-strategy-per-turn assumption was never theoretically motivated; it was an engineering simplification that made strategy prediction a clean classification problem and strategy-conditioned generation a standard seq2seq task. This paper demonstrates that this simplification has real downstream costs: dialogues take more turns (9.56 vs. 8.46), strategies are deployed less efficiently (the baseline needs 9.56 strategies to achieve 13.85% SR, while the best multi-strategy system achieves 40.00% SR with 10.29 strategies), and human annotators consistently prefer the multi-strategy outputs. The paper effectively argues — through data, not rhetoric — that the structure of the output should mirror the structure of skilled human communication, and when it doesn't, the system underperforms in ways that surface-level metrics (BLEU, ROUGE) can mask.

The paper also resolves the ambiguity left by Bai et al. (2025), who identified that multi-strategy utterances exist in ESConv but could not determine whether modeling them was beneficial. The answer, per this paper, is conditional: multi-strategy generation alone provides marginal gains (SR 13.85% → 16–18%), but multi-strategy generation combined with cognitive reasoning and RL provides large gains (SR → 35–40%). This is a non-obvious conditional finding — one might reasonably have expected that simply allowing the model to produce multiple strategies would help, or alternatively that it would introduce too much noise to be worthwhile. The truth is in between: the formulation opens a capability that standard SFT cannot effectively learn, but reasoning supervision and RL optimization unlock it.

A methodological contribution is the demonstration that utterance-level and dialogue-level evaluation can diverge in opposite directions. The One-by-One + Rea. + RL model slightly degrades BLEU scores compared to the reasoning-only version (B-4: 3.88 → 3.71) while substantially improving dialogue success rate (26.15% → 40.00%). This is a concrete warning to the ESC community: optimizing for surface-level generation metrics may actively worsen the system's ability to help seekers. The paper implicitly argues — through its evaluation design — that dialogue-level success with simulated interaction should be the primary evaluation paradigm, with utterance-level metrics serving as diagnostics rather than objectives. This aligns with broader trends in dialogue evaluation but provides a specific, reproducible protocol for ESC that future work can adopt.

The research directions that become more attractive after this work:

Multi-strategy generation with better composition mechanisms. The paper shows that the base models under-produce multi-strategy utterances (7.7–8.4% vs. 18.9% reference rate) even after reasoning and RL. This gap is now quantified and actionable. Research on explicit strategy composition models — sequence-to-sequence models that explicitly predict strategy count, strategy set, and strategy order before generating text — becomes directly motivated by a measured performance deficit.
Reasoning-augmented training for structured dialogue outputs. The paper's strongest empirical signal is that reasoning is the single largest contributor to performance gains (e.g., All-in-One EMR: 23.61 → 29.72 from reasoning alone, versus 29.72 → 29.97 from adding RL). This suggests that for structured-output dialogue tasks, investing in high-quality reasoning supervision (whether from LLM distillation, human annotation, or automated rationale generation) may yield better returns than investing in more sophisticated decoding or RL algorithms.
Dialogue-level evaluation with strong simulators. The paper demonstrates that GPT-5 can serve as both seeker and critic for reproducible, large-scale ESC evaluation. This protocol — while limited (see Section 6.1) — enables rapid iteration that would be impossible with human evaluation alone. The 40.00% SR ceiling leaves substantial room for improvement, and the protocol provides a shared benchmark that the community can adopt.

Research directions that become somewhat less urgent after this work:

Incremental improvements to single-strategy ESC systems. The paper demonstrates a 3× gap between single-strategy and multi-strategy approaches in dialogue success rate. While single-strategy systems can certainly be improved (e.g., through better strategy prediction, more empathetic response generation), the structural ceiling is low: a perfect single-strategy system that always picks the right strategy and generates flawless responses would still be unable to match the efficiency of a system that can compose multiple strategies within a turn. Research effort may be better directed at multi-strategy approaches.
Pure architectural search for ESC without reasoning. The base multi-strategy models (no reasoning, no RL) achieve only modest gains over the single-strategy baseline (SR 16–18% vs. 14%). This suggests that architectural innovation alone — new decoding strategies, different model architectures — is unlikely to substantially improve ESC performance without the supervisory signal that reasoning provides. The field should expect that future ESC systems will incorporate explicit reasoning components, and that purely generative approaches will underperform.

Follow-Up Research This Work Enables

Ablate the contribution of the multi-strategy formulation versus reasoning + RL. The paper's central claim — that multi-strategy generation is beneficial — cannot be fully separated from the benefits of reasoning and RL with the current experimental design. A single clean experiment would resolve this: train a Single-Strategy baseline with the same cognitive reasoning distillation and GRPO-based RL as the multi-strategy models, then evaluate at both utterance and dialogue levels. If Single-Strategy + Rea. + RL achieves, say, 25–30% SR (versus 40% for One-by-One + Rea. + RL and 13.85% for the base Single-Strategy), then roughly half the gain is from better strategy selection generally and half is uniquely from multi-strategy composition. If it achieves 35–38% SR, then multi-strategy generation per se contributes only a few points, and the paper's main finding would be about reasoning + RL rather than output structure. This experiment requires no new methods — it is purely a control condition that the current paper omits. It should be the highest-priority follow-up for anyone building on this work, because it determines where to invest engineering effort.

Investigate why the models under-produce multi-strategy utterances and design mechanisms to close the gap. The 8.4% multi-strategy rate (All-in-One + Rea. + RL) versus 18.9% reference rate (Table 4) is a clearly quantified performance deficit. Several hypotheses are directly testable with the existing framework. (1) Data imbalance hypothesis: The training data is 82.3% single-strategy. Run an experiment where multi-strategy instances are oversampled during SFT (e.g., to achieve 50/50 balance) and measure whether the generation rate increases and whether it helps or hurts overall performance. (2) Reward structure hypothesis: The Levenshtein Ratio provides partial credit for single-strategy predictions on multi-strategy references (LR = 0.5 for predicting [Affirmation] when the reference is [Affirmation, Question]). Design a reward variant that explicitly penalizes under-generation — for example, subtract a penalty proportional to (reference_strategy_count - predicted_strategy_count) when the predicted count is lower — and test whether this increases multi-strategy rates without degrading response quality. (3) Reasoning signal hypothesis: The distilled reasoning chains, generated by LLMs shown the gold multi-strategy reference, may not provide strong signals about when to compose multiple strategies. Collect or generate reasoning chains that explicitly discuss whether multiple strategies are needed and why (e.g., "The seeker needs both validation and practical guidance in this turn because..."), and test whether this improves multi-strategy usage. Each of these experiments would provide diagnostic information about why the gap exists and whether it can be closed with existing tools, or whether fundamentally different training paradigms are needed.

Test whether the multi-strategy benefit generalizes across emotion types, problem types, and dialogue phases. The paper reports aggregate results on the full ESConv test set but does not analyze performance as a function of the dialogue metadata. ESConv annotates each dialogue with an emotion type (e.g., anxiety, depression, grief, anger) and a problem type (e.g., problems with friends, work issues, health concerns). A stratified analysis would reveal whether multi-strategy generation is universally beneficial or concentrated in specific contexts. For example: does multi-strategy generation help more for anxiety (where validation + suggestion might be an effective combination) than for grief (where a single strategy of empathetic listening might be more appropriate)? Does it help more in early dialogue turns (where building rapport and exploring the problem happen simultaneously) or in later turns (where the focus narrows to specific coping strategies)? Does it help more for relationship problems (which often involve multiple perspectives) than for practical problems (which may have more linear solution paths)? This analysis requires no new data or models — only the stratified evaluation of the existing systems — and would provide practical guidance about when to deploy multi-strategy versus single-strategy generation in a production system.

Generalize the multi-strategy framework to other dialogue domains with structured discourse functions. The All-in-One and One-by-One methods are not ESC-specific. Any dialogue domain where a single turn can fulfill multiple discourse functions — patient counseling (reflecting feelings + providing medical information), customer service (apologizing + offering solution + verifying satisfaction), educational tutoring (giving feedback + asking probing question + providing hint), negotiation (acknowledging other party's position + making counter-offer + justifying proposal) — could benefit from the same decomposition. A strong follow-up would select one such domain with existing annotated data (or annotate a small dataset with discourse function labels), replicate the All-in-One and One-by-One training pipeline (including reasoning distillation and RL), and measure whether multi-function generation improves task success rates. The key research question is whether the benefit is specific to emotional support (where composing multiple strategies may be particularly important for perceived empathy) or generalizes to any domain where human communication naturally composes discourse functions within turns. A negative result (no benefit in customer service, for example) would be informative — it would suggest that the multi-strategy benefit is tied to the relational and affective dimensions of ESC rather than to utterance-level information density, refining our understanding of when structured output decomposition matters.

Develop and validate a sim-to-real calibration methodology for ESC evaluation. The dialogue-level success rates (40.00% for the best system) are measured against a GPT-5 seeker and a GPT-5 critic. The paper acknowledges that this "may not fully capture the nuances of real human emotional interactions" but does not propose how to bridge the gap. A critical follow-up study would recruit a modest number of human seekers (perhaps 20–30 participants with mild to moderate emotional concerns, in a structured but naturalistic interaction) and have them interact with two or three of the paper's systems (e.g., Single-Strategy baseline vs. One-by-One + Rea. + RL). For each interaction, collect both the GPT-5 critic's success judgment and human-reported outcome measures (e.g., pre-post mood ratings, perceived empathy scales, session evaluation questionnaires). The research questions are: (1) Does the GPT-5 critic's success/failure classification agree with human self-reported improvement? (2) Does the relative ranking of systems (Single-Strategy < Multi-Strategy) hold for human seekers, and is the magnitude of the difference comparable? (3) Are there systematic biases — for example, does the critic over-estimate success because it is more easily satisfied than a real person, or under-estimate it because it applies overly rigid criteria? This study would transform the dialogue-level SR metric from an uncalibrated relative measure to a partially validated absolute measure with known relationship to human outcomes. It would also clarify whether the 40.00% SR represents a promising starting point (if human SR is similar) or a vastly over-optimistic estimate (if human SR is much lower). The paper's open-source release of code and data makes this follow-up directly feasible.

Combine dialogue-level strategy planning with turn-level multi-strategy generation. The paper studies multi-strategy generation within a single turn but does not model strategy sequences across turns — each turn's strategy choice is made independently given the dialogue context. Prior work (Cheng et al., 2022; Zhao et al., 2023) has modeled turn-level strategy transitions and lookahead planning, but under the single-strategy-per-turn assumption. A natural integration would be: a high-level planner selects a set of strategies for the upcoming turn (considering both the immediate seeker state and the anticipated trajectory of the conversation), and a low-level generator (All-in-One or One-by-One) composes those strategies into a coherent utterance. This addresses a limitation visible in the current paper: the models sometimes generate strategies that are individually appropriate but collectively redundant or poorly sequenced (visible in the <50% EMR rates even for the best models). A planner could enforce strategy diversity within a turn and optimize the sequence for conversational impact. The key experiment would compare a planner-generator system against the current turn-independent approach on both EMR (expecting better strategy sequencing) and dialogue SR (expecting more efficient emotional progress because strategies are coordinated within and across turns). This experiment is enabled by the paper's demonstration that multi-strategy generation is feasible at the turn level; it adds a hierarchical layer that the current work does not explore.

Test whether smaller or specialized teacher models can match the ensemble of large LLMs for reasoning distillation. The paper uses four very large, computationally expensive teacher LLMs (GPT-5, DeepSeek-R1, Qwen3-235B, Gemini-2.5-Flash) for reasoning distillation, incurring an 800 USD API cost. It does not test whether a single smaller model (e.g., LLaMA-3.1-70B, Mixtral-8×7B) can achieve comparable distillation quality, or whether a model fine-tuned specifically for ESC reasoning could outperform the general-purpose ensemble. A practical follow-up would compare distillation quality (measured by student model EMR and SR) from: (1) a single strong but smaller teacher (e.g., LLaMA-3.1-70B), (2) the full four-model ensemble, (3) a model fine-tuned on the same ESC reasoning task (e.g., LLaMA-3.1-8B fine-tuned on the gold reasoning chains from the ensemble, then used as a teacher). If a single fine-tuned teacher matches the ensemble, the distillation cost drops dramatically and the approach becomes far more accessible. If the ensemble is necessary, the finding suggests that reasoning diversity is genuinely important and that cost-reduction strategies should focus on cheaper ensembles rather than better individual models. Table 5 already provides some evidence on this question — showing that individual teachers achieve EMR 26.66–28.46 versus 29.72 for the ensemble — but it does not test whether those individual teachers, if strengthened through fine-tuning, could close the gap.

Practical Applications and Downstream Use Cases

Mental health support chatbots and peer support platforms. The most direct application is in systems that provide emotional support to users experiencing everyday distress — not clinical therapy, but the kind of supportive conversation a friend, peer counselor, or helpline volunteer might offer. The paper's dialogue-level results (40.00% SR for the best system vs. 13.85% for the single-strategy baseline) translate to a system that resolves roughly 3× as many conversations as a conventionally designed ESC system using the same base model. In a deployment context where each conversation has a cost (API calls, user time, human escalation if the chatbot fails), this efficiency gain is substantial. The average turn reduction (8.46 vs. 9.56 turns) means the multi-strategy system reaches resolution about 1 turn faster on average — a ~12% reduction in interaction length that improves user experience and reduces per-conversation compute cost. The paper's finding that the One-by-One method with reasoning and RL achieves the highest dialogue success rate (40.00%) makes it the recommended configuration for such deployments, though the human evaluation's slight preference for All-in-One (Table 7, Overall: 1.66 vs. 2.00) suggests that user-facing systems might benefit from an ensemble or a hybrid approach that optimizes both for task success and perceived conversational quality.

Data generation for training better ESC systems. The compute-optimal framework from the example paper (not this work) suggests a different use case: using the multi-strategy model as a data generation engine. A system with 40.00% dialogue success rate can generate a large volume of successful support conversations (with the GPT-5 seeker simulator), which can then be used to train future ESC models via distillation or supervised fine-tuning. The key advantage over using a single-strategy system for data generation is dialogue efficiency: the multi-strategy model produces shorter, more successful dialogues (8.46 turns, 40.00% SR) than the baseline (9.56 turns, 13.85% SR). This means each successful generated dialogue costs fewer inference calls, and a higher proportion of generated dialogues are successful (reducing the need to filter failures). In a self-improvement loop where the model generates training data for its own next iteration, the multi-strategy approach would produce higher-quality data at lower cost. The paper's open-source release makes this directly actionable: a practitioner can use the released One-by-One + Rea. + RL model to generate dialogues with the GPT-5 seeker simulator, filter for successful conversations, and fine-tune a new model on those trajectories.

Integration with human-in-the-loop support systems. Not all emotional support conversations can or should be fully automated. In many settings — crisis hotlines, employee assistance programs, online therapy platforms — the goal is to support human counselors, not replace them. A multi-strategy ESC system could serve as a real-time suggestion engine for human supporters: as the supporter types or speaks, the system proposes one or more strategies and draft responses based on the conversation so far, which the human can accept, modify, or ignore. The multi-strategy formulation is particularly well-suited to this use case because human supporters naturally think in terms of composing multiple supportive functions (e.g., "I should validate their feelings and then ask about coping strategies"), so the system's suggestions match their cognitive frame. The One-by-One architecture is especially appropriate for an interactive suggestion interface: the system could propose strategies one at a time (with a "add another strategy?" prompt), allowing the human supporter to accept or skip each proposed strategy sequentially. The paper's finding that reasoning improves strategy quality (EMR: 24.99 → 33.53 for One-by-One with reasoning + RL) means the system could also surface its reasoning chain to the human supporter, explaining why it recommends a particular strategy sequence — making the suggestion more trustworthy and educational. The 3× improvement in dialogue success rate over the single-strategy baseline suggests the suggestions would be substantially more useful than those from a conventional ESC system.

Domain adaptation of supportive dialogue systems. Organizations deploying emotional support systems often have domain-specific needs — a university counseling center needs different strategies and language than a veterans' support organization or a corporate wellness program. The paper's distillation pipeline (reasoning from large LLMs, followed by SFT and RL on domain-specific data) provides a template for domain adaptation that is more cost-effective than collecting large volumes of domain-specific annotated dialogues. An organization could: (1) collect a modest set of domain-relevant supportive conversations (perhaps 50–100 dialogues), (2) use large LLMs to annotate them with strategies and generate reasoning chains (following the paper's distillation prompts in Appendix E, Figures 8 and 11), (3) fine-tune LLaMA-3.1-8B (or a similar model) on this data using the One-by-One + reasoning format, and (4) optionally run RL with a domain-specific reward function. The paper's results (Table 5) show that reasoning from any strong LLM helps, and combining multiple LLMs helps more, giving organizations flexibility in teacher model choice based on available APIs and budget. The 800 USD cost reported by the paper provides a concrete budget estimate for the distillation step. The key open question — which a domain adaptation follow-up would need to answer — is whether the multi-strategy benefit (3× improvement in SR) transfers to domains with different strategy distributions or conversation dynamics, or whether it is specific to the everyday emotional challenges captured in ESConv.