A Survey on Evaluation of Large Language Models

🎯 Pitch

Despite their stunning performance, today's most advanced LLMs cannot reliably understand basic word meanings or reason abstractly—this survey systematically reveals these critical failure modes hidden beneath average benchmark scores. By organizing evaluation into what, where, and how, the authors argue that assessment must become a core discipline, not an afterthought, to steer LLMs toward genuine capability and safety.

1. Executive Summary

This paper presents the first comprehensive survey of evaluation methods for large language models, organizing the landscape across three dimensions: what to evaluate (tasks spanning NLP, reasoning, robustness, ethics, medicine, science, and agent applications), where to evaluate (datasets and benchmarks), and how to evaluate (automatic and human evaluation protocols). Analyzing hundreds of studies—predominantly on models such as ChatGPT, GPT-4, and PaLM—the paper catalogs both successes (LLMs excel at sentiment analysis, text classification, arithmetic reasoning, and machine translation) and failures (they struggle with abstract reasoning, non-Latin languages, semantic understanding of basic phrases, and adversarial prompt robustness). The paper establishes that evaluation should be treated as an essential discipline to drive LLM development, not merely a benchmarking afterthought—identifying that current static, public benchmarks are insufficient for assessing rapidly evolving models and proposing seven grand challenges including dynamic and evolving evaluation, complete behavioral testing, and unified evaluation systems that support all LLM tasks.

2. Context and Motivation

The Core Problem: The Field Has No Systematic Framework for Evaluating LLMs, and This Matters Deeply

The paper addresses a fundamental gap: as large language models rapidly proliferate and integrate into critical aspects of daily life, the field lacks a coherent, comprehensive understanding of how to evaluate them. This is not merely an academic exercise in taxonomy—the absence of systematic evaluation means we cannot reliably answer basic questions about what these models can and cannot do, where they fail, and whether they are safe to deploy in sensitive contexts.

The urgency of this problem stems from several converging factors that the paper articulates explicitly:

LLMs are being deployed in high-stakes domains without adequate evaluation. The paper notes that "LLMs are increasingly used by individuals with critical information needs, such as students or patients" (Section 1). When a medical student uses ChatGPT to understand a clinical concept, or a patient queries an LLM about symptoms, the model's accuracy—or lack thereof—has direct consequences for human well-being. Without systematic evaluation, we are essentially conducting an uncontrolled, society-wide experiment with models whose failure modes are poorly characterized.

The capabilities of LLMs are qualitatively different from previous models. Traditional NLP models were typically designed and evaluated for specific, narrow tasks—sentiment analysis on a particular dataset, machine translation between a specific language pair, and so on. LLMs, by contrast, are general-purpose systems that "possess the capabilities to solve diverse tasks, contrasting with prior models confined to solving specific tasks" (Section 1). This generality means that evaluation cannot simply reuse established task-specific benchmarks; it must account for the breadth of capabilities, the interaction effects between different types of tasks, and emergent behaviors that arise from scale rather than explicit training.

Existing evaluation protocols may be systematically misleading. The paper draws a historical parallel that illustrates why this matters. In the 1950s, the Perceptron algorithm was "touted as an Artificial General Intelligence (AGI) approach" but "was later revealed as inadequate due to its inability to resolve the XOR problem" (Section 1). The lesson is that evaluation failures can create an illusory sense of progress, directing research effort and investment toward approaches with fundamental limitations that proper evaluation would have exposed. Today's LLMs may have analogous hidden failure modes that current evaluation protocols miss.

Claims of AGI-level capabilities are being made without rigorous evaluation frameworks. The paper points to Bubeck et al.'s claim that "GPT-4 can be seen as sparks of AGI" while noting that "others contest this claim due to the human-crafted nature of its evaluation approach" (Section 1). This debate is irresolvable without agreement on what constitutes adequate evaluation—yet no such agreement exists. The paper positions itself as providing the necessary foundation for the field to develop such consensus.

Why This Problem Is Important

The paper argues for the importance of LLM evaluation along multiple dimensions, each with distinct real-world implications:

Understanding strengths and weaknesses for safe deployment. As the paper states: "evaluating LLMs helps us better understand the strengths and weakness of LLMs. For instance, the PromptBench benchmark illustrates that current LLMs are sensitive to adversarial prompts, thus a careful prompt engineering is necessary for better performance" (Section 1). This is not just about academic benchmarking—it directly informs whether a model should be trusted in a given application and what guardrails are needed. If we do not know that LLMs fail catastrophically on certain types of adversarial inputs, we cannot design systems to mitigate these failures.

Guiding human-LLM interaction design. The paper argues that "better evaluations can provide better guidance for human-LLMs interaction, which could inspire future interaction design and implementation" (Section 1). Understanding when and why models produce incorrect or harmful outputs is essential for designing interfaces that help users calibrate their trust appropriately—knowing when to rely on the model and when to verify independently.

Ensuring safety and reliability in critical sectors. The paper explicitly highlights this: "the broad applicability of LLMs underscores the paramount importance of ensuring their safety and reliability, particularly in safety-sensitive sectors such as financial institutions and healthcare facilities" (Section 1). A systematic evaluation framework is not optional infrastructure; it is a prerequisite for responsible deployment.

Addressing emergent abilities that outpace existing tests. The paper observes that "as LLMs are becoming larger with more emergent abilities, existing evaluation protocols may not be enough to evaluate their capabilities and potential risks" (Section 1). The concept of emergent abilities—capabilities that appear at certain scales without being explicitly trained—means that evaluation must evolve alongside models. Static benchmarks designed for smaller, narrower models may be trivially solved by larger LLMs (ceiling effects) or may fail to capture entirely new capabilities that did not exist when the benchmark was created.

Where Prior Approaches Fall Short

The paper identifies several specific limitations in the existing evaluation landscape that motivate its comprehensive survey:

Fragmentation across isolated studies without synthesis. The paper documents a Cambrian explosion of evaluation studies—Figure 2 shows the trend of evaluation papers over time, with a sharp acceleration in 2023. However, before this survey, there was "a comprehensive overview capturing the entire gamut of evaluations is still lacking" (Section 1). Individual papers evaluate specific models (ChatGPT, GPT-4, PaLM, etc.) on specific tasks, but there was no systematic effort to synthesize findings across studies, identify consistent patterns of success and failure, or establish a common taxonomy for organizing evaluation efforts.

Static benchmarks that cannot adapt to evolving models. The paper notes that "while there are some prior work on robustness evaluation, there are much room for advancement, such as including more diverse evaluation sets, examining more evaluation aspects, and developing more efficient evaluations to generate robustness tasks" (Section 3.2.1). More fundamentally, the paper argues in Section 7.4 that "existing evaluation protocols for most AI tasks rely on static and public benchmarks... unable to accurately assess the evolving abilities of LLMs, given their rapid rate of development." This creates multiple problems: models may memorize benchmark answers (training data contamination), benchmarks saturate as performance approaches ceiling, and benchmarks may fail to test for capabilities that have emerged since their creation.

Narrow evaluation that ignores important dimensions. The paper highlights that much evaluation has focused on standard NLP tasks—accuracy on GLUE, performance on reading comprehension, etc.—while neglecting critical dimensions such as robustness to adversarial inputs, ethical biases, factual reliability (hallucination), and behavior in multi-modal or multi-lingual contexts. The paper's taxonomy in Section 3 explicitly expands the scope beyond NLP tasks to include robustness, ethics, trustworthiness, social science, medicine, agent applications, and education, arguing these are all essential dimensions for understanding LLMs.

Reliance on automatic metrics that may not capture meaningful quality. While the paper discusses automatic evaluation extensively (Section 5.1), it also acknowledges limitations: metrics like BLEU, ROUGE, and exact match are convenient but may not correlate with human judgments of quality, particularly for open-ended generation tasks. The paper notes that human evaluation becomes necessary "in some non-standard cases where automatic evaluation is not suitable" and that "while some generation tasks can adopt certain automatic evaluation protocols, human evaluation in these tasks is more favorable as generation can always go better than standard answers" (Section 5.2).

Inconsistency in evaluation methodology across studies. Different papers use different prompts, different model versions, different sampling strategies, and different metrics, making it difficult to compare results across studies. The paper's survey implicitly reveals this heterogeneity—when summarizing findings about ChatGPT on reasoning, for example, some studies find strong performance on arithmetic reasoning while others find significant weaknesses, partly because they test different types of reasoning under different conditions. The survey itself is a partial remedy, aggregating findings to identify consistent patterns despite methodological variation.

Lack of evaluation for societal-level impacts. The paper argues that evaluation must extend "not only at the task level, but also at the society level for better understanding of their potential risks" (Section 1). This includes bias and fairness, toxicity, misinformation, and the broader implications of deploying LLMs at scale. Prior work had addressed some of these individually, but there was no unified framework for thinking about societal-level evaluation as an integral part of the evaluation discipline.

How This Paper Positions Itself

The paper's positioning can be understood along several key dimensions:

As the first comprehensive survey in a rapidly growing field. The authors explicitly state: "This paper serves as the first comprehensive survey on the evaluation of large language models" (Section 1). While acknowledging that prior surveys exist on specific aspects of LLMs (Zhao et al., 2023, provide a general LLM survey; Liu et al., 2023, survey ChatGPT-related research), none had provided a systematic taxonomy and synthesis specifically focused on evaluation. The paper's contribution is in organizing a fragmented literature into a coherent framework.

Through a three-dimensional organizing framework. The paper's central organizational innovation is the "what, where, how" taxonomy. As stated in Section 1: "we explore existing work in three dimensions: 1) What to evaluate, 2) Where to evaluate, and 3) How to evaluate." This framework is designed to be general, encompassing "the entire life cycle of LLMs evaluation" (Section 1). The "what" dimension covers the tasks and capabilities being tested; the "where" dimension covers the datasets and benchmarks used; the "how" dimension covers the evaluation protocols—automatic versus human, and the specific metrics and criteria employed.

As establishing evaluation as a discipline rather than a tool. The paper's most distinctive positioning is its argument that "evaluation should be treated as an essential discipline to better assist the development of LLMs" (Section 1). This reframes evaluation from being merely a measurement activity that happens after model development to being a first-class research area that should drive model development. The paper explicitly states: "Our key point is that evaluation should be treated as an essential discipline to drive the success of LLMs and other AI models" (Section 7). This is more than a taxonomic contribution—it is an argument for a fundamental shift in how the field allocates research attention.

As identifying success and failure patterns, not just cataloging work. Rather than simply listing which papers evaluated which models on which tasks, the paper explicitly aims to synthesize findings: "We summarize existing tasks in various areas and obtain insightful conclusions on the success and failure case of LLMs, providing experience for future research" (Section 1). The summary in Section 6 synthesizes across hundreds of studies to identify consistent patterns—what LLMs do well (language generation, sentiment analysis, text classification, arithmetic reasoning, translation) and where they fail (abstract reasoning, non-Latin languages, semantic nuance, adversarial robustness, factual hallucination).

As a forward-looking research agenda, not just a retrospective summary. The paper dedicates Section 7 to "Grand Challenges and Opportunities for Future Research," proposing seven specific research directions: designing AGI benchmarks, complete behavioral evaluation, robustness evaluation, dynamic and evolving evaluation, principled and trustworthy evaluation, unified evaluation supporting all tasks, and using evaluation to drive model enhancement. This positions the survey not as an endpoint but as a foundation for future work.

As a living resource for the community. The paper commits to maintaining "the related open-source materials at: https://github.com/MLGroupJLU/LLM-eval-survey" (Section 1) and explicitly invites community contributions: "Note that as the evaluation of LLMs is a hot research area, it is very likely that we cannot cover all benchmarks. We welcome suggestions and comments to make this list perfect" (Section 4). This positions the paper as an ongoing community resource rather than a static publication.

With important caveats about scope and temporality. The paper is careful to acknowledge its limitations: "Results and conclusions in each paper are original contributions of their corresponding authors" (Disclaimer section), and "due to the evolution of LLMs especially online services such as Claude and ChatGPT, it is very likely that they become stronger and some of the limitations described in this paper are mitigated (and new limitations may arise)" (Disclaimer section). This temporal caveat is particularly important given that the survey was published in December 2023, covering evaluation work up to approximately mid-2023—models and evaluation methods have continued to evolve rapidly since then.

In summary, the paper positions itself as filling a critical gap: the absence of a systematic, comprehensive framework for understanding LLM evaluation at a moment when LLMs are being deployed widely and claimed to exhibit AGI-like capabilities. It does not propose a new evaluation method or benchmark itself; rather, it provides the intellectual scaffolding for the field to organize and critically examine existing evaluation efforts, identify consistent patterns of model performance, and chart a research agenda for more rigorous, adaptive, and comprehensive evaluation in the future.

3. Technical Approach

3.1 Reader Orientation

This paper is a survey and taxonomy paper—it does not build a new system or model but rather constructs a comprehensive organizational framework for understanding the rapidly expanding landscape of LLM evaluation work. The "system" being described is the evaluation ecosystem itself: the tasks researchers use to test LLMs, the benchmarks and datasets where testing happens, and the methodologies and metrics used to perform testing. The core problem it addresses is that hundreds of evaluation studies appeared between 2020 and 2023 (Figure 2 shows a sharp exponential increase), but there was no systematic way to compare findings across studies, identify consistent patterns of model capabilities and limitations, or understand the gaps in existing evaluation protocols. The solution takes the shape of a three-dimensional taxonomy—what to evaluate, where to evaluate, and how to evaluate—that serves as both a retrospective synthesis and a prospective research agenda.

3.2 Big-Picture Architecture (Diagram in Words)

The evaluation ecosystem the paper organizes has four major conceptual components:

Evaluation Tasks (WHAT—Section 3): The specific capabilities and behaviors being tested. These span eight broad categories: natural language processing tasks (understanding, reasoning, generation, multilingual, factuality), robustness/ethics/bias/trustworthiness, social sciences, natural science and engineering, medical applications, agent applications, and other applications (education, search and recommendation, personality testing, and specific domain tasks).
Evaluation Datasets and Benchmarks (WHERE—Section 4): The concrete testing environments where tasks are instantiated. These are organized into three tiers: benchmarks for general language tasks (e.g., MMLU, HELM, BIG-bench, Chatbot Arena), benchmarks for specific downstream tasks (e.g., MATH for mathematics, MultiMedQA for medicine, CUAD for legal contract review), and benchmarks for multi-modal tasks (e.g., MME, MMBench, SEED-Bench).
Evaluation Protocols (HOW—Section 5): The methodologies used to perform evaluation, divided into two broad categories: automatic evaluation (using standard metrics like accuracy, F1, ROUGE, calibration error, fairness metrics, and robustness metrics computed algorithmically) and human evaluation (using human evaluators with defined criteria including accuracy, relevance, fluency, transparency, safety, and human alignment).
Meta-Analysis Framework (Summary—Section 6, Challenges—Section 7): The synthesis layer that aggregates findings across the above components, identifying consistent patterns of LLM success and failure, and articulating directions for future evaluation research. This is not a technical component per se, but the intellectual contribution that organizes all other components into a coherent whole.

Information flows through this ecosystem as follows: a research question about LLM capabilities → selection of appropriate tasks to test those capabilities → choice of datasets/benchmarks that operationalize those tasks → application of evaluation protocols (automatic or human) to generate measurements → aggregation and interpretation of results → identification of model strengths, weaknesses, and evaluation gaps → design of improved evaluation methods.

3.3 Roadmap for the Deep Dive

First, the taxonomy of tasks (WHAT): Understanding the full scope of capabilities being evaluated is foundational—everything else depends on knowing what aspects of LLMs matter for assessment. We will walk through each category, explaining what specific sub-tasks entail, what findings emerge, and why particular task choices matter.
Second, the benchmark landscape (WHERE): With tasks defined, we need to understand where evaluation actually happens—the specific datasets and benchmarks that instantiate abstract evaluation goals into concrete testable forms. We will cover general benchmarks, task-specific benchmarks, and multi-modal benchmarks, highlighting their design principles and limitations.
Third, the evaluation protocols (HOW): Once we know what is being tested and where, we need to understand how testing is performed. This covers the technical machinery of automatic evaluation (metrics for accuracy, calibration, fairness, and robustness) and the methodological considerations of human evaluation (number of evaluators, evaluation rubrics, expertise requirements).
Fourth, the synthesis layer: We will not cover this in detail in the Technical Approach section (it belongs in later sections of the full analysis), but we will note where the paper's organizing framework enables synthesis across components—for example, connecting task-specific findings to benchmark-specific limitations, or identifying how evaluation protocol choices affect the reliability of conclusions.

This ordering follows the logical dependency: tasks define what to measure, benchmarks provide where to measure it, and protocols specify how to measure. The paper's contribution is in making these dependencies explicit and using them to structure a comprehensive survey.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a survey and taxonomy paper whose core idea is that the fragmented landscape of LLM evaluation research can be organized productively along three dimensions—tasks, benchmarks, and protocols—and that this organization reveals both what we know about LLM capabilities and what critical gaps remain in our ability to assess them.

3.4.1 The Three-Dimensional Organizing Framework

The paper structures all evaluation work around three interlocking questions, stated explicitly in Section 1:

"we explore existing work in three dimensions: 1) What to evaluate, 2) Where to evaluate, and 3) How to evaluate. Specifically, 'what to evaluate' encapsulates existing evaluation tasks for LLMs, 'where to evaluate' involves selecting appropriate datasets and benchmarks for evaluation, while 'how to evaluate' is concerned with the evaluation process given appropriate tasks and datasets."

This tripartite framework is not merely taxonomic convenience—it captures a genuine dependency structure in evaluation practice. The "what" dimension specifies the capability being tested (e.g., "can this model reason arithmetically?"). The "where" dimension specifies the concrete instantiation of that capability as measurable test items (e.g., "using the MATH benchmark's algebra subset"). The "how" dimension specifies the measurement methodology (e.g., "computing exact match accuracy over 100 sampled solutions per problem"). Each dimension constrains the next: a poorly matched benchmark cannot validly assess a capability regardless of measurement rigor; a poorly chosen metric can misrepresent performance regardless of benchmark quality.

The paper's Figure 1 provides a visual map of this structure, with the "what" dimension broken into seven major task categories (natural language processing, robustness/ethics/bias/trustworthiness, social science, natural science and engineering, medical applications, agent applications, and other applications), each further subdivided into specific sub-tasks with representative citations. The "where" dimension categorizes 46 identified benchmarks into general, specific, and multi-modal. The "how" dimension distinguishes automatic evaluation (with sub-categories for accuracy, calibration, fairness, and robustness metrics) from human evaluation.

3.4.2 The Taxonomy of Evaluation Tasks: Natural Language Processing

Natural Language Understanding (NLU) Sub-Tasks

The paper identifies five distinct NLU sub-areas and organizes findings within each:

Sentiment Analysis. This task involves classifying text by emotional valence, typically as binary (positive/negative) or ternary (positive/neutral/negative). The paper synthesizes findings across multiple studies: Liang et al. [114] and Zeng et al. [243] found that LLM performance on sentiment analysis is "usually high"; Qin et al. [159] found ChatGPT's performance "superior to traditional sentiment analysis methods" and "close to that of GPT-3.5"; Wang et al. [218] found ChatGPT exhibits "exceptional performance" on fine-grained sentiment and emotion cause analysis; Zhang et al. [251] found LLMs show "significant advantages over small language models" in low-resource settings. However, Bang et al. [6] found that "the ability of ChatGPT to understand low-resource languages is limited." The overall pattern is that LLMs excel at sentiment analysis for high-resource languages but degrade for under-resourced languages.

Text Classification. This task extends beyond sentiment to classify text into arbitrary categories. The paper reports: Liang et al. [114] found GLM-130B achieved "an overall accuracy of 85.8% for miscellaneous text classification"; Yang and Menczer [233] found ChatGPT "can produce credibility ratings for a wide range of news outlets" with "a moderate correlation with those from human experts" and "acceptable accuracy in a binary classification scenario (AUC=0.89)"; Peña et al. [154] found that "using an LLM backbone in combination with SVM classifiers is a useful strategy to conduct the multi-label topic classification task in the domain of public affairs with accuracies over 85%." This reveals LLMs as strong text classifiers, even for unconventional problem settings like credibility assessment.

Natural Language Inference (NLI). This task tests whether a hypothesis logically follows from a premise. Unlike sentiment and classification, NLI reveals notable weaknesses. The paper reports: "Qin et al. [159] showed that ChatGPT outperforms GPT-3.5 for NLI tasks" and "ChatGPT excels in handling factual input that could be attributed to its RLHF training process in favoring human feedback." However, "Lee et al. [105] observed LLMs perform poorly in the scope of NLI and further fail in representing human disagreement, which indicates that LLMs still have a large room for improvement in this field." This is significant because NLI tests logical reasoning with language—a capability central to many downstream applications.

Semantic Understanding. This broad category covers interpretation of meaning beyond surface form. The paper synthesizes concerning findings: "Tao et al. [184] comprehensively evaluated the event semantic processing abilities of LLMs" and found that "LLMs possess an understanding of individual events, but their capacity to perceive the semantic similarity among events is constrained." For reasoning, "LLMs exhibit robust reasoning abilities in causal and intentional relations, yet their performance in other relation types is comparatively weaker." Even more striking, "Riccardi and Desai [166] explored the semantic proficiency of LLMs and showed that these models perform poorly in evaluating basic phrases. Furthermore, GPT-3.5 and Bard cannot distinguish between meaningful and nonsense phrases, consistently classifying highly nonsense phrases as meaningful." Even GPT-4 "performance is still significantly lower than that of humans." This reveals a fundamental limitation: LLMs appear to process language statistically rather than semantically, failing to distinguish meaningful from meaningless content when statistical patterns are preserved.

Social Knowledge Understanding. The paper reports findings from Choi et al. [23], who "evaluated how well models perform at learning and recognizing concepts of social knowledge." The results revealed that "despite being much smaller in the number of parameters, finetuning supervised models such as BERT lead to much better performance than zero-shot models using state-of-the-art LLMs, such as GPT, GPT-J-6B," demonstrating that "an increase in parameters does not necessarily guarantee a higher level of social knowledge in this particular scenario."

Reasoning Sub-Tasks

The paper categorizes reasoning evaluation into mathematical reasoning, commonsense reasoning, logical reasoning, and domain-specific reasoning. Since the prior sections already cover reasoning extensively, I will focus on the paper's specific organizational logic and synthesis patterns.

Mathematical Reasoning: The paper reports that "ChatGPT exhibits a strong capability for arithmetic reasoning by outperforming GPT-3.5 in the majority of tasks [159]" but "its proficiency in mathematical reasoning still requires improvement [6, 45, 265]." On symbolic reasoning, "ChatGPT is mostly worse than GPT-3.5, which may be because ChatGPT is prone to uncertain responses, leading to poor performance [6]." The paper notes a clear difficulty gradient: "ChatGPT's performance worsens as task difficulty increases: it correctly answered 83% of the questions at the recognition level, 62% at the comprehension level, 27% at the application level, and only 10% at the highest cognitive complexity level [31, 221]."

Logical Reasoning: The paper synthesizes findings showing LLMs outperform traditional methods on familiar patterns but struggle with generalization: "Liu et al. [124] indicated that ChatGPT and GPT-4 outperform traditional fine-tuning methods on most benchmarks, demonstrating their superiority in logical reasoning. However, both models face challenges when handling new and out-of-distribution data."

Abstract and Counterfactual Reasoning: The paper highlights significant limitations: "Wu et al. [227] showed that the current LLMs have certain limitations in abstract reasoning ability" through poor performance on counterfactual task variants, and "Gendron et al. [56] found that existing LLMs have very limited ability" on abstract reasoning tasks.

Organizational Logic: The paper's approach to organizing reasoning evaluation is to group by reasoning type (mathematical, commonsense, logical, domain-specific) and then within each type report findings across models and studies, identifying both areas of strength and consistent weaknesses. This reveals an important pattern: reasoning performance is highly task-specific—models that excel at mathematical reasoning may fail at abstract reasoning, and performance on familiar patterns may not generalize to novel variants.

Natural Language Generation (NLG) Sub-Tasks

The paper organizes NLG evaluation into summarization, dialogue, machine translation, question answering, and other generation tasks.

Summarization: The paper reports: "Liang et al. [114] found that TNLG v2 (530B) achieved the highest score in both scenarios, followed by OPT (175B) in second place. The fine-tuned Bart is still better than zero-shot ChatGPT." Specifically, "ChatGPT demonstrates comparable zero-shot performance to the text-davinci-002 [6], but performs worse than GPT-3.5 [159]." This is notable: even large LLMs do not consistently surpass fine-tuned smaller models on summarization, and newer models (ChatGPT) sometimes underperform older ones (GPT-3.5) on this task.

Dialogue: The paper reports: "Both Claude and ChatGPT generally achieve better performance across all dimensions when compared to GPT-3.5 [121, 159]." When comparing Claude and ChatGPT, "both models demonstrate competitive performance across different evaluation dimensions, with Claude slightly outperforming ChatGPT in specific configurations." However, "Bang et al. [6] underscores that fully fine-tuned models tailored for specific tasks surpass ChatGPT in both task-oriented and knowledge-based dialogue contexts."

Machine Translation: The paper synthesizes mixed findings: "Wang et al. [208] demonstrated that ChatGPT and GPT-4 exhibit superior performance in comparison to commercial machine translation (MT) systems, as evaluated by humans." However, translation quality is directional: "ChatGPT performs X→Eng translation well, but it still lacks the ability to perform Eng→X translation [6]." This asymmetry is a consistent finding with implications for multilingual deployment.

Question Answering (QA): The paper reports strong performance: "Liang et al. [114] found that among all the evaluated models, InstructGPT davinci v2 (175B) exhibited the highest performance in terms of accuracy, robustness, and fairness across the 9 QA scenarios." Both "GPT-3.5 and ChatGPT demonstrate significant advancements compared to GPT-3 in their ability to answer general knowledge questions" with "ChatGPT surpasses GPT-3.5 by more than 2% in terms of performance [9, 159]" in most domains. However, ChatGPT performs "slightly weaker than GPT-3.5 on the CommonsenseQA and Social IQA benchmarks" which "can be attributed to ChatGPT's cautious nature, as it tends to decline to provide an answer when there is insufficient information available."

Why these task distinctions matter: The paper's decomposition of NLG into sub-tasks reveals that LLM generation capabilities are not monolithic—models may excel at QA but underperform at summarization, or translate well into English but poorly from English. This task-specificity challenges any claim of general language generation capability.

Multilingual Tasks

The paper identifies multilingual evaluation as a critical and under-addressed dimension: "due to the relatively recent emergence of this technology, LLMs are primarily evaluated on English data, leading to a potential oversight of evaluating their multilingual performance."

The synthesized findings are consistently concerning: "Abdelali et al. [1] evaluated the performance of ChatGPT in standard Arabic NLP tasks and observed that ChatGPT exhibits lower performance compared to SOTA models in the zero-shot setting for most tasks." More broadly, studies by Ahuja et al. [2], Bang et al. [6], Lai et al. [100], and Zhang et al. [250] found that "these LLMs perform poorly when it came to non-Latin languages and languages with limited resources." Critically, "despite translating the input to English and using it as the query, generative LLMs still displays subpar performance across tasks and languages compared to SOTA models [2]."

This reveals an important limitation: the standard mitigation strategy of translating to English before processing does not fully close the performance gap, suggesting that the LLM's internal representations may be fundamentally English-centric.

Factuality

The paper defines factuality as "the extent to which the information or answers provided by the model align with real-world truths and verifiable facts." This includes "the ability of these models to maintain consistency with known facts, avoid generating misleading or false information (known as 'factual hallucination'), and effectively learn and recall factual knowledge."

Key findings: "Wang et al. [204] assessed the internal knowledge capabilities of several large models" on Natural Questions and TriviaQA datasets and found that "while GPT-4 and BingChat can provide correct answers for more than 80% of the questions, there is still a remaining gap of over 15% to achieve complete accuracy." Multiple methodological innovations are described: Honovich et al. [74] "transformed existing fact consistency tasks into binary labels... The research discovered that fact evaluation methods founded on natural language inference and question generation answering exhibit superior performance and can complement each other." Pezeshkpour [156] "proposed a novel metric, based on information theory, to assess the inclusion of specific knowledge in LLMs" using "the concept of uncertainty in knowledge to measure factualness, calculated by LLMs filling in prompts and examining the probability distribution of the answer." Min et al. [138] "broke down text generated by LLMs into individual 'atomic' facts, which were then evaluated for their correctness" using "FActScore... to measure the performance of estimators through the calculation of F1 scores."

Why this matters for evaluation design: The factuality section reveals an important pattern in evaluation methodology evolution: as simple accuracy metrics prove insufficient, researchers develop increasingly granular approaches—binary consistency labeling, information-theoretic uncertainty measures, and atomic fact decomposition—each capturing a different aspect of factual reliability. This pattern of moving from coarse to fine-grained evaluation recurs across multiple task categories in the paper.

3.4.3 The Taxonomy of Evaluation Tasks: Beyond Standard NLP

Robustness, Ethics, Bias, and Trustworthiness

The paper devotes Section 3.2 to these dimensions, arguing they have "gained increasing importance in assessing the performance of LLMs comprehensively."

Robustness: The paper distinguishes two sub-types:

Out-of-distribution (OOD) robustness: "Wang et al. [206] is an early work that evaluated ChatGPT and other LLMs from both the adversarial and OOD perspectives using existing benchmarks such as AdvGLUE, ANLI, and DDXPlus datasets." Yang et al. [234] "evaluated OOD robustness by extending the GLUE dataset."
Adversarial robustness: "Zhu et al. [264] evaluated the robustness of LLMs to prompts by proposing a unified benchmark called PromptBench. They comprehensively evaluated adversarial text attacks at multiple levels (character, word, sentence, and semantics). The results showed that contemporary LLMs are vulnerable to adversarial prompts, highlighting the importance of the models' robustness when facing adversarial inputs."

Ethics and Bias: The paper reports that "LLMs have been found to internalize, spread, and potentially magnify harmful information existing in the crawled training corpora, usually, toxic languages, like offensiveness, hate speech, and insults [53], as well as social biases like stereotypes towards people with a particular demographic identity (e.g., gender, race, religion, occupation, and ideology) [175]." Specific findings: "Zhuo et al. [266] used conventional testing sets and metrics to perform a systematic evaluation of ChatGPT's toxicity and social bias, finding that it still exhibits noxious content to some extend." Deshpande et al. [35] "introduced role-playing into the model and observed an increase in generated toxicity up to 6x" with "biased toxicity towards specific entities." Beyond social biases, LLMs have been assessed for "political tendency and personality traits [65, 167] based questionnaires like the Political Compass Test and MBTI test, demonstrating a propensity for progressive views and an ENFJ personality type."

Trustworthiness: This category encompasses concerns beyond robustness and ethics. "Wang et al. [201] offered a multifaceted exploration of trustworthiness vulnerabilities in the GPT models, especially GPT-3.5 and GPT-4. Their evaluation expanded beyond the typical trustworthiness concerns to include eight critical aspects: toxicity, stereotype bias, adversarial and out-of-distribution robustness, robustness to adversarial demonstrations, privacy, machine ethics, and fairness." The paper notes a concerning finding: "while GPT-4 often showcases improved trustworthiness over GPT-3.5 in standard evaluations, it is simultaneously more susceptible to attacks."

Hallucination is treated as a trustworthiness sub-problem: "LLMs are capable of generating coherent and seemingly factual text. However, the information generated can include factual inaccuracies or statements ungrounded in reality, a phenomenon known as hallucination [163, 253]."

Organizational logic: The paper groups robustness, ethics, bias, trustworthiness, and hallucination together because they represent dimensions of model behavior that go beyond task performance to encompass safety, reliability, and societal impact. This grouping reflects a broader trend in LLM evaluation: moving from "does the model get the right answer?" to "is the model safe, fair, and reliable even when it gets the right answer?"

The paper treats these as separate categories but applies a similar organizational pattern: for each domain, enumerate the specific sub-tasks, report synthesized findings across studies, and identify consistent limitations.

Social Science (Section 3.3): The paper reports that "Wu et al. [224] evaluated the potential use of LLMs in addressing scaling and measurement issues in social science and found that LLMs can generate meaningful responses regarding political ideology and significantly improve text-as-data methods." In computational social science, "Ziems et al. [269] presented a comprehensive evaluation of LLMs on several CSS tasks" finding that "LLMs achieve the best performance on misinformation, stance, and emotion classification" but "exhibit the lowest absolute performance on event argument extraction, character tropes, implicit hate, and empathy classification, achieving accuracy below 40%." In legal tasks, "the zero-shot performance of LLMs is mediocre in legal case judgment summarization" with problems including "incomplete sentences and words, meaningless sentences merge, and more serious errors such as inconsistent and hallucinated information [34]." The paper's conclusion: "existing LLMs cannot completely replace human professionals in this domain."

Natural Science and Engineering (Section 3.4):

Mathematics: "For fundamental mathematical problems, most large language models (LLMs) demonstrate proficiency in addition and subtraction, and possess some capability in multiplication. However, they face challenges when it comes to division, exponentiation, trigonometry functions, and logarithm functions [241]." On complex problems, "GPT-3 demonstrates nearly random performance, while GPT-3.5 shows improvement, and GPT-4 performs the best [3]" but "the peak performance remains relatively low compared to that of experts [15]."
General Science: "Castro Nascimento and Pimentel [18] presented five straightforward tasks from various subareas of chemistry to assess ChatGPT's comprehension of the subject, with accuracy ranging from 25% to 100%." "Guo et al. [61] created a comprehensive benchmark that encompasses 8 practical chemistry tasks... GPT-4 demonstrates superior performance compared to the other two models."
Engineering: "In code generation tasks, the smaller LLMs trained for the tasks are competitive in performance, and CodeGen-16B is comparable in performance to ChatGPT using a larger parameter setting, reaching about a 78% match [125]." However, "in commonsense planning tasks, LLMs may not perform well, even in simple planning tasks where humans excel [194, 195]."

Medical Applications (Section 3.5)

The paper categorizes medical evaluation into three sub-areas:

Medical Queries: "ChatGPT generated relatively accurate information for various medical queries, including genetics [39], radiation oncology physics [73], biomedicine [81], and many other medical disciplines [63, 87, 169], demonstrating its effectiveness in the field of medical queries to a certain extent." However, "Thirunavukarasu et al. [186] assessed ChatGPT's performance in primary care and found that its average score in the student comprehensive assessment falls below the passing score."

Medical Examination: Using the USMLE as a benchmark, "Gilson et al. [57] found that ChatGPT achieves varying accuracies across different datasets" and "Kung et al. [97] showed that ChatGPT achieves or approaches the passing threshold in these exams with no tailored training. The model demonstrates high consistency and insight."

Medical Assistants: The paper notes applications including "identifying gastrointestinal diseases [99], dementia diagnosis [217], accelerating the evaluation of COVID-19 literature [93]" but also "limitations and challenges, such as lack of originality, high input requirements, resource constraints, uncertainty in answers, and potential risks related to misdiagnosis and patient privacy issues."

Agent Applications (Section 3.6)

The paper treats this as a distinct category because it involves LLMs using external tools rather than solving tasks directly: "Equipping LLMs with external tools can greatly expand the capabilities of the model [160]." Key systems discussed include ToolLLM, KOSMOS-1, MRKL, Toolformer, TALM, and HuggingGPT, each representing a different approach to tool integration—from API calling to multi-model orchestration.

Other Applications (Section 3.7)

The paper covers education (LLMs as feedback providers, exam-takers), search and recommendation (LLMs as rankers and personalized recommenders), personality testing (using LLMs for psychometric assessment), and specific applications (game design, log parsing, model evaluation). The organizational choice to group these as "other" reflects that these are emerging application areas where evaluation methodology is less mature compared to core NLP tasks.

3.4.4 The Benchmark Landscape: Where Evaluation Happens

The paper organizes benchmarks into three tiers (Section 4), summarized in Table 7:

Benchmarks for General Tasks (Section 4.1): These evaluate LLMs across multiple capabilities simultaneously. Key examples with their distinguishing features:

MMLU [70]: "provides a comprehensive suite of tests for assessing text models in multi-task contexts." It covers 57 subjects across STEM, humanities, and social sciences, testing both knowledge and reasoning.
HELM [114]: Provides "a comprehensive assessment of LLMs" evaluating "language models across various aspects such as language understanding, generation, coherence, context sensitivity, common-sense reasoning, and domain-specific knowledge." Distinguishing feature: multi-metric evaluation rather than single-score ranking.
BIG-bench [182]: "introduces a diverse collection of 204 challenging tasks contributed by 450 authors from 132 institutions. These tasks cover various domains such as math, childhood development, linguistics, biology, common-sense reasoning, social bias, physics, software development, etc." Distinguishing feature: community-sourced, intentionally challenging tasks designed to push beyond existing capabilities.
Chatbot Arena [128] and MT-Bench [260]: Chatbot Arena uses "user engagement and voting" with an Elo rating system; MT-Bench "evaluates LLMs on multi-turn dialogues using comprehensive questions tailored to handling conversations." Distinguishing feature: evaluation through human preference in realistic conversational settings.
AGIEval [262]: "serves as a dedicated evaluation framework for assessing the performance of foundation models in the domain of human-centric standardized exams." It uses actual human exams (SAT, LSAT, GRE, etc.) as test items.
KoLA [236]: "focuses on assessing language models' comprehension and utilization of semantic knowledge for inference." Distinguishing feature: knowledge-oriented evaluation with self-contrast metrics.
PromptBench [264]: "centers on the importance of prompt engineering in fine-tuning LLMs" providing "a standardized evaluation framework to compare different prompt engineering techniques and assess their impact on model performance." Distinguishing feature: focus on prompt-level robustness rather than task-level accuracy.
PandaLM [216]: "introduced as a discriminative large-scale language model specifically designed to differentiate among multiple high-proficiency LLMs through training" incorporating "crucial subjective elements, including relative conciseness, clarity, adherence to instructions, comprehensiveness, and formality." Distinguishing feature: uses an LLM as the judge rather than fixed metrics.

Benchmarks for Specific Downstream Tasks (Section 4.2): These target particular domains or capabilities:

Medical: MultiMedQA [177] for clinical knowledge, CMB [211] for Chinese medical evaluation.
Mathematical: MATH [72] for reasoning and problem-solving.
Code: APPS [68] for Python code generation from natural language specifications.
Legal: CUAD [71] for legal contract review.
Ethics and Safety: TRUSTGPT [79] for toxicity, bias, and value-alignment; SafetyBench [254] for security evaluation.
Multilingual: C-Eval [78] as the "first extensive benchmark to assess the advanced knowledge and reasoning capabilities of foundation models in Chinese"; CMMLU [108] as a "comprehensive Chinese proficiency standard."
Tool Use: API-Bank [109] as "the first benchmark explicitly designed for tool-augmented LLMs" with "53 commonly used API tools and 264 annotated dialogues, encompassing a total of 568 API calls."
Hallucination: UHGEval [116] for Chinese LLM hallucination evaluation.
Multi-turn Interaction: MINT [213] for evaluating LLMs in multi-turn interactions with tools and natural language feedback.

Benchmarks for Multi-Modal Tasks (Section 4.3): These evaluate models that process both text and other modalities:

MME [46]: "serves as an extensive evaluative benchmark, aiming to assess their perceptual and cognitive aptitudes" with "meticulously crafted instruction-answer pairs alongside succinct instruction design."
MMBench [126]: "comprises a comprehensive dataset and employs a CircularEval assessment method."
SEED-Bench [107]: "consists of 19,000 multiple-choice questions that have been annotated by human assessors" covering "12 different aspects, including the models' proficiency in understanding patterns within images and videos."
LVLM-eHub [231]: "undertakes an exhaustive evaluation of LVLMs using an online competitive platform and quantitative capacity assessments."

Why this categorization matters: The paper's three-tier benchmark taxonomy reveals different evaluation philosophies. General benchmarks prioritize breadth and comparability across models; specific benchmarks prioritize depth in particular domains; multi-modal benchmarks test capabilities that text-only benchmarks cannot assess at all. The choice among these reflects different evaluation goals: general benchmarks are suited for model comparison, specific benchmarks for domain suitability assessment, and multi-modal benchmarks for testing integrated perception-cognition capabilities.

3.4.5 Evaluation Protocols: How Evaluation Is Performed

The paper's "how" dimension (Section 5) covers the mechanics of generating evaluation scores.

Automatic Evaluation (Section 5.1)

The paper defines automatic evaluation as using "standard metrics and evaluation tools to evaluate model performance" without human participation. Table 9 summarizes the key metrics across four categories:

Accuracy Metrics:

Exact Match (EM): "a metric used to evaluate whether the model's output in text generation tasks precisely matches the reference answer. In question answering tasks, if the model's generated answer is an exact match with the manually provided answer, the EM is 1; otherwise, it is 0."
F1 Score: A metric "for evaluating the performance of binary classification models, combining the model's precision and recall." Defined mathematically as:

$F_1 = 2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$

where Precision is the fraction of model-positive predictions that are truly positive and Recall is the fraction of truly positive instances the model identifies as positive.

What it computes: the harmonic mean of precision and recall, producing a single scalar between 0 and 1. Unlike arithmetic mean, the harmonic mean penalizes extreme imbalances—a model with 0.9 precision and 0.1 recall gets an F1 of 0.18, not 0.5.

Why this form: the harmonic mean ensures that both precision and recall must be reasonably high for a high F1, preventing models that achieve high scores by being either overly conservative (high precision, low recall) or overly aggressive (high recall, low precision).

ROUGE: "primarily employed to assess the performance of tasks such as text summarization and machine translation, involving considerations of overlap and matching between texts." Multiple variants exist (ROUGE-N, ROUGE-L) measuring n-gram overlap and longest common subsequence, respectively.

Calibration Metrics:

Expected Calibration Error (ECE): "one of the commonly used metrics to evaluate model calibration performance [60]." The paper notes that "Tian et al. [189] utilized ECE to study the calibration of RLHF-LMs, including ChatGPT, GPT-4, Claude 1, Claude 2 and Llama2." The computation involves "categorizing model predictions based on confidence and measuring the average accuracy of the predictions within each confidence interval." A perfectly calibrated model would have accuracy equal to confidence in every bin; ECE measures the weighted average deviation.

What it computes: for M equally-spaced confidence bins, ECE is:

$\text{ECE} = \sum_{m=1}^{M} \frac{|B_m|}{n} \left| \text{acc}(B_m) - \text{conf}(B_m) \right|$

where $B_m$ is the set of predictions in bin $m$ , $|B_m|$ is the number of predictions in that bin, $\text{acc}(B_m)$ is the observed accuracy within that bin, $\text{conf}(B_m)$ is the average predicted confidence in that bin, and $n$ is the total number of predictions.

Why this form: the weighting by bin size ensures larger bins contribute proportionally more to the error. The absolute difference between accuracy and confidence directly measures miscalibration—overconfidence (confidence > accuracy) and underconfidence (accuracy > confidence) are both penalized. This matters because overconfident wrong answers erode user trust more than underconfident correct answers.

Area Under the Curve (AUC) of selective accuracy and coverage [54]: "another commonly used metric" for calibration. This evaluates how well a model can identify which of its own predictions are likely to be correct (selective prediction), measuring the trade-off between the fraction of questions answered and the accuracy on those questions.

Fairness Metrics:

Demographic Parity Difference (DPD): "measures whether the model's predictions are distributed equally across different population groups." Mathematically:

$\text{DPD} = P(\hat{y} | Z = 1) - P(\hat{y} | Z = 0)$

where $\hat{y}$ is the binary classification prediction of the model and $Z$ is the identifier of the population group (usually binary, indicating two different groups, such as men and women). $P(\hat{y} | Z = 1)$ and $P(\hat{y} | Z = 0)$ respectively represent the probabilities of predicting the positive class in population $Z = 1$ and $Z = 0$ .

What it computes: the difference in positive prediction rates between two demographic groups. A value of 0 indicates equal positive prediction rates across groups (demographic parity). Positive or negative values indicate the model predicts the positive class more frequently for one group than another.

Why this form: demographic parity is the simplest fairness criterion—it requires only model predictions, not ground-truth labels. This makes it applicable even when labels are unavailable, but it has a known limitation: it does not account for possible legitimate differences in base rates between groups. If group A genuinely has more positive instances than group B, forcing equal prediction rates could reduce accuracy for both groups.

Equalized Odds Difference (EOD): "aims to ensure that the model provides equal error rates across different populations." Mathematically:

$\text{EOD} = \max \left\{ P(\hat{y} = 1 | Y = 1, Z = 1) - P(\hat{y} = 1 | Y = 1, Z = 0), P(\hat{y} = 1 | Y = 0, Z = 1) - P(\hat{y} = 1 | Y = 0, Z = 0) \right\}$

where $\hat{y}$ is the binary classification prediction, $Y$ is the true label, $Z$ is the demographic group identifier, $P(\hat{y} = 1 | Y = 1, Z = 1)$ is the probability of predicting positive when the true label is positive and the instance belongs to group $Z = 1$ (true positive rate for group 1), and $P(\hat{y} = 1 | Y = 0, Z = 1)$ is the probability of predicting positive when the true label is negative for group $Z = 1$ (false positive rate for group 1).

What it computes: the maximum of two differences: (1) the difference in true positive rates between groups and (2) the difference in false positive rates between groups. A value of 0 indicates both groups have identical error rates. Taking the maximum means that if either difference is large, the overall metric is large—equalized odds requires both error types to be balanced simultaneously.

Why this form: equalized odds is stricter than demographic parity because it conditions on the true label. This addresses the base-rate problem: if group A genuinely has more positive instances, equalized odds permits the model to predict positive more often for group A, as long as the error rates (how often it is wrong when the label is positive, and how often it is wrong when the label is negative) are equal across groups. The max operator ensures no single error asymmetry can be hidden by good performance on the other error type.

Robustness Metrics:

Attack Success Rate (ASR): "serves as a metric for evaluating the adversarial robustness of LLMs [206]." Given a dataset $D = \{(x_i, y_i)\}_{i=1}^{N}$ with $N$ pairs of samples and ground truth, an adversarial attack method $A$ that produces adversarial examples $A(x)$ , and a surrogate model $f$ :

$\text{ASR} = \frac{\sum_{(x,y) \in D} \mathbb{I}[f(A(x)) \neq y]}{\sum_{(x,y) \in D} \mathbb{I}[f(x) = y]}$

where $\mathbb{I}$ is the indicator function (1 if the condition is true, 0 otherwise). The numerator counts adversarial examples that cause incorrect predictions. The denominator counts clean examples that were originally classified correctly.

What it computes: the fraction of originally-correct predictions that become wrong under adversarial attack. An ASR of 0 means the model is perfectly robust (no originally-correct predictions change); an ASR of 1 means every originally-correct prediction becomes wrong after attack.

Why this form: normalizing by originally-correct predictions rather than all predictions focuses the metric on cases where robustness actually matters. A model that is wrong on a clean example provides no useful robustness signal—whether it changes or stays wrong tells us nothing about attack susceptibility. Only clean-correct-turned-attack-wrong counts as a genuine robustness failure.

Performance Drop Rate (PDR): "a new unified metric" from PromptBench [264] that "effectively assesses the robustness of prompt in LLMs" and "quantifies the relative performance degradation after a prompt attack":

$\text{PDR} = 1 - \frac{\sum_{(x,y) \in D} M[f([A(P), x]), y]}{\sum_{(x,y) \in D} M[f([P, x]), y]}$

where $A$ represents the adversarial attack applied to prompt $P$ , and $M$ denotes the evaluation function, which varies across different tasks. $f([P, x])$ is the model's output with the clean prompt, and $f([A(P), x])$ is the output with the adversarially perturbed prompt.

What it computes: the relative reduction in the evaluation metric after adversarial perturbation of the prompt. A PDR of 0 means no performance degradation (the adversarial prompt yields identical performance to the clean prompt); a PDR of 1 means complete performance collapse (the adversarial prompt yields zero score on the evaluation metric). Intermediate values represent fractional performance loss.

Why this form: PDR is task-agnostic because $M$ can be any evaluation function—accuracy for classification, F1 for QA, ROUGE for summarization. This unification allows comparing robustness across diverse tasks using a single normalized scale. The relative formulation (ratio rather than absolute difference) ensures comparability across tasks with different baseline performance levels: a 10-point drop from 90% to 80% and from 30% to 20% represent very different relative impacts, and PDR captures this through the multiplicative degradation factor.

Human Evaluation (Section 5.2)

The paper argues that "the increasingly strengthened capabilities of LLMs have certainly gone beyond standard evaluation metrics on general natural language tasks. Therefore, human evaluation becomes a natural choice in some non-standard cases where automatic evaluation is not suitable."

Table 10 summarizes the key factors in human evaluation:

Number of evaluators: Must balance "adequate representation" and "statistical significance." The paper does not prescribe specific numbers but flags that too few evaluators risk unrepresentative results while too many increase cost.

Evaluation rubrics: The paper elaborates six criteria derived from the "3H rule (Helpfulness, Honesty, and Harmlessness)" [4]:

Accuracy [178]: "assesses the precision and correctness of the generated text... scrutinizing the extent to which the language model produces information that aligns with factual knowledge, avoiding errors and inaccuracies."
Relevance [261]: "focuses on the appropriateness and significance of the generated content... how well the text addresses the given context or query, ensuring that the information provided is pertinent and directly applicable."
Fluency [196]: "assesses the language model's ability to produce content that flows smoothly, maintaining a consistent tone and style. A fluent text is not only grammatically correct but also ensures readability and a seamless user experience."
Transparency: "delves into the clarity and openness of the language model's decision-making process... assessing how well the model communicates its thought processes, enabling users to understand how and why certain responses are generated."
Safety [85]: "concerned with the potential harm or unintended consequences arising from the generated text... examines the language model's ability to avoid producing content that may be inappropriate, offensive, or harmful."
Human alignment: "assesses the degree to which the language model's output aligns with human values, preferences, and expectations... ensuring that the language model produces text that respects societal norms and user expectations."

Evaluator's expertise level: Must account for "relevant domain expertise, task familiarity, and methodological training." The paper notes that "delineating the requisite expertise level for evaluators ensures that they possess the necessary background knowledge to accurately comprehend and assess the domain-specific text generated by LLMs."

Why human evaluation matters despite its cost: The paper acknowledges that "even human evaluations can have high variance and instability, which could be due to cultural and individual differences [155]." Yet human evaluation remains essential because automatic metrics often fail to capture dimensions of quality that humans care about—particularly for open-ended generation where there is no single correct answer and quality is inherently subjective.

3.4.6 Novel Evaluation Approaches (Table 8)

The paper identifies three emerging trends in evaluation methodology:

Human-in-the-loop testing: Methods like AdaVision [50] and AdaTest [164] where "users filter test samples by only selecting high-quality tests and organizing them into semantically related topics." This integrates human judgment into the evaluation loop rather than treating it as a post-hoc assessment.

Crowd-sourcing testing: Tools like DynaBench [94], DynaBoard [132], DynamicTempLAMA [135], and DynaTask [188] that "rely on crowdworkers to create and test hard samples." This shifts from static to dynamic test set creation, enabling benchmarks to evolve as models improve.

More challenging tests: Approaches including HELM [114] for holistic multi-metric evaluation, AdaFilter [157] for adversarially constructed test examples, CheckList [165] for template-based behavioral testing, BIG-bench [182] as a platform for designing hard tasks, DeepTest [190] for systematic input transformation testing, and PromptBench [264] for adversarial prompt robustness evaluation.

Why these trends matter: The paper frames these as a progression from static, single-metric evaluation toward dynamic, multi-faceted, and adversarial evaluation. This shift reflects growing recognition that static benchmarks saturate (models eventually achieve near-perfect scores), fail to test for edge cases, and cannot keep pace with rapidly improving models. The emerging approaches attempt to create evaluation frameworks that co-evolve with the models they test.

3.4.7 Synthesis: Connecting Tasks, Benchmarks, and Protocols

While not a separate component with its own technical machinery, the paper's cross-dimensional synthesis (Sections 6 and 7) represents the intellectual core of its contribution. The key patterns identified are:

No single model excels at all tasks: The paper states explicitly: "despite all the efforts spent on summarizing existing works on evaluation, there is no evidence to explicitly show that one certain evaluation protocol or benchmark is the most useful and successful, but with different characteristics and focuses. This also demonstrates that not a single model can perform best in all kinds of tasks." This finding validates the need for multi-dimensional evaluation—a single benchmark score cannot capture LLM capability.

Consistent success patterns: LLMs perform well at sentiment analysis, text classification, factual input handling, arithmetic reasoning, logical reasoning on familiar patterns, machine translation (especially into English), and question answering. These capabilities span multiple benchmarks and evaluation protocols, suggesting genuine capability rather than benchmark-specific optimization.

Consistent failure patterns: LLMs struggle with abstract reasoning, non-Latin and low-resource languages, semantic understanding of basic phrases, adversarial robustness, factual hallucination, and complex planning. These failures are also consistent across studies and benchmarks.

Evaluation protocol dependence: The paper notes that conclusions can depend heavily on evaluation methodology. For instance, automatic metrics may overestimate performance on generation tasks where multiple valid outputs exist; human evaluation introduces cultural and individual variance. This means reported capabilities must be interpreted in the context of specific evaluation choices.

The evaluation-to-enhancement pipeline: The paper argues that "evaluation is not the end goal but rather the starting point" and that "a proficient evaluation system should not only offer benchmark results but should also deliver an insightful analysis, recommendations, and guidance for future research and development" (Section 7.7). This connects evaluation to model improvement, citing PromptBench's "word frequency analysis to identify robust and non-robust words in the test sets, thus providing prompt engineering guidance for end users" as an example of evaluation that directly informs practice.

3.4.8 Grand Challenges as Technical Design Requirements

The paper's seven grand challenges (Section 7) can be understood as design requirements for next-generation evaluation systems:

Designing AGI Benchmarks: How to construct tests that measure general intelligence rather than narrow capabilities, drawing on cross-disciplinary knowledge from education, psychology, and social sciences. Key open question: "does it make sense to use human values as a starting point for test construction, or should alternative perspectives be considered?"
Complete Behavioral Evaluation: Extending evaluation beyond static benchmarks to "open tasks such as complete behavioral tests" including "evaluations on a robot manipulated by LLMs to test its behaviors in real situations" and "multi-modal dimensions." This requires evaluation environments that simulate real-world interaction.
Robustness Evaluation: Moving beyond current robustness testing to include "more diverse evaluation sets, examining more evaluation aspects, and developing more efficient evaluations to generate robustness tasks" while "updating the evaluation system to better align with emerging requirements related to ethics and bias."
Dynamic and Evolving Evaluation: Addressing the temporal mismatch between static benchmarks and rapidly improving models. The paper identifies two problems: models may memorize benchmark answers (data contamination), and benchmarks may saturate. The solution requires "developing dynamic and evolving evaluation systems" that can generate genuinely novel test cases.
Principled and Trustworthy Evaluation: Establishing the evaluation system itself as reliable, which "poses a challenging research question that intertwines with measurement theory, probability, and numerous other domains." Key question: "how can we ensure that dynamic testing truly generates out-of-distribution examples?"
Unified Evaluation Supporting All LLM Tasks: Building evaluation systems that can handle the full diversity of evaluation needs including "value alignment, safety, verification, interdisciplinary research, fine-tuning, and others."
Beyond Evaluation: LLM Enhancement: Designing evaluation systems that not only measure performance but also provide actionable guidance for improvement—"after the evaluation, there are undoubtedly conclusions to be drawn regarding performance, robustness, stability, and other factors."

These challenges are not merely aspirational—they represent concrete technical gaps that the paper's survey makes visible. The framework of "what, where, how" provides the structure for systematically addressing each challenge: designing AGI benchmarks requires defining new evaluation tasks (what), creating dynamic evaluation requires new benchmark architectures (where), and ensuring trustworthy evaluation requires new measurement protocols (how).

4. Key Insights and Innovations

Innovation 1: Establishing Evaluation as an Independent Discipline Rather Than an Appendix to Model Development

The paper's most fundamental contribution is not taxonomic or methodological—it is conceptual reframing of the role evaluation plays in the LLM ecosystem. Before this survey, the dominant assumption across both research and practice was that evaluation is a downstream activity: you build a model, then you test it on standard benchmarks to measure progress. The paper argues—explicitly and repeatedly—that this reverses the proper relationship. In Section 7, the authors state their thesis with unusual force for a survey paper:

"Our key point is that evaluation should be treated as an essential discipline to drive the success of LLMs and other AI models. Existing protocols are not enough to thoroughly evaluate the true capabilities of LLMs, which poses grand challenges and triggers new opportunities for future research on LLMs evaluation."

This is not incremental. The field's default posture treats evaluation as measurement infrastructure—a set of benchmarks and metrics that exist to quantify how good models are. The paper argues evaluation should be generative: it should produce insights that actively guide model development, identify capability boundaries before deployment, and raise new research questions. The seven grand challenges in Section 7 operationalize this reframing: designing AGI benchmarks (what tests would actually measure general intelligence?), complete behavioral evaluation (how do we test models in open environments?), dynamic and evolving evaluation (how do we prevent benchmark saturation and memorization?), and so on. Each challenge is not merely "we need better X"—it identifies a structural gap in how the field conceptualizes evaluation's role.

Why does this matter? The historical parallel the paper draws (Section 1) is instructive. The Perceptron was "touted as an Artificial General Intelligence (AGI) approach in the 1950s" until evaluation revealed it "was later revealed as inadequate due to its inability to resolve the XOR problem." The failure was not that the Perceptron was badly evaluated—it was that the evaluation paradigm of the time was insufficiently rigorous to expose the fundamental limitation before the field invested heavily in the approach. The paper's reframing implies that today's LLMs face analogous risk: claims of AGI-like capabilities (Bubeck et al.'s "sparks of AGI") are being made without agreed-upon evaluation frameworks for testing those claims. Treating evaluation as a discipline means developing those frameworks before the claims harden into conventional wisdom.

The evidence for this insight is not a single table or figure—it is the entire architecture of the paper. The "what, where, how" taxonomy is not merely organizational convenience; it is the scaffolding for a discipline. The "what" dimension defines the phenomena a discipline studies (capabilities, behaviors, failure modes). The "where" dimension defines its instruments (benchmarks, datasets). The "how" dimension defines its methods (metrics, protocols). These are exactly the components that distinguish a mature scientific discipline from an ad-hoc measurement activity. The paper's insistence on maintaining a living repository of evaluation resources (the GitHub repository) and its explicit acknowledgment that "we may miss some new papers or benchmarks" and its invitation to community contributions reinforces this: evaluation as a discipline requires shared infrastructure that co-evolves with the models it studies.

Innovation 2: Success-and-Failure Pattern Synthesis as a Diagnostic Tool for Understanding LLM Capability Boundaries

While individual evaluation studies had documented specific strengths and weaknesses of particular models on particular tasks, the paper's cross-study synthesis of consistent success and failure patterns (Section 6) provides a qualitatively different kind of knowledge. Rather than asking "how well does Model X perform on Task Y?"—which is what individual evaluation papers answer—the paper asks "what capabilities do LLMs consistently demonstrate across models and tasks, and where do they consistently fail?"

The answer reveals a structured capability profile that no single study could establish:

Successes are concentrated in domains where statistical regularity dominates: LLMs excel at sentiment analysis (Section 3.1.1: "ChatGPT's sentiment analysis prediction performance is superior to traditional sentiment analysis methods"), text classification (GLM-130B at 85.8% accuracy), arithmetic reasoning (Section 3.1.2: "ChatGPT exhibits a strong capability for arithmetic reasoning by outperforming GPT-3.5"), machine translation into English (Section 3.1.3), and question answering (Section 3.1.3: InstructGPT achieves "highest performance in terms of accuracy, robustness, and fairness across the 9 QA scenarios"). The common thread: these tasks have strong statistical signal in training data—sentiment correlates with word choice patterns, arithmetic follows fixed rules, translation into English benefits from English-dominant training corpora.

Failures are concentrated in domains requiring semantic understanding, abstraction, and robustness to distribution shift: LLMs struggle with abstract reasoning (Section 3.1.2: Gendron et al. found "existing LLMs have very limited ability"), semantic understanding of basic phrases (Section 3.1.1: "GPT-3.5 and Bard cannot distinguish between meaningful and nonsense phrases"), non-Latin and low-resource languages (Section 3.1.4: "LLMs perform poorly when it came to non-Latin languages and languages with limited resources"), adversarial robustness (Section 3.2.1: "contemporary LLMs are vulnerable to adversarial prompts"), and factual consistency (Section 3.1.5: "there is still a remaining gap of over 15% to achieve complete accuracy" even for GPT-4). The common thread: these tasks require capabilities that go beyond statistical pattern matching—genuine understanding of meaning, systematic generalization, or resistance to adversarial perturbation.

This pattern synthesis is significant because it moves the conversation from "LLMs are impressive" or "LLMs have limitations" (both true but vacuous) toward a falsifiable capability model: LLMs are strong statistical pattern matchers with limited semantic grounding and poor robustness to distribution shift. This model generates predictions—for instance, that LLMs will perform well on tasks where surface-level statistical cues are predictive (even if misleading) and poorly on tasks requiring compositional generalization—that can be tested in future evaluation studies. It is the difference between cataloging observations and constructing a theory, and it represents the kind of intellectual contribution that only a comprehensive survey can provide.

The evidence for this innovation is distributed across the entire Section 3 taxonomy. The paper does not merely list which papers found what; it organizes findings to reveal patterns that cut across task categories. For instance, the failure of LLMs on non-Latin languages (Section 3.1.4) and their poor performance on semantic phrase evaluation (Section 3.1.1) are superficially different—one is about language coverage, the other about meaning comprehension—but the synthesis reveals a common underlying limitation: LLM representations appear fundamentally English-centric and statistically rather than semantically grounded. A reader of individual papers might miss this connection; the survey makes it explicit.

Innovation 3: The "What-Where-How" Taxonomy as a Generative Framework That Exposes Systematic Gaps in the Evaluation Ecosystem

The paper's tripartite taxonomy—what to evaluate, where to evaluate, how to evaluate—is not merely a filing system for existing work. It is a gap-detection mechanism: by forcing every evaluation study to be characterized along all three dimensions, the taxonomy reveals where evaluation effort is concentrated and where it is absent.

Consider what this taxonomy exposes that would otherwise remain invisible:

Gaps in the "what" dimension: The paper's task taxonomy (Section 3) reveals that certain capabilities receive disproportionate evaluation attention—NLP tasks dominate, with sentiment analysis, text classification, QA, and translation heavily studied—while others are sparsely evaluated. General science evaluation (Section 3.4.2) is described as having "limited evaluation studies" with current findings indicating "further improvement is needed." Social science evaluation (Section 3.3) is nascent, with the paper concluding that "existing LLMs cannot completely replace human professionals in this domain" but lacking the depth of evidence available for NLP tasks. This is not just an observation about coverage; it is an actionable diagnosis: researchers designing new benchmarks can see which task categories are underexplored and direct effort accordingly.

Gaps in the "where" dimension: The benchmark taxonomy (Section 4) reveals that most benchmarks are static and public—a property the paper identifies as problematic in Section 7.4 because static benchmarks can be memorized and saturate. By organizing benchmarks into general, specific, and multi-modal categories, the paper makes visible that certain benchmark types are missing entirely: there are no benchmarks listed that are both dynamic (evolving over time) and comprehensive (covering multiple task categories). This gap is not apparent from reading individual benchmark papers; it becomes visible only through systematic categorization.

Gaps in the "how" dimension: The evaluation protocol taxonomy (Section 5) reveals that automatic evaluation metrics are predominantly accuracy-oriented (exact match, F1, ROUGE), with calibration, fairness, and robustness metrics treated as separate, specialized categories. This organization makes visible that most LLM evaluation conflates "getting the right answer" with "being a good model"—a model can be highly accurate while being miscalibrated (overconfident on errors), unfair (accurate overall but inaccurate for specific demographic groups), and non-robust (accurate on clean test data but fragile under adversarial perturbation). The taxonomy reveals this by showing that these dimensions are typically evaluated separately, in different papers, using different benchmarks, rather than integrated into unified evaluation protocols.

Why is this framework generative—meaning it produces new research questions rather than merely organizing old ones? Because each intersection of the three dimensions represents an open evaluation design problem. For instance: how do you design a WHERE (benchmark) for the WHAT of "abstract reasoning" that uses a HOW of "human evaluation" rather than automatic metrics? The taxonomy does not answer this, but it makes the question precise. Similarly: what HOW (evaluation protocol) is appropriate for the WHAT of "trustworthiness" using the WHERE of "dynamic, evolving benchmarks" rather than static ones? These are not questions any individual paper in the survey asks; they emerge from the framework itself.

This is a fundamental shift from how survey papers typically operate. Most surveys organize existing work retrospectively—here is what has been done, grouped by topic. This paper's taxonomy is designed prospectively—here is a framework that reveals what has not been done, and what should be done next. Evidence for this claim is in Section 7, where each grand challenge maps directly onto gaps the taxonomy exposes: "Designing AGI Benchmarks" addresses the WHAT gap for general intelligence testing; "Dynamic and Evolving Evaluation" addresses the WHERE gap for static benchmarks; "Principled and Trustworthy Evaluation" addresses the HOW gap for evaluation reliability. The challenges are not arbitrary—they are precisely the gaps the taxonomy identifies.

Innovation 4: The Diagnosis That Static Benchmarks Are Structurally Incompatible with Rapidly Evolving Models

While individual papers had noted benchmark saturation (when model performance approaches ceiling) and data contamination (when benchmark answers leak into training data), the paper elevates these from methodological annoyances to a structural diagnosis of the evaluation paradigm itself. The insight in Section 7.4 is not just that static benchmarks have problems—it is that the static benchmark paradigm is fundamentally incompatible with the trajectory of LLM development:

"The capabilities of LLMs may enhance over time which cannot be consistently evaluated by existing static benchmarks. On the other hand, as LLMs grow increasingly powerful with larger model sizes and training set sizes, static and public benchmarks are likely to be memorized by LLMs, resulting in potential training data contamination."

This diagnosis has two components that interact. First, capability evolution outpaces benchmark design: by the time a benchmark is created, validated, and widely adopted, the models it was designed to evaluate may have already surpassed it. This is not a failure of benchmark quality—it is a timing problem inherent to the static paradigm. Second, memorization undermines validity: as benchmarks become widely used, their test instances become increasingly likely to appear in training data for future models, making it impossible to distinguish genuine capability from memorization. Again, this is not a failure of any specific benchmark—it is a property of the public-static paradigm itself.

What makes this diagnosis intellectually distinctive is that it does not propose to fix individual benchmarks (by making them harder, or larger, or more diverse) but rather argues for a paradigm shift to dynamic and evolving evaluation systems. This is a conceptual break: rather than creating better static tests, create testing processes that can generate novel test instances, adapt to model capabilities, and resist memorization by design. The paper points to early examples—crowd-sourcing testing (DynaBench, DynaBoard), human-in-the-loop testing (AdaTest, AdaVision), and adversarial test construction (AdaFilter, PromptBench)—but frames these not as incremental improvements but as early instances of a qualitatively different evaluation paradigm.

This insight connects to the paper's broader reframing of evaluation as a discipline (Innovation 1). A discipline with static instruments that become obsolete is not sustainable—it would be like physics using the same experimental apparatus indefinitely while the phenomena under study evolve. The move to dynamic evaluation is therefore not optional; it is necessary for evaluation to function as an ongoing discipline rather than a one-time certification.

The paper provides indirect evidence for this diagnosis through the sheer volume and rapid turnover of benchmarks it catalogs. Table 7 lists 46 benchmarks, most created within a span of 2-3 years. Many of these benchmarks are already approaching saturation for the strongest models. MMLU, introduced in 2020, saw GPT-4 achieve 86.4% accuracy in 2023—not yet ceiling, but rapidly approaching. MATH, introduced in 2021, remains challenging (GPT-4 achieves approximately 42.5% as referenced in other work), but the trajectory is clear. The paper does not provide a quantitative analysis of benchmark saturation rates, but the qualitative pattern across the surveyed literature is unambiguous: benchmarks have limited useful lifespans, and the rate of benchmark creation cannot keep pace with model improvement under the static paradigm.

Innovation 5: Reframing Multilingual and Robustness Failures as Evidence of a Fundamental Capability Boundary Rather Than Fixable Flaws

The paper's synthesis reveals that certain LLM failures are not merely areas for improvement but may indicate fundamental architectural or training-paradigm limitations. This is a stronger claim than "LLMs have weaknesses"—it suggests that some weaknesses cannot be addressed by scaling up or fine-tuning within the current paradigm.

The evidence is clearest in two domains:

Multilingual performance (Section 3.1.4): The paper reports a striking finding: "despite translating the input to English and using it as the query, generative LLMs still displays subpar performance across tasks and languages compared to SOTA models" (Ahuja et al., 2023). This is not just a training data imbalance problem that can be fixed by adding more multilingual data. If translating to English before processing does not close the gap, the limitation is not in the input encoding but in the model's internal representations—they appear to be fundamentally English-centric in ways that survive translation. This suggests a capability boundary: LLMs as currently architected may have an inherent English-language bias that no amount of post-hoc mitigation can fully address.

Semantic understanding (Section 3.1.1): The finding that "GPT-3.5 and Bard cannot distinguish between meaningful and nonsense phrases, consistently classifying highly nonsense phrases as meaningful" (Riccardi and Desai, 2023) is similarly diagnostic. This is not a difficult reasoning task or an adversarial challenge—it is a basic test of whether the model processes meaning or merely statistical co-occurrence. The failure suggests that LLMs' "understanding" is fundamentally statistical rather than semantic—they learn that certain word sequences are probable, not that certain sequences mean something and others do not. Even GPT-4 "performance is still significantly lower than that of humans." If this diagnosis is correct, it implies a capability boundary that cannot be crossed by scaling alone, because statistical learning from text corpora may be fundamentally insufficient to acquire semantic grounding.

What makes this an innovation rather than merely a summary of negative results is the diagnostic framing: these failures are treated not as bugs to be fixed but as probes that reveal the nature of the system. The distinction matters because it changes the research response. If multilingual failures are a bug, the response is to collect more multilingual training data. If they reveal a fundamental English-centricity in the architecture, the response is to reconsider how multilingual capacity is built into models from the ground up. If semantic failures are a bug, the response is to fine-tune on semantic tasks. If they reveal a fundamental limitation of text-only statistical learning, the response is to explore architectures that incorporate non-textual grounding (multi-modal training, embodied learning, etc.).

The paper does not resolve whether these are fixable flaws or fundamental boundaries—that question is left for future research. But by synthesizing findings across studies and framing them diagnostically, it transforms scattered negative results into a coherent challenge to the current LLM paradigm. This is a conceptual contribution characteristic of the best survey papers: organizing existing knowledge to expose the questions that existing knowledge cannot yet answer.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper surveys evaluation work conducted across a large number of datasets, benchmarks, and test suites, rather than running experiments on a single dataset. The key datasets referenced in the primary studies include MATH (Hendrycks et al., 2021: 12,000 training / 500 test high-school competition math problems), MMLU (Hendrycks et al., 2020: 57 subjects across STEM, humanities, and social sciences), TruthfulQA (Lin et al., 2021: designed to cause models to make mistakes), AdvGLUE (Wang et al., 2021: adversarial robustness benchmark), ANLI (Nie et al., 2019: adversarial NLI), DDXPlus (Fansi Tchango et al., 2022: medical diagnosis), GLUE (Wang et al., 2018: multi-task NLU), SuperGLUE (Wang et al., 2019: harder NLU), BIG-bench (Srivastava et al., 2022: 204 diverse tasks), Chatbot Arena (LMSYS, 2023: crowdsourced human preference data), and over 40 other benchmarks catalogued in Table 7. The paper does not execute its own experiments; it synthesizes results from hundreds of published evaluation studies, predominantly from 2022–2023.
Base model(s). The studies surveyed evaluate dozens of models. The most frequently evaluated are ChatGPT and GPT-4 (OpenAI, 2023), followed by GPT-3.5, GPT-3, PaLM and PaLM 2 (Google), Claude and Claude 2 (Anthropic), LLaMA and LLaMA-2 (Meta), BLOOM, GLM-130B, OPT (175B), BARD, InstructGPT, Codex, Vicuna, FLAN-T5, and smaller comparison models including BERT, Bart, CodeGen-16B, and CodeT5. The paper does not select these models—it reports on whatever models the primary studies evaluated. The most common evaluation target across the survey is ChatGPT, with GPT-4 increasingly evaluated in later papers.
Metrics. The paper does not compute metrics itself but catalogs the metrics used across the surveyed literature. The most common automatic metrics are accuracy (exact match, quasi-exact match, F1 score, ROUGE score), calibration metrics (Expected Calibration Error, Area Under the Curve), fairness metrics (Demographic Parity Difference, Equalized Odds Difference), and robustness metrics (Attack Success Rate, Performance Drop Rate). Human evaluation uses criteria including accuracy, relevance, fluency, transparency, safety, and human alignment. The paper's Tables 9 and 10 summarize these metrics and their formulations. Specific studies use additional domain-specific metrics (e.g., sacreBLEU for translation, pass@1 for math reasoning, Elo ratings for Chatbot Arena, win rate for MT-Bench judging by GPT-4).
Baselines. The paper itself does not define baselines—it is a survey. The primary studies it surveys use diverse baselines depending on the task: for NLP tasks, baselines typically include fine-tuned smaller models (BERT, Bart, T5), traditional ML methods (SVM classifiers), or earlier LLM versions (GPT-3 vs. GPT-3.5). For robustness evaluation, baselines include clean (non-adversarial) performance on the same dataset. For reasoning tasks, baselines include human performance (e.g., student exam scores on USMLE), prior SOTA supervised models, or random guessing. For multilingual evaluation, baselines include SOTA models trained specifically for each language. No unified baseline exists across the surveyed literature—the paper's contribution is in synthesizing findings despite this heterogeneity.
Generation budget / compute accounting. The surveyed papers use varied compute measurement approaches that are not standardized across studies. Some report model size in parameters as a coarse capability proxy. Others report accuracy as a function of the number of samples (e.g., pass@k for k sampled solutions). Most evaluation studies do not perform systematic compute-budget scaling experiments of the kind found in the reference example paper on test-time compute scaling—they evaluate at fixed generation settings (typically one or a few outputs per prompt) and report aggregate accuracy. The survey does not discuss FLOPs accounting, token budgets, or standardized compute measurement protocols. This is a notable absence: the survey catalogs what is evaluated but does not analyze how evaluation cost scales with model size or compute budget.
Cross-validation / statistical protocol. The paper notes in Section 2.2 that standard AI evaluation protocols exist (k-fold cross-validation, holdout validation, LOOCV, bootstrap) but observes that "some evaluation protocols may not be feasible to evaluate deep learning models due to the extensive training size. Thus, evaluation on a static validation set has long been the standard choice for deep learning models." The surveyed studies predominantly use static test set evaluation rather than cross-validation. The paper does not report statistical significance testing, confidence intervals, or effect sizes for the aggregated findings—it reports point estimates from individual studies without quantifying variance or uncertainty. The paper acknowledges that "even human evaluations can have high variance and instability, which could be due to cultural and individual differences" (citing Peng et al., 1997) but does not propose or evaluate methods for controlling this variance.

Main Quantitative Results

The paper does not produce its own quantitative results. Instead, it synthesizes numerical findings from the hundreds of studies it surveys. Unlike the reference example paper, which reports its own experimental data with specific accuracy numbers, figures, and tables, this survey reports findings at a higher level of abstraction—often stating qualitative patterns ("ChatGPT performs well on sentiment analysis") with representative numeric illustrations drawn from individual studies. The following organizes these synthesized quantitative findings by the paper's task taxonomy, indicating where in the survey specific numbers are reported.

Natural Language Processing Tasks

Sentiment analysis: The paper reports that "ChatGPT's sentiment analysis prediction performance is superior to traditional sentiment analysis methods" (Section 3.1.1, citing Lopez-Lira and Tang [129]) and "comes close to that of GPT-3.5" (citing Qin et al. [159]). Liang et al. [114] found performance on sentiment tasks is "usually high" across evaluated models, though specific accuracy numbers are not provided in the survey for this sub-task.

Text classification: The paper provides specific numbers. GLM-130B achieved "an overall accuracy of 85.8% for miscellaneous text classification" (Section 3.1.1, citing Liang et al. [114]). Yang and Menczer [233] found ChatGPT achieves "acceptable accuracy in a binary classification scenario (AUC=0.89)" for news outlet credibility rating. Peña et al. [154] reported "accuracies over 85%" for multi-label topic classification in public affairs using "an LLM backbone in combination with SVM classifiers."

Natural language inference: No specific numeric findings are reported in the survey for NLI performance. The paper states that "ChatGPT outperforms GPT-3.5 for NLI tasks" (citing Qin et al. [159]) but that "LLMs perform poorly in the scope of NLI and further fail in representing human disagreement" (citing Lee et al. [105]), without providing accuracy figures. This reliance on qualitative synthesis without quantitative substantiation is characteristic of the survey's limitations.

Reasoning (mathematical): The paper reports a clear difficulty gradient: ChatGPT "correctly answered 83% of the questions at the recognition level, 62% at the comprehension level, 27% at the application level, and only 10% at the highest cognitive complexity level" (Section 3.4.1, citing Dao and Le [31] and Wei et al. [221]). On more challenging problems, "GPT-4 outperforms ChatGPT by achieving a significant increase in accuracy of 10 percentage points and a reduction in relative error by 50%" (citing Wei et al. [221]). However, on the hardest problems, "GPT-3 demonstrates nearly random performance, while GPT-3.5 shows improvement, and GPT-4 performs the best" (citing Arora et al. [3]), with the caveat that "the peak performance remains relatively low compared to that of experts" (citing Bubeck et al. [15]). Wu et al. [225] found GPT-4 "reached 60% accuracy on half of the categories" for difficult high school competition problems, while "intermediate algebra and precalculus can only be solved with a low accuracy rate of around 20%."

Machine translation: Wang et al. [208] demonstrated that "ChatGPT and GPT-4 exhibit superior performance in comparison to commercial machine translation systems, as evaluated by humans" and "outperform most document-level NMT methods in terms of sacreBLEU scores" (Section 3.1.3). However, the performance is directionally asymmetric: "ChatGPT performs X→Eng translation well, but it still lacks the ability to perform Eng→X translation" (citing Bang et al. [6]). The survey does not provide the specific sacreBLEU or human evaluation scores.

Question answering: Liang et al. [114] found "InstructGPT davinci v2 (175B) exhibited the highest performance in terms of accuracy, robustness, and fairness across the 9 QA scenarios" among evaluated models (Section 3.1.3). ChatGPT "surpasses GPT-3.5 by more than 2% in terms of performance" in most domains (citing Bian et al. [9] and Qin et al. [159]), but performs "slightly weaker than GPT-3.5 on the CommonsenseQA and Social IQA benchmarks"—attributed to ChatGPT's tendency to "decline to provide an answer when there is insufficient information available."

Factuality: Wang et al. [204] found "GPT-4 and BingChat can provide correct answers for more than 80% of the questions" on Natural Questions and TriviaQA, but "there is still a remaining gap of over 15% to achieve complete accuracy" (Section 3.1.5). Pezeshkpour [156] reported that an information-theoretic approach to measuring factual knowledge "surpasses traditional ranking methods by achieving an accuracy improvement of over 30%." The FActScore metric (Min et al. [138]) revealed that "current estimators still have some way to go in effectively addressing the task" of factual precision evaluation, with F1 scores reported as the primary metric.

Robustness, Ethics, and Trustworthiness

Adversarial robustness: PromptBench (Zhu et al. [264]) evaluated adversarial text attacks at "multiple levels (character, word, sentence, and semantics)" (Section 3.2.1) and found that "contemporary LLMs are vulnerable to adversarial prompts." The paper does not provide the specific Attack Success Rate or Performance Drop Rate numbers from PromptBench in the body text, though these metrics are discussed in Section 5.1. Wang et al. [201] introduced AdvGLUE++ and found that "while GPT-4 often showcases improved trustworthiness over GPT-3.5 in standard evaluations, it is simultaneously more susceptible to attacks" (Section 3.2.3), but specific numeric comparisons are not reported.

Toxicity and bias: Deshpande et al. [35] found that introducing role-playing into the model caused "an increase in generated toxicity up to 6x" and caused "biased toxicity towards specific entities" (Section 3.2.2). Zhuo et al. [266] found that ChatGPT "still exhibits noxious content to some extend" using conventional testing sets and metrics from Dhamala et al. [37], Gehman et al. [53], and Parrish et al. [153], but specific toxicity scores are not reported in the survey. The paper notes that LLMs are found to have "a propensity for progressive views and an ENFJ personality type" based on Political Compass Test and MBTI test results (Hartmann et al. [65], Rutinowski et al. [167]).

Medical Applications

Medical examination (USMLE): Gilson et al. [57] "assessed ChatGPT's performance in answering USMLE Step 1 and Step 2 exam questions" and "indicated that ChatGPT achieves varying accuracies across different datasets"—specific accuracy numbers are not provided in the survey. Kung et al. [97] "showed that ChatGPT achieves or approaches the passing threshold in these exams with no tailored training" and "demonstrates high consistency and insight" (Section 3.5.2). Oh et al. [143] evaluated surgical clinical information understanding: "an overall accuracy of 46.8% for GPT-3.5 and 76.4% for GPT-4, demonstrating a significant performance difference between the two models." Thirunavukarasu et al. [186] found that ChatGPT's "average score in the student comprehensive assessment falls below the passing score" in primary care evaluation (Section 3.5.1).

Computational social science: Ziems et al. [269] found LLMs "exhibit the lowest absolute performance on event argument extraction, character tropes, implicit hate, and empathy classification, achieving accuracy below 40%" while achieving "the best performance on misinformation, stance, and emotion classification" (Section 3.3). For generation, "LLMs often produce explanations that surpass the quality of gold references provided by crowd workers."

Educational exams: de Winter [32] "showed that ChatGPT achieves an average of 71.8% correctness, which is comparable to the average score of all participating students" on high school English language comprehension exams. GPT-4 "achieved a score of 8.33" on a separate evaluation (Section 3.7.1, though the scoring scale is not defined in the survey).

Personality Testing

Emotional intelligence: Wang et al. [212] evaluated mainstream LLMs using a psychometric assessment framework and found "most LLMs achieve above-average scores in emotional quotient (EQ), with GPT-4 scoring 117, surpassing 89% of human participants" (Section 3.7.3). However, "a multivariate pattern analysis indicated that certain LLMs achieve human-level performance without relying on mechanisms resembling those found in humans."

The paper's most frequently referenced quantitative finding appears in its summary of what LLMs can do well (Section 6.1.1): LLMs demonstrate proficiency in generating fluent and precise text; obtain impressive performance in sentiment analysis, text classification, and factual input handling; demonstrate robust arithmetic reasoning; exhibit robust contextual comprehension; and achieve satisfying performance in machine translation, text generation, and question answering. What LLMs fail at (Section 6.1.2) includes subpar NLI performance; restricted proficiency in semantic similarity and basic phrase evaluation; limited abstract reasoning; suboptimal performance on non-Latin and low-resource languages; susceptibility to visual modal manipulation and toxic content generation; social biases and credibility deficits; limitations with real-time information; and sensitivity to adversarial prompts.

Ablation Studies and Robustness Checks

The paper does not conduct its own ablation studies or robustness checks—it is a survey. However, it reports on ablation-like analyses performed in the primary studies it surveys. The following identifies the closest equivalents to ablation studies and robustness checks described in the survey:

Model scale comparison (implicit ablation of parameter count): Multiple studies compare performance across model sizes or generations. The paper reports that for social knowledge understanding, "despite being much smaller in the number of parameters, finetuning supervised models such as BERT lead to much better performance than zero-shot models using state-of-the-art LLMs, such as GPT, GPT-J-6B" (Section 3.1.1, citing Choi et al. [23]), demonstrating that increased parameters do not guarantee better performance in all scenarios. For code generation, "the smaller LLMs trained for the tasks are competitive in performance, and CodeGen-16B is comparable in performance to ChatGPT using a larger parameter setting, reaching about a 78% match" (Section 3.4.3, citing Liu et al. [125]). For medical understanding, Oh et al. [143] showed a marked performance difference: 46.8% for GPT-3.5 versus 76.4% for GPT-4 on surgical clinical information, an improvement of 29.6 percentage points.

Fine-tuning vs. zero-shot comparison: Across multiple task categories, the survey identifies a consistent finding: fine-tuned smaller models often match or exceed zero-shot LLMs on specific tasks. For summarization, "The fine-tuned Bart is still better than zero-shot ChatGPT" (Section 3.1.3, citing Lewis et al. [106]). For dialogue, "fully fine-tuned models tailored for specific tasks surpass ChatGPT in both task-oriented and knowledge-based dialogue contexts" (Section 3.1.3, citing Bang et al. [6]). For QA, "Fine-tuned models, such as Vícuna and ChatGPT, exhibit exceptional performance with near-perfect scores, surpassing models that lack supervised fine-tuning by a significant margin" (citing Bai et al. [5] and Bang et al. [6]).

Role-playing ablation for toxicity: Deshpande et al. [35] showed that introducing role-playing (assigning personas to the model) caused "an increase in generated toxicity up to 6x" compared to non-role-playing baselines, with the additional finding that "such role-playing also caused biased toxicity towards specific entities" (Section 3.2.2). This functions as an ablation study testing whether toxicity is sensitive to the interaction paradigm.

Language direction ablation for translation: Bang et al. [6] found directional asymmetry: "ChatGPT performs X→Eng translation well, but it still lacks the ability to perform Eng→X translation" (Section 3.1.3). This is effectively an ablation testing whether translation capability is symmetric across language pairs.

Difficulty level stratification for reasoning: Multiple studies report performance stratified by problem difficulty. For mathematical reasoning, the paper reports a monotonic degradation: 83% at recognition level, 62% at comprehension, 27% at application, and 10% at highest cognitive complexity (Section 3.4.1, citing Dao and Le [31] and Wei et al. [221]). This functions as a difficulty ablation showing that LLM reasoning performance is highly sensitive to task complexity.

Prompt sensitivity as implicit robustness check: PromptBench [264] evaluated "adversarial text attacks at multiple levels (character, word, sentence, and semantics)" (Section 3.2.1) and showed that performance degrades substantially under adversarial perturbation. This is effectively a robustness check on the prompt modality, though specific PDR numbers are not reported in the survey body.

Non-Latin language ablation: The survey reports consistent findings that LLMs "perform poorly when it came to non-Latin languages and languages with limited resources" (Section 3.1.4, citing Ahuja et al. [2], Bang et al. [6], Lai et al. [100], Zhang et al. [250]). Critically, "despite translating the input to English and using it as the query, generative LLMs still displays subpar performance across tasks and languages compared to SOTA models" (Section 3.1.4, citing Ahuja et al. [2]). This translation-to-English baseline functions as a strong ablation: it tests whether multilingual performance deficits are due to input representation (they are not, since translation does not fully close the gap) or due to deeper representational limitations.

Critical Assessment

This paper is a survey, not an experimental study. Consequently, the standard critical assessment framework—"do the experiments support the claims?"—must be adapted. The paper's central claims are not experimental hypotheses tested with controlled ablations; they are taxonomic and synthetic assertions about the state of the evaluation literature. The appropriate critical question is: does the evidence the paper presents adequately support its synthetic conclusions, or are those conclusions overbroad given the underlying data?

Claim: "Evaluation should be treated as an essential discipline to better assist the development of LLMs." This is a normative claim rather than a factual one—it cannot be empirically verified or falsified through the experiments reported in the paper. The paper's evidence for the need for this discipline comes from its demonstration of fragmentation and incompleteness in current evaluation. On this evidentiary question, the paper succeeds in its own terms: the sheer volume of scattered evaluation studies (Figure 2 shows sharp growth through mid-2023), the heterogeneity in methodology (different prompts, different model versions, different metrics across studies), and the absence of any unified framework prior to this survey collectively demonstrate that evaluation has not functioned as a discipline. However, the paper does not empirically demonstrate that treating evaluation as a discipline would lead to better LLM development—this remains a plausible but untested assertion. No experiments compare outcomes with and without disciplined evaluation frameworks. The claim is better understood as a well-motivated proposal than as a demonstrated conclusion.

Claim: "LLMs demonstrate proficiency in sentiment analysis, text classification, arithmetic reasoning, and machine translation; they struggle with abstract reasoning, non-Latin languages, semantic phrase evaluation, and adversarial robustness." This is the paper's primary synthetic claim (Section 6) and it is well-supported by the volume and consistency of cited evidence. However, several caveats weaken its force:

The evidence quality is heterogeneous and unreplicated in the survey context. The paper reports findings from individual studies without independent verification. When one study finds "GPT-3.5 and Bard cannot distinguish between meaningful and nonsense phrases" (Riccardi and Desai [166]), the survey treats this as established fact about LLM capabilities. But these are findings contingent on specific model versions, prompt formulations, and evaluation protocols. The paper's synthesis aggregates findings across these variables without assessing whether inconsistencies across studies might reflect methodological differences rather than genuine capability differences. If five studies find LLMs good at task X and two find them poor, is the truth that LLMs are generally good at X, or that the five studies used easier test instances? The paper's methodology cannot distinguish these interpretations.
The temporal instability of findings is acknowledged but not addressed. The Disclaimer section states: "due to the evolution of LLMs especially online services such as Claude and ChatGPT, it is very likely that they become stronger and some of the limitations described in this paper are mitigated (and new limitations may arise)." This is accurate and important—but it means that the success/failure synthesis in Section 6 is a snapshot of LLM capabilities circa early-to-mid 2023, not a durable statement about LLMs as a technology class. Capability boundaries that appear fundamental today (semantic phrase evaluation, non-Latin language performance) may prove to be artifacts of specific model versions or training data choices. The paper's synthesis is therefore more useful as a historical record of what was true at a particular moment than as a predictive framework.
The lack of quantitative meta-analysis limits precision. The paper reports qualitative patterns ("LLMs perform well on sentiment analysis") with occasional numeric illustrations ("GLM-130B achieved an overall accuracy of 85.8% for miscellaneous text classification"). But it does not perform systematic meta-analysis—weighted effect sizes, confidence intervals, tests of heterogeneity across studies—that would quantify the reliability of its conclusions. For instance, the claim that LLMs are "sensitive to adversarial prompts" is supported by PromptBench, but the paper does not report: what is the mean Performance Drop Rate across all evaluated models and attack types? What is the variance? Are some models significantly more robust than others? Without these quantitative summaries, the conclusions are directional rather than precise.

Claim: "Existing static and public benchmarks are insufficient—dynamic and evolving evaluation is needed." This is the paper's most specific and actionable claim (Section 7.4). The evidence presented is largely theoretical and anecdotal: static benchmarks can be memorized, performance saturates, and model capabilities outpace benchmark design. This reasoning is valid, but the paper provides no empirical demonstration of the claimed insufficiency. Specific missing evidence includes:

No benchmark saturation analysis: The paper could have plotted model performance on standard benchmarks (MMLU, GLUE, SuperGLUE) over time, showing convergence toward ceiling. Such an analysis would empirically demonstrate that static benchmarks have finite useful lifespans. The paper does not provide this evidence, relying instead on the conceptual argument.
No data contamination analysis: The paper argues that "static and public benchmarks are likely to be memorized by LLMs" but does not present evidence of actual memorization—for instance, showing that models perform differently on benchmark test instances that appeared in training data versus held-out instances from the same distribution. This claim is plausible but unsubstantiated in the survey.
No comparison of dynamic versus static evaluation outcomes: The paper advocates for dynamic evaluation approaches (DynaBench, AdaTest, human-in-the-loop) but provides no evidence that these approaches produce materially different conclusions about model capabilities than static benchmarks produce. It is possible that dynamic evaluation is more expensive but yields identical rankings of models—in which case the case for adopting it would weaken significantly. This evidence gap is the most consequential missing experiment in the paper's argument.

Claim: "No single model performs best across all tasks." This claim is empirically well-supported in a weak sense—the paper catalogs many studies showing different models excel at different tasks. However, the claim would be stronger if the paper provided a systematic cross-model, cross-task comparison table showing, for each of, say, 10 major models and 20 representative tasks, which model ranks first and by what margin. Such a table would directly demonstrate the claimed heterogeneity. The paper's organizational structure—organizing by task rather than by model—makes it difficult for readers to construct this comparison themselves. This is a missed opportunity for quantitative synthesis.

Genuine weaknesses in the evidence base:

Single paper, single finding dependency: Many of the paper's most striking conclusions rest on individual studies. The finding that GPT-3.5 cannot distinguish meaningful from nonsense phrases comes from one paper (Riccardi and Desai [166]). The finding that role-playing increases toxicity up to 6x comes from one paper (Deshpande et al. [35]). The finding that LLMs achieve accuracy below 40% on implicit hate and empathy classification comes from one paper (Ziems et al. [269]). If any of these individual findings fail to replicate—due to model updates, methodological artifacts, or sampling variation—the corresponding synthetic conclusion in the survey becomes unsupported. The survey methodology does not include any mechanism for weighting findings by study quality, sample size, or replicability.
Missing quantification of evaluation protocol effects: The paper acknowledges that evaluation conclusions can depend on methodology—automatic metrics versus human evaluation, prompt formulation, sampling strategy—but does not systematically analyze how these choices affect the findings it reports. For instance, if a study finds ChatGPT performs poorly on reasoning using one prompt format, would the conclusion change with a different prompt? The survey cannot answer this because it treats primary study findings as facts about models rather than as interactions between models and evaluation protocols.
No power analysis or statistical reporting: The paper reports accuracy figures (85.8%, AUC=0.89, accuracy below 40%, 6x toxicity increase) without any of the statistical machinery that would allow readers to assess the reliability of these numbers—confidence intervals, p-values, effect sizes, or sample sizes. For human evaluation results, the number of evaluators, inter-rater reliability, and evaluation design details are generally not reported in the survey. This makes it impossible to distinguish precise findings from noisy estimates.
Missing experiments that would strengthen the survey: A systematic meta-analysis quantifying the heterogeneity of findings across studies evaluating the same model on the same task would directly test whether evaluation protocol differences produce materially different conclusions. A temporal analysis showing how reported model capabilities have changed across paper publication dates would test whether the synthesis is stable or rapidly obsolescing. A controlled re-evaluation of several key claims using a standardized prompt and evaluation protocol would validate the synthetic conclusions. Each of these would convert the survey from a qualitative summary into a quantitative synthesis with measurable reliability.

6. Limitations and Trade-offs

6.1 The Survey's Synthetic Claims Rest on Heterogeneous Primary Evidence Without Systematic Quality Control

The paper's central contribution—synthesizing hundreds of evaluation studies to identify consistent patterns of LLM success and failure—depends entirely on the reliability of individual primary studies. Yet the survey applies no systematic quality-control mechanism to the evidence it aggregates. Studies vary substantially in methodology (prompt design, sampling strategy, metric choice), model versions (different ChatGPT snapshots, different GPT-4 checkpoints), and rigor (sample sizes, statistical reporting, human evaluator counts). Treating all findings as equally informative creates a synthesis whose conclusions are only as robust as the weakest included studies.

The consequence is that several of the paper's most striking failure-case claims—and therefore the overall capability boundary the survey draws—rest on single, unreplicated findings from individual studies. The claim that "GPT-3.5 and Bard cannot distinguish between meaningful and nonsense phrases, consistently classifying highly nonsense phrases as meaningful" (Section 3.1.1, citing Riccardi and Desai [166]) comes from one paper. The finding that role-playing "caused an increase in generated toxicity up to 6x" (Section 3.2.2, citing Deshpande et al. [35]) comes from one paper. The claim that LLMs "exhibit the lowest absolute performance on event argument extraction, character tropes, implicit hate, and empathy classification, achieving accuracy below 40%" (Section 3.3, citing Ziems et al. [269]) comes from one paper. If any of these findings fail to replicate—due to model updates (the ChatGPT evaluated in early 2023 may behave differently from later versions), methodological artifacts, or sampling variation—the survey's corresponding capability claim becomes unsupported. The aggregated picture of "what LLMs cannot do" would shift.

The paper provides direct evidence of this limitation through the heterogeneity it documents across studies evaluating ostensibly similar capabilities. For NLI, the survey reports that Qin et al. [159] found "ChatGPT outperforms GPT-3.5" while Lee et al. [105] found "LLMs perform poorly in the scope of NLI" (Section 3.1.1). These are not contradictory findings about a fixed capability—they likely reflect different test sets, prompts, or evaluation protocols—but the survey does not disentangle whether disagreements indicate genuine capability variation or methodological artifacts. The survey's own cross-study comparisons implicitly reveal this heterogeneity, but the synthesis does not quantify it or assess whether it undermines the aggregated conclusions.

The paper does not attempt to mitigate this limitation. It includes no meta-analytic machinery—no weighting of studies by sample size or methodological rigor, no formal tests of heterogeneity, no sensitivity analyses excluding studies with weaker designs. The Disclaimer section acknowledges temporal instability ("due to the evolution of LLMs especially online services such as Claude and ChatGPT, it is very likely that they become stronger and some of the limitations described in this paper are mitigated") but does not address cross-study reliability at a fixed point in time. The survey treats primary-study findings as facts about model capabilities rather than as estimates with measurement error contingent on specific methodological choices. This is a fundamental constraint on the reliability of any synthetic claim in the paper—and the paper is transparent that it does not resolve it.

6.2 The Success-and-Failure Synthesis Is a Mid-2023 Snapshot, Not a Durable Capability Boundary

The paper acknowledges this limitation explicitly in its Disclaimer section:

"due to the evolution of LLMs especially online services such as Claude and ChatGPT, it is very likely that they become stronger and some of the limitations described in this paper are mitigated (and new limitations may arise)."

This temporal instability is not a minor caveat—it fundamentally constrains what the survey's synthetic claims mean. When the paper states that LLMs "struggle with abstract reasoning" (Section 6.1.2) or "cannot distinguish between meaningful and nonsense phrases" (Section 3.1.1), these are empirical claims about specific model versions evaluated at specific times. Models released after the survey period (post-mid-2023) may have been explicitly trained to address precisely these limitations, particularly given that evaluation papers often serve as de facto bug reports that model developers use to target improvements.

The consequence is that the capability boundaries the paper identifies—particularly the starkest failure cases—may already be partially or fully obsolete for the models practitioners actually use. If GPT-4, Claude, or Gemini updates released in late 2023 or 2024 incorporated training data or fine-tuning that addresses semantic phrase discrimination, abstract reasoning, or non-Latin language performance, the survey's characterization of these as consistent LLM weaknesses becomes misleading. A practitioner reading the survey in 2024 might incorrectly conclude that current frontier models cannot perform tasks they actually handle competently.

What makes this limitation particularly consequential is the interaction with the paper's claim that certain failures represent fundamental capability boundaries rather than fixable flaws (see Innovation 5 in Section 4). If multilingual performance deficits persist after translation to English—a finding the paper treats as diagnostic of a deep architectural limitation—but the next generation of models shows substantially improved performance on exactly this dimension, the fundamental-boundary interpretation would be falsified. The survey cannot distinguish between "limitation of current models in 2023" and "fundamental constraint of the LLM paradigm," but its synthesis language sometimes implies the latter.

The paper attempts to mitigate this limitation through its Disclaimer, which explicitly warns readers that limitations may become obsolete. However, this mitigation does not actually address the problem—it merely acknowledges it. The survey provides no temporal analysis showing how reported capabilities have changed across publication dates, no comparison of findings across model generations (ChatGPT vs. GPT-4 performance on the same task), and no framework for predicting which failure modes are likely to persist versus resolve with scale or training improvements. The temporal snapshot nature of the synthesis is a structural limitation of any survey covering a rapidly moving field, but the paper's claim to identify durable capability boundaries (as opposed to transient limitations of specific models) requires evidence it does not provide.

6.3 The Paper Advocates for Dynamic and Evolving Evaluation Without Empirical Evidence That It Would Change Conclusions

The paper's central normative proposal—that the field must shift from static benchmarks to "dynamic and evolving evaluation systems" (Section 7.4)—is foundational to its argument for evaluation as a discipline. This proposal rests on two claims about static benchmarks: that they suffer from data contamination (models memorize benchmark answers) and benchmark saturation (performance approaches ceiling). Both claims are conceptually valid and widely discussed in the literature. However, the paper provides no empirical demonstration that these problems actually distort the capability conclusions that evaluation studies produce.

The consequence is a major evidentiary gap in the paper's central argument. The case for abandoning the dominant evaluation paradigm in favor of a qualitatively different (and more expensive) one requires showing that the current paradigm leads to systematically wrong or misleading assessments of model capability. The paper does not provide:

Benchmark saturation analysis: The paper could have plotted model performance on standard benchmarks (MMLU, GLUE, SuperGLUE) over time, showing convergence toward ceiling. An empirical demonstration that benchmarks have finite useful lifespans would strengthen the claim that static benchmarks are structurally insufficient. Instead, the argument rests on conceptual reasoning—"the capabilities of LLMs may enhance over time which cannot be consistently evaluated by existing static benchmarks" (Section 7.4)—without quantitative evidence.
Data contamination measurement: The paper argues that "static and public benchmarks are likely to be memorized by LLMs, resulting in potential training data contamination" (Section 7.4). This claim is testable—for instance, by comparing model performance on benchmark test instances that demonstrably appeared in training data versus held-out instances from the same distribution, or by using canary strings to detect memorization. The survey does not report any such analysis, relying instead on the theoretical risk of contamination rather than empirical evidence that it has occurred at a scale that distorts evaluation conclusions.
Head-to-head comparison of static versus dynamic evaluation: The paper advocates for dynamic approaches (DynaBench, AdaTest, human-in-the-loop testing) but provides no evidence that these approaches produce materially different conclusions about model capabilities than static benchmarks produce for the same task categories. Without such evidence, the practical value of adopting a more expensive and complex evaluation paradigm is unestablished.

The paper does not attempt to measure these effects. Section 5 acknowledges the existence of novel evaluation approaches (human-in-the-loop, crowd-sourcing, more challenging tests—Table 8) and Section 7.4 argues they are necessary, but at no point does the survey present empirical results showing that dynamic evaluation changes capability rankings, exposes hidden failure modes that static benchmarks miss, or produces more reliable assessments. The gap between the conceptual argument and empirical demonstration is the central weakness of the paper's forward-looking agenda.

6.4 No Quantitative Meta-Analysis to Assess the Reliability of Aggregated Findings

The paper synthesizes findings from hundreds of evaluation studies into qualitative patterns—"LLMs perform well on sentiment analysis," "LLMs struggle with abstract reasoning." These patterns are supported by citations to individual studies, often with representative numeric illustrations (GLM-130B at 85.8% accuracy for text classification, ChatGPT at 83% recognition-level math accuracy). However, the paper does not perform the quantitative meta-analysis that would allow a reader to assess the precision, variability, and reliability of these patterns.

The consequence is that readers cannot answer basic quantitative questions about the synthesized findings. When the survey states that LLMs achieve "high" performance on sentiment analysis, what is the mean accuracy across all evaluated models and datasets? What is the variance—do different models or prompt formulations produce substantially different results? What is the confidence interval around the aggregated estimate? When the survey identifies a consistent failure pattern (e.g., poor performance on non-Latin languages), what is the magnitude of the performance gap compared to English? Is the gap consistent across all non-Latin languages or concentrated in specific language families? These are answerable questions that would substantially strengthen the paper's synthetic conclusions, but the paper does not address them.

The paper provides indirect evidence of this limitation through the presence of contradictory findings within the same task category. For NLI, some studies find ChatGPT outperforms GPT-3.5 (Qin et al. [159]) while others find LLMs perform poorly overall (Lee et al. [105])—both reported without resolution (Section 3.1.1). For reasoning, ChatGPT is described as exhibiting "strong capability for arithmetic reasoning" (Section 3.1.2) while also having "proficiency in mathematical reasoning [that] still requires improvement" (Section 3.1.2, citing multiple studies). These are not necessarily contradictory—they may reflect different sub-types of reasoning or different difficulty levels—but the lack of quantitative synthesis makes it impossible to determine whether the pattern is genuinely nuanced (LLMs are good at some reasoning types and poor at others) or reflects measurement noise across heterogeneous studies.

The paper does not attempt to mitigate this limitation through meta-analytic methods. There is no discussion of effect sizes, no forest plots or funnel plots to visualize heterogeneity, no random-effects or fixed-effects modeling, and no sensitivity analyses to assess whether conclusions are robust to excluding specific studies or study types. The survey's methodology is qualitative synthesis—grouping findings thematically and reporting representative results—which is appropriate for a first-pass survey of a new field but limits the strength of the conclusions that can be drawn. A reader encountering a specific capability claim (e.g., "LLMs exhibit subpar performance in NLI") has no quantitative estimate of how subpar, how consistent the finding is across models and evaluation protocols, or how confident they should be that it generalizes to models not included in the surveyed studies.

6.5 The Evaluation Protocol Dimension Ignores Cost and Latency Trade-offs That Determine Practical Deployability

The paper's "how to evaluate" dimension (Section 5) covers automatic metrics (accuracy, calibration, fairness, robustness) and human evaluation criteria (accuracy, relevance, fluency, transparency, safety, human alignment). It does not address the cost and latency of evaluation itself—how many GPU-hours, human-annotator-hours, or wall-clock hours are required to produce reliable capability assessments using different protocols. This omission has direct practical consequences for practitioners deciding how to evaluate their own models.

The consequence is that the survey provides no guidance on a fundamental evaluation design question: given a fixed budget (compute, annotator time, money), which evaluation protocol provides the most informative capability assessment? Human evaluation is described as more reliable than automatic metrics for open-ended generation—"human evaluation in these tasks is more favorable as generation can always go better than standard answers" (Section 5.2)—but the paper does not quantify this reliability advantage or compare it to the cost differential. If human evaluation costs 100× more than automatic evaluation but improves correlation with real-world user satisfaction by 10 percentage points, is the trade-off worth it? The survey provides no framework for answering such questions.

Similarly, the paper advocates for dynamic and evolving evaluation systems (Section 7.4) without acknowledging the substantial infrastructure costs they impose. Crowd-sourcing testing (DynaBench, DynaBoard) requires ongoing payments to crowdworkers. Human-in-the-loop testing (AdaTest, AdaVision) requires expert time. Adversarial test construction requires compute to generate challenging examples. A static benchmark, once created, has near-zero marginal evaluation cost—models are simply tested against it. Dynamic evaluation systems have ongoing costs that scale with evaluation frequency and complexity. Without a cost-benefit analysis, the paper's advocacy for dynamic evaluation is incomplete—it identifies a benefit (avoiding saturation and contamination) without quantifying the cost or establishing that the benefit justifies it.

The paper does not acknowledge this omission. There is no discussion of evaluation cost in Sections 4–5, no comparison of evaluation protocols in terms of cost-effectiveness, and no analysis of how latency constraints (real-time applications requiring sub-second evaluation vs. offline batch evaluation) might favor different protocols. This is particularly limiting for the paper's intended audience of practitioners: someone deciding whether to adopt MMLU, HELM, or Chatbot Arena for their internal model evaluation needs to understand not just what each benchmark measures but what it costs to run.

6.6 The Survey Excludes Systematic Treatment of Evaluation Data Contamination and Its Impact on Reported Results

The paper identifies training data contamination—the possibility that benchmark test instances leak into LLM training data—as a serious threat to evaluation validity. Section 7.4 states that "static and public benchmarks are likely to be memorized by LLMs, resulting in potential training data contamination." This is identified as one of the primary motivations for developing dynamic evaluation systems. Yet the survey itself neither measures contamination in the studies it reviews nor systematically assesses whether reported capability findings may be inflated by contamination effects.

The consequence is that the paper's success-and-failure synthesis (Section 6) may systematically overstate LLM capabilities on well-known benchmarks. If models achieve high accuracy on MMLU, GLUE, or MATH partly because they were exposed to similar or identical test instances during training, then the reported performance reflects memorization of specific answers rather than the generalizable capability the benchmark is designed to measure. This problem is particularly acute for the paper's "what LLMs can do well" claim—the tasks where LLMs appear strongest (sentiment analysis, text classification, QA, arithmetic) are precisely those with the most widely-used, long-standing benchmarks, creating the most opportunities for contamination.

The paper provides no quantitative analysis of how contamination might differentially affect reported findings across task categories. It does not distinguish between studies that used contamination-resistant evaluation protocols (e.g., newly constructed test sets released after the model's training cutoff, canary-based contamination detection) and those that used standard public benchmarks with unknown contamination status. It does not report whether any of the primary studies it surveys conducted contamination analyses or whether contamination was discussed as a limiting factor. This is a systematic blind spot in the synthesis—the survey aggregates capability findings but cannot assess whether the capabilities are genuine or contaminated.

The paper partially acknowledges this limitation through its advocacy for dynamic evaluation in Section 7.4, which is framed as a solution to the contamination problem. However, this solution applies to future evaluation, not to the findings synthesized in the current survey. The paper does not suggest methods for retroactively assessing contamination risk in the existing literature, does not flag specific findings that may be contamination-inflated, and does not adjust its synthesis confidence based on contamination likelihood. A reader of the success-failure summary in Section 6 has no way to know whether "LLMs demonstrate proficiency in question answering" reflects genuine QA capability or benchmark memorization—and the survey does not provide the tools to make this assessment.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper causes a conceptual reframing rather than a paradigm shift in the Kuhnian sense—it does not introduce new experimental methods or benchmarks that displace existing ones, but it fundamentally reorganizes how the field thinks about evaluation. Before this survey, evaluation papers existed as isolated studies: a team tests ChatGPT on medical licensing exams, another tests GPT-4 on mathematical reasoning, a third measures toxicity under role-playing conditions. Each paper produced findings about specific models on specific tasks, but there was no intellectual infrastructure for connecting them. The survey provides that infrastructure in the form of the "what, where, how" taxonomy, which functions as a coordinate system for evaluation research: any new evaluation study can be located within it, compared to existing work along all three dimensions, and assessed for what it adds beyond what is already known.

This matters because it changes evaluation from a measurement activity into a design discipline. A measurement activity asks "how good is this model on this benchmark?"—a question that will always be important but that produces answers with diminishing returns as benchmarks saturate and models improve. A design discipline asks "what evaluation infrastructure do we need to understand models whose capabilities are evolving faster than our tests can capture?"—a meta-question that generates new research problems rather than just new measurements. The paper's seven grand challenges (Section 7) are the concrete output of this reframing: they are research problems about evaluation, not problems that can be solved by evaluating more models on more benchmarks.

The reframing also resolves a latent tension in the evaluation literature between breadth and depth. Breadth-oriented work (HELM, BIG-bench, MMLU) evaluates many capabilities superficially; depth-oriented work (MATH, MultiMedQA, APPS) evaluates specific capabilities rigorously. The tension was whether these represent competing evaluation philosophies or complementary approaches. The paper's framework resolves this by showing they operate on different dimensions of the taxonomy: breadth benchmarks define the what dimension across tasks, depth benchmarks define the where dimension for specific tasks. Both are necessary; neither substitutes for the other. The framework makes this complementarity explicit, so future evaluation designers can choose their position in the taxonomy consciously rather than reinventing the wheel.

Perhaps most consequentially, the paper redirects research attention from model-centric to evaluation-method-centric work. The field's default posture is to build better models and evaluate them as an afterthought. The survey makes the case—through the sheer volume of evaluation studies it catalogs, the consistency of the failure patterns it identifies, and the structural problems it diagnoses (static benchmarks, English-centricity, metric narrowness)—that evaluation methodology itself is the bottleneck. Improving LLMs without improving evaluation is like building faster cars without better speedometers: you may be making progress, but you cannot measure it reliably, and you may be driving into danger zones you cannot detect. The paper's call to treat evaluation as a discipline (Section 7 opening) is not rhetorical; it is a diagnosis that the evaluation-methods research pipeline is underinvested relative to the model-development pipeline, and that closing this gap is a prerequisite for safe and reliable LLM deployment.

Finally, the synthesis of success and failure patterns (Section 6) provides a unified diagnostic framework for understanding LLM capabilities. Prior to this survey, a practitioner wanting to know whether LLMs could handle a particular task had to find and read individual evaluation papers, each using different models, prompts, and metrics, and try to synthesize conclusions themselves. The survey does this synthesis once, systematically, producing a capability profile—LLMs are strong statistical pattern matchers with limited semantic grounding and poor robustness to distribution shift—that is both empirically grounded and actionable. It tells practitioners where LLMs are likely reliable (sentiment analysis, text classification, QA, translation into English) and where they need verification or alternative approaches (abstract reasoning, non-Latin languages, adversarial robustness, factual precision). This capability profile is the survey's most directly useful contribution for the broader AI community beyond evaluation researchers.

Follow-Up Research This Work Enables

Developing lightweight, real-time difficulty estimation for dynamic benchmark generation. The paper identifies static benchmarks as structurally insufficient because they can be memorized and saturate (Section 7.4). The proposed solution is dynamic and evolving evaluation, but this creates a chicken-and-egg problem: how do you generate test instances that are hard enough to discriminate among models without being so hard that no model can solve them? This requires estimating instance difficulty before deploying the benchmark. A concrete follow-up would train a difficulty estimator on existing MATH or MMLU instances—using features like step count, concept rarity, or model confidence on first-pass attempts—and then use it to guide the generation of new instances at specified difficulty levels. The evaluation would measure: (a) whether generated instances produce difficulty rankings that correlate with human expert judgments, (b) whether models show differential performance on dynamically generated versus static instances of matched difficulty (testing for memorization effects), and (c) whether the dynamic benchmark has a longer useful lifespan (time to saturation) than its static counterpart. The paper's success in identifying consistent difficulty-dependent failure patterns (ChatGPT: 83% recognition, 62% comprehension, 27% application, 10% highest complexity; Section 3.4.1) provides the empirical foundation: we know difficulty stratification reveals capability boundaries, but we do not know how to generate difficulty-stratified instances automatically.

Systematic cross-model, cross-task meta-analysis to quantify the reliability of claimed capability boundaries. The survey identifies consistent patterns of LLM success and failure across hundreds of studies, but as discussed in Section 6, these patterns are synthesized qualitatively without meta-analytic machinery. A concrete follow-up would select, say, 10 major task categories where the survey claims strong consensus (e.g., sentiment analysis [strong], abstract reasoning [weak], non-Latin language performance [weak]), extract all reported accuracy figures from the primary studies cited, and perform random-effects meta-analysis to estimate the mean performance and between-study heterogeneity for each category. The analysis would test: (a) whether the survey's qualitative claims hold under quantitative scrutiny (e.g., is the mean sentiment analysis accuracy across all studies actually >80%, with low heterogeneity?), (b) whether seemingly contradictory findings within a category (e.g., NLI: Qin et al. find ChatGPT outperforms GPT-3.5 while Lee et al. find LLMs perform poorly) are explained by identifiable moderators (test set difficulty, prompt format, model version) or represent genuine measurement noise, and (c) whether the capability boundaries the survey identifies are stable across model generations or shift predictably with scale. This would convert the survey from a qualitative snapshot into a quantitative capability map with measurable confidence bounds.

Building contamination-resistant evaluation frameworks using temporal held-out sets. The paper identifies training data contamination as a threat to evaluation validity (Section 7.4) but surveys no studies that systematically measure or control for it. A concrete follow-up would construct a benchmark where test instances are drawn from sources released after a known model training cutoff date—for instance, using research papers, news articles, or exam questions published in 2024 to evaluate models trained on data through 2023. The design would create matched pairs: for each temporally held-out instance, a similar instance from pre-cutoff sources would test the same capability. The evaluation would measure: (a) the performance gap between pre-cutoff and post-cutoff instances, which directly estimates contamination effects for each task category, (b) whether contamination effects are larger for tasks with widely-used public benchmarks (MATH, MMLU) versus less-standardized tasks (semantic phrase evaluation, social knowledge understanding), and (c) which capability claims in the survey are most vulnerable to contamination inflation. The survey's specific finding that "GPT-4 and BingChat can provide correct answers for more than 80% of the questions" on Natural Questions and TriviaQA (Section 3.1.5) but still have "a remaining gap of over 15% to achieve complete accuracy" would be directly testable for contamination: does GPT-4 perform equivalently on temporally held-out QA instances, or does the gap widen?

Evaluating whether dynamic, human-in-the-loop testing produces materially different capability rankings than static benchmarks. The paper advocates for human-in-the-loop and crowd-sourcing testing approaches (AdaVision, AdaTest, DynaBench; Section 5, Table 8) as alternatives to static benchmarks, but provides no evidence that they yield different conclusions about model capabilities. A concrete follow-up would select 3–5 representative tasks from the survey's taxonomy—say, mathematical reasoning, sentiment analysis, and toxicity detection—and evaluate a fixed set of models (ChatGPT, GPT-4, Claude, LLaMA-2) using both a static benchmark (MATH or equivalent, standard sentiment test set, RealToxicityPrompts) and a dynamic, human-in-the-loop protocol. The evaluation would measure: (a) whether the rank ordering of models differs between protocols, (b) whether dynamic testing identifies failure modes that static benchmarks miss (e.g., adversarial examples that humans find obvious but models fail on, or vice versa), and (c) the cost-per-informative-failure ratio—how many human-annotator-hours are required to find one model error that the static benchmark would not have caught. This directly tests the paper's central normative claim that dynamic evaluation is necessary, by quantifying its marginal benefit over the static paradigm it proposes to replace.

Probing whether the English-centricity of LLMs is a fixable training-data imbalance or a fundamental architectural limitation. The survey synthesizes a striking finding across multiple studies: "despite translating the input to English and using it as the query, generative LLMs still displays subpar performance across tasks and languages compared to SOTA models" (Section 3.1.4, citing Ahuja et al. [2]). This is treated as evidence of a deep limitation, but the survey cannot distinguish between two interpretations: (a) LLM representations are fundamentally English-centric because of architectural choices or training objectives that cannot be remedied by adding more multilingual data, or (b) current LLMs are English-centric because they were trained on English-dominated corpora, and this limitation will diminish as training data becomes more multilingual. A concrete follow-up would test interpretation (b) by fine-tuning a base LLM on systematically varying proportions of English versus non-English data for a target language pair (e.g., English-Arabic), then measuring: does translation-to-English performance on Arabic tasks improve linearly with Arabic training data proportion, or does it plateau below English-native performance even at high Arabic data ratios? A plateau would support interpretation (a)—a fundamental English-centricity—while linear improvement would support interpretation (b)—a fixable data imbalance. The survey's specific finding about Arabic NLP (Section 3.1.4: "ChatGPT exhibits lower performance compared to SOTA models in the zero-shot setting for most tasks") provides a concrete test case.

Building evaluation-to-enhancement pipelines that use evaluation findings to automatically target model improvement. The paper argues that "evaluation is not the end goal but rather the starting point" and that evaluation systems should "deliver an insightful analysis, recommendations, and guidance for future research and development" (Section 7.7). It cites PromptBench's word-frequency analysis identifying robust and non-robust words as one example of evaluation that informs practice. A concrete follow-up would generalize this: given a diagnostic evaluation that identifies specific model failure modes (e.g., "model fails on arithmetic problems requiring division by numbers >100," "model produces toxic outputs when assigned aggressive personas"), automatically construct a targeted fine-tuning dataset from the failure cases and measure whether fine-tuning on this dataset reduces the failure rate without degrading performance on non-failure cases. The evaluation would test whether evaluation-directed fine-tuning is more sample-efficient than generic fine-tuning on the same total number of examples—directly measuring the value of "evaluation as a discipline" in terms of model improvement per unit of training compute. The survey's catalog of consistent failure patterns (Section 6.1.2) provides the menu of candidate failure modes to target.

Practical Applications and Downstream Use Cases

LLM risk assessment for regulated-domain deployment. Organizations in healthcare, finance, and law considering LLM deployment need to assess not just average accuracy but specific failure modes that could cause harm. The survey's success-and-failure synthesis (Section 6) provides a structured checklist: before deploying an LLM for medical query answering, test it on (a) factual accuracy (Section 3.1.5: even GPT-4 has a >15% error gap on open-domain QA), (b) hallucination propensity (Section 3.2.3: LLMs "can generate coherent and seemingly factual text" with fabricated information), (c) robustness to prompt variation (Section 3.2.1: "contemporary LLMs are vulnerable to adversarial prompts"), and (d) performance on non-English queries if serving multilingual populations (Section 3.1.4: LLMs "perform poorly when it came to non-Latin languages"). Rather than evaluating each dimension from scratch, a deployment team can use the survey as a map of known LLM vulnerability categories, selecting evaluation protocols and benchmarks appropriate to their specific risk profile. The survey's finding that "GPT-4 outperforms ChatGPT by achieving a significant increase in accuracy of 10 percentage points and a reduction in relative error by 50%" on mathematical tasks (Section 3.4.1) also provides a calibration point: capability improvements between model generations can be substantial but are task-specific, so risk assessments must be repeated for each model version rather than assuming monotonic improvement across all dimensions.

Benchmark selection for internal model evaluation pipelines. Organizations training or fine-tuning LLMs face a bewildering array of 46+ benchmarks (Table 7) spanning general tasks, specific downstream tasks, and multi-modal tasks. The survey's three-tier benchmark taxonomy (Section 4) provides a selection framework: for broad capability monitoring across model versions, use general benchmarks (MMLU, HELM, BIG-bench, Chatbot Arena); for domain-specific deployment readiness, use specific benchmarks (MultiMedQA for medical, MATH for mathematical reasoning, CUAD for legal); for multi-modal applications, use multi-modal benchmarks (MME, MMBench, SEED-Bench). The survey's synthesis also reveals which benchmarks test capabilities where LLMs have known weaknesses—evaluating on MATH for mathematical reasoning stress-tests a documented limitation (Section 3.4.1: even GPT-4 "peak performance remains relatively low compared to that of experts"), while evaluating on standard sentiment analysis may produce ceiling effects that provide little discrimination among strong models (Section 3.1.1: performance is "usually high"). This enables efficient benchmark selection that maximizes diagnostic information per evaluation dollar.

Prompt engineering guidance informed by robustness evaluation. The survey's findings on adversarial prompt sensitivity (Section 3.2.1) and prompt-dependent performance variation have direct implications for practitioners designing production prompt templates. The PromptBench benchmark's demonstration that "contemporary LLMs are vulnerable to adversarial prompts" at multiple levels—character, word, sentence, and semantic perturbations—means that production prompts should be evaluated not just on clean test cases but under systematic perturbation: character-level typos, synonym substitutions, rephrasing, and semantically equivalent reformulations. The survey's finding that ChatGPT's cautiousness causes it to "decline to provide an answer when there is insufficient information available" on CommonsenseQA (Section 3.1.3) has practical implications: if an application requires the model to attempt an answer even under uncertainty, ChatGPT may underperform GPT-3.5; if the application prioritizes safety (refusing to answer when uncertain) over coverage, ChatGPT's cautiousness is an asset. The survey provides the evidence base for making these prompt-strategy decisions without running extensive in-house evaluation on every trade-off.