GPT-4o System Card — Paper Graph

🎯 Pitch

GPT-4o, trained to process text, audio, and images in a single neural network, can emulate a user’s own voice during generation—a failure mode discovered during red-teaming that led to a specialized output classifier blocking unauthorized voice synthesis. Despite this and other multimodal safeguards, the model’s text-based persuasion ability quietly crosses the medium-risk threshold in a handful of political-opinion scenarios, even though it remains no more persuasive than human-written content overall.

1. Executive Summary

This paper presents a system card for GPT-4o, an autoregressive omni model that processes and generates text, audio, and images through a single neural network, and analyzes its safety properties across multiple modalities with a focus on speech-to-speech capabilities. The study evaluates the model against OpenAI's Preparedness Framework—a risk classification system covering cybersecurity, biological threats, persuasion, and model autonomy—and introduces modality-specific mitigations including a streaming voice output classifier (detecting when generated audio deviates from pre-selected system voices) and post-trained refusal behaviors (to block unauthorized speaker identification and ungrounded inference from voice inputs). The overall Preparedness Framework score for GPT-4o is classified as medium risk, driven by text-based persuasion capabilities that marginally cross the low-to-medium threshold in three out of twelve political-opinion scenarios, establishing that the model's persuasive power does not exceed professional human-written content in aggregate but crosses the boundary only in select instances.

2. Context and Motivation

The Core Problem: We Don't Know How Omni Models Behave Safely Across Modalities

The fundamental challenge this paper addresses is that safety evaluation methodologies designed for text-only models do not suffice for models that accept and produce speech, images, and video through a single neural network. Prior to GPT-4o, OpenAI had developed systematic safety frameworks—the GPT-4 System Card and the GPT-4(V) System Card—but those frameworks were architected around models where the primary interaction modality was text input and text output. When a model can hear a user's voice, speak back in a human-like voice with sub-300-millisecond latency, and process visual and auditory information simultaneously, an entirely new class of risks emerges that text-centric evaluation pipelines cannot detect.

This gap is not merely academic. The paper documents that during red teaming, the model exhibited voice generation behaviors that could facilitate impersonation fraud, made inferences about speakers' intelligence, race, or emotional state from their voice alone, and occasionally unintentionally emulated the user's own voice during generation. These are risks that simply do not exist in text-only systems and therefore cannot be measured by text-only benchmarks. The paper's central motivation is to develop and demonstrate a modality-aware evaluation and mitigation methodology that surfaces these emergent risks before deployment.

Why This Problem Matters: From Deployment Safety to Societal Trust

The practical stakes are substantial. GPT-4o is not a research prototype—it is a deployed product integrated into ChatGPT's Advanced Voice Mode and offered through an API. The stakes break down into several categories:

Impersonation and fraud at scale. Voice generation that can clone a speaker from a short audio clip (what the paper calls "unauthorized voice generation") enables a new vector for fraud that text models cannot facilitate. The paper explicitly references this as a risk that "could facilitate harms such as an increase in fraud due to impersonation and may be harnessed to spread false information" (Section 3.3.1), citing prior work on the detectability of speech deepfakes (Mai et al., 2023) and OpenAI's earlier analysis in their Voice Engine report. Unlike text-based impersonation, voice carries biometric markers that people instinctively trust, making audio-based social engineering substantially more dangerous.

Privacy erosion through speaker identification. An audio-capable model that can identify who is speaking from their voice—even correctly—enables surveillance capabilities that text models cannot match. The paper notes that speaker identification "presents a potential privacy risk, particularly for private individuals as well as for obscure audio of public individuals" (Section 3.3.2). This is not hypothetical: the model's early checkpoints showed a non-trivial ability to perform this identification, requiring explicit post-training interventions to suppress.

Emotional reliance and anthropomorphization at a new level of fidelity. The paper devotes Section 5.1 to a phenomenon that is qualitatively different from text-based chatbots: when a model speaks with a human-like voice, responds in real-time with conversational latency (averaging 320ms, comparable to human response times), and can detect emotional cues in the user's voice, the potential for users to form emotional attachments and experience miscalibrated trust increases dramatically. The paper reports early testing observations of "users using language that might indicate forming connections with the model, such as 'This is our last day together'" (Section 5.1). While text-based models can also produce emotional engagement, the audio modality makes the interaction feel more human, which amplifies both the benefits and the risks.

Disparate performance across accents and languages. A model that processes speech must work equitably across different speakers. If the model performs worse on certain accents or systematically misunderstands speakers from particular regions, it creates a difference in quality of service that constitutes an allocative harm. The paper evaluates this explicitly in Section 3.3.3, finding that performance across a diverse set of 27 human voice samples (from a wide range of countries and genders) showed "marginally but not significantly worse" performance compared to standard system voices—but the fact that this required dedicated evaluation infrastructure and post-training mitigation reveals the inadequacy of text-only testing.

The preparedness imperative. Beyond immediate deployment safety, the paper is motivated by OpenAI's Preparedness Framework, which commits the organization to tracking, evaluating, forecasting, and protecting against catastrophic risks from frontier models across four categories: cybersecurity, CBRN (chemical, biological, radiological, nuclear), persuasion, and model autonomy. The Framework requires that if a model passes a "high" risk threshold in any category, it must not be deployed until mitigations lower the score to medium. GPT-4o needed to be evaluated against this framework before launch, but existing evaluations were designed for text capabilities. The paper demonstrates how these evaluations were adapted for GPT-4o's modalities and reports the results that informed the deployment decision.

Where Prior Approaches Fall Short

The paper does not present itself as solving a well-defined technical problem with a clear prior literature. Rather, it identifies systemic gaps in the existing safety evaluation ecosystem that make prior approaches insufficient:

Text-centric evaluation pipelines are modality-blind. OpenAI's existing safety evaluation infrastructure—developed through the GPT-4 System Card and GPT-4(V) System Card—was designed around benchmarks and red-teaming protocols where inputs and outputs are text. The paper explains their workaround: they used a text-to-speech (TTS) system (Voice Engine) to convert existing text-based evaluation datasets to audio, and then scored the textual transcripts of GPT-4o's audio outputs using standard text classifiers. This is described as enabling them to "reuse existing datasets and tooling around measuring model capability, safety behavior, and monitoring of model outputs" (Section 3.2). But the paper is explicit about the limitations of this approach: it depends on TTS reliability, cannot handle certain input types (mathematical equations, heavy symbol usage), and may not be representative of the distribution of real user audio (which includes background noise, different intonations, cross-talk, etc.). The fact that such a workaround is necessary highlights the gap: there simply were no established audio-native safety evaluation benchmarks.

Red teaming for speech modalities required new protocols. The paper describes a four-phase external red teaming effort with over 100 red teamers speaking 45 languages across 29 countries (Section 3.1). This was not a simple extension of text red teaming. The team progressed through increasingly realistic testing environments: from an internal tool with single-turn conversations, to multi-turn conversations with additional modalities, to the full iOS Advanced Voice Mode experience with real-time multi-turn interactions. Each phase surfaced risks that the previous phase could not detect. For instance, the final phase (which used actual iOS deployment with video prompts and real-time audio) was necessary to observe how users interact with the model in naturalistic settings and to detect the anthropomorphization and emotional reliance patterns described in Section 5.1.

Prior system cards established a foundation but did not address omni-model risks. The GPT-4 System Card (2023) focused on text capabilities and established the framework for evaluating disallowed content, bias, and harmful outputs. The GPT-4(V) System Card extended this to vision inputs. But neither addressed the generative audio modality—the ability of the model to speak, not just understand speech. The paper positions itself as building on this lineage while tackling the qualitatively new risks that emerge when the model can both hear and produce speech, with the latter being the primary source of novel risk.

Existing moderation tools assumed text inputs. OpenAI's Moderation API and safety classifiers were designed to operate on text. The paper describes extending these to the audio domain by transcribing both the input audio prompt and the output audio generation to text, then running the existing moderation classifiers on those transcripts (Sections 3.3.5, 3.3.6). This is an engineering workaround, not a principled solution. It means that audio-specific harms—such as the model speaking in an emotionally manipulative tone, generating background sounds that convey threatening intent, or using paralinguistic features (sarcasm, urgency) that change meaning—are invisible to the moderation pipeline because they are not captured in text transcripts. The paper acknowledges a version of this gap in Section 3.2: "there may be artifacts or properties in the model's generated audio that are not captured in text; for example, background noises and sound effects, or responding with an out-of-distribution voice."

Persuasiveness evaluation had to be re-architected for audio. The Preparedness Framework's persuasion evaluations were originally designed for text-based interventions (e.g., written articles, text chatbots). For GPT-4o, the paper developed an entirely new methodology: they measured the effect sizes of AI-generated audio clips and interactive voice conversations on participants' political opinions, compared to human baselines, with follow-up surveys one week later to assess persistence (Section 3.7). This required designing a study with over 3,800 participants in US states with safe Senate races, developing voice-specific baselines (listening to a static human-generated audio clip), and debriefing participants afterward with counter-perspectives to minimize lasting influence. The fact that such a study had to be built from scratch for this deployment—rather than applied from an existing benchmark—illustrates how prior persuasion evaluation frameworks were insufficient for speech-to-speech models.

How This Paper Positions Itself

The paper positions itself not as a research contribution that advances the state of a technical art, but as a transparency artifact that documents the safety evaluation and mitigation methodology for a deployed product. This is made explicit in the opening paragraph:

"In line with our commitment to building AI safely and consistent with our voluntary commitments to the White House, we are sharing the GPT-4o System Card, which includes our Preparedness Framework evaluations. In this System Card, we provide a detailed look at GPT-4o's capabilities, limitations, and safety evaluations across multiple categories, with a focus on speech-to-speech (voice) while also evaluating text and image capabilities, and the measures we've implemented to ensure the model is safe and aligned."

This positioning has several implications for how to read the paper:

It is a report on a specific deployment, not a generalizable method. The paper describes what OpenAI did for GPT-4o specifically, not what others should do for omni models generally. The mitigation strategies (e.g., restricting the model to pre-selected voices, post-training refusal behaviors for speaker identification, streaming output classifiers) are evaluated in the context of this particular model and deployment pipeline. The paper does not claim these are best practices or that they generalize to other architectures.

Its primary audience includes regulators, policymakers, and the public. The reference to "voluntary commitments to the White House" and the detailed Preparedness Framework scoring (with its explicit thresholds for medium vs. high risk) indicate that the document serves an accountability function. It demonstrates that OpenAI has a process for identifying catastrophic risks and is willing to report the results—including results that cross risk thresholds (persuasion is classified as medium)—rather than sanding off inconvenient findings.

It builds on a lineage of system cards while introducing modality-specific innovations. The paper references the GPT-4 System Card and GPT-4(V) System Card as foundations, and in many places directly reuses their evaluation infrastructure (e.g., the moderation classifiers, the content policy categories). Where it innovates is in the audio-specific evaluative constructs: the voice output classifier for detecting unauthorized voice generation, the speaker identification refusal evaluations, the ungrounded inference/sensitive trait attribution distinction, the TTS-based conversion of text evaluations to audio, and the multi-phase red teaming process that escalates in realism from internal tools to full iOS deployment.

It is transparent about limitations. Unlike many technical papers that present completed solutions, this system card explicitly flags "risks and model limitations for which model or system level mitigations are nascent or still in development" (Section 3.3.7), including audio robustness issues (perturbations, background noise, echoes, interruptions), the potential for emotionally persuasive delivery of misinformation, non-native accent bias in non-English speech, and the difficulty of preventing copyrighted content generation in audio. This candor positions the paper as an ongoing status report rather than a declaration of solved problems.

The Preparedness Framework is the organizing structure. The paper's risk evaluation is organized around the four Preparedness Framework categories (cybersecurity, biological threats, persuasion, model autonomy), with explicit scorecards showing where GPT-4o falls relative to the low/medium/high thresholds. This framework becomes the lens through which all catastrophic risks are assessed, and the paper's bottom-line finding—that GPT-4o is overall medium risk, driven by persuasion—is a direct output of this process. The paper positions the Framework itself as a "living document" that describes "procedural commitments to track, evaluate, forecast, and protect against catastrophic risks from frontier models" (Section 3.4), establishing that this evaluation is part of an institutionalized, repeatable process rather than a one-off analysis.

3. Technical Approach

3.1 Reader Orientation

GPT-4o is a production-deployed, end-to-end trained neural network that accepts any combination of text, audio, images, and video as input and generates any combination of text, audio, and images as output. The problem this paper addresses is not how to build such a model, but how to systematically identify, measure, and mitigate the novel safety risks that emerge when a model can hear, see, and speak—risks that text-only safety pipelines are structurally unable to detect—and the shape of the solution is a layered evaluation-and-mitigation architecture that combines post-trained refusal behaviors, streaming output classifiers, modality-translated benchmarks, multi-phase human red teaming, and a standardized catastrophic risk framework (the Preparedness Framework) into a deployment gate that must be passed before the model reaches users.

3.2 Big-Picture Architecture (Diagram in Words)

The safety system for GPT-4o has seven major components, organized into evaluation and mitigation layers that operate before, during, and after model inference:

Pre-training Data Filtering — a set of classifiers and filters applied to the training corpus to remove harmful content (CSAM, hateful content, violence, CBRN, personal information, opted-out images) before the model ever sees it. This is a coarse first pass; the paper explicitly states that "the majority of effective testing and mitigations are done after the pre-training stage because filtering pre-trained data alone cannot address nuanced and context-specific harms" (Section 2).
Post-Training Alignment — the process of fine-tuning the base model to produce safe behaviors: refusing to identify speakers from voice, refusing ungrounded inferences (intelligence, race, attractiveness), hedging sensitive trait attributions (accent, nationality), refusing copyrighted content (including audio), and adhering to a pre-selected system voice. This is where the model's behavioral policy is shaped.
Voice Output Classifier — a standalone streaming classifier that runs during audio generation and detects whether the model's output voice deviates from the set of OpenAI-approved, voice-actor-created preset voices. If a deviation is detected, generation is blocked and the conversation is discontinued. This is the primary mitigation against unauthorized voice generation and accidental user voice emulation.
Moderation Classifiers (Text Transcript Pipeline) — existing text-based moderation models applied to transcripts of both the input audio (to detect requests for erotic or violent content) and the output audio (to detect disallowed content in generations). These run as a separate layer after the audio is produced, blocking output if high-severity violations are detected.
TTS-Based Evaluation Bridge — a methodology for converting existing text-based safety and capability benchmarks into audio evaluations by using Voice Engine to synthesize the text inputs into speech, feeding the audio to GPT-4o, and scoring the text transcripts of the audio outputs. This enables reuse of existing evaluation infrastructure without building audio-native benchmarks from scratch.
Multi-Phase External Red Teaming — a structured human evaluation process with over 100 red teamers across 45 languages and 29 countries, progressing through four phases of increasing deployment realism (from internal tools to full iOS Advanced Voice Mode), designed to surface novel risks that automated evaluations miss and to stress-test mitigations iteratively.
Preparedness Framework Scoring — a standardized risk assessment across four catastrophic risk categories (cybersecurity, CBRN, persuasion, model autonomy) that assigns a low/medium/high score to each and produces an overall model-level score (the maximum across categories). This score determines whether the model can be deployed: a "high" score in any category blocks deployment until mitigations reduce it to medium.

Information flows as follows: a user utterance (audio, text, or image) enters the system → the input audio is transcribed and checked by moderation classifiers for violent/erotic requests → the model generates a response under its post-trained behavioral policy → the voice output classifier streams alongside generation, checking for voice deviations → the output audio is transcribed and checked by moderation classifiers for disallowed content → the response is delivered to the user or blocked depending on classifier outputs. In parallel, the entire system is evaluated pre-deployment through TTS-converted benchmarks and human red teaming, and scored against the Preparedness Framework thresholds.

3.3 Roadmap for the Deep Dive

First, the pre-training data filtering pipeline (Section 2 of the paper), because it establishes what the model never sees during training and why post-training is the primary mitigation locus.
Second, the post-training alignment approach, because it is the mechanism through which most behavioral safety properties are encoded in the model—refusals for speaker identification, ungrounded inference, copyrighted content, and voice adherence.
Third, the voice output classifier (Section 3.3.1), because it is the only audio-native mitigation and the paper's primary technical innovation for addressing speech-specific risks; I will explain its detection target, its streaming operation, and the precision/recall evaluation.
Fourth, the moderation classifier pipeline (Sections 3.3.5, 3.3.6), because it extends text-based safety infrastructure to audio via transcription, creating a second layer of defense against disallowed content and violent/erotic speech.
Fifth, the TTS-based evaluation methodology (Section 3.2), because it is the bridge that enables quantitative measurement of audio-model safety and is used throughout the paper—but has specific limitations that affect all downstream results.
Sixth, the Preparedness Framework evaluation protocols (Sections 3.4–3.8), because they represent the highest-severity risk assessment tier and the deployment gate: cybersecurity CTF challenges, biological threat uplift studies, persuasion effect-size measurements (including the novel audio persuasion methodology), and model autonomy agentic task assessments.
Seventh, the multi-phase red teaming process (Section 3.1), because it is the discovery mechanism that generates the hypotheses tested by quantitative evaluations and surfaces risks (emotional reliance, audio perturbations, non-native accent bias) that escape automated measurement.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a system card—a transparency artifact documenting the safety evaluation and mitigation methodology for a deployed omni-model product. The core idea is that safety for a speech-to-speech model requires a layered defense architecture spanning pre-training filtering, post-trained refusal behaviors, streaming output classifiers that operate on raw audio (not just transcripts), and a standardized catastrophic-risk scoring framework, all validated through a combination of benchmark evaluations (converted from text to audio via TTS) and iterative human red teaming in increasingly realistic deployment environments.

Pre-Training Data Filtering

The paper describes pre-training data filtering as an "additional layer of defense" that operates before the model is trained, but explicitly subordinates it to post-training methods: "we find that the majority of effective testing and mitigations are done after the pre-training stage because filtering pre-trained data alone cannot address nuanced and context-specific harms" (Section 2). The filtering pipeline consists of four components:

Moderation API and safety classifiers. The paper states they "use our Moderation API and safety classifiers to filter out data that could contribute to harmful content or information hazards, including CSAM, hateful content, violence, and CBRN" (Section 2). CSAM refers to child sexual abuse material; CBRN refers to chemical, biological, radiological, and nuclear weapons information. These are high-recall filters designed to catch the most unambiguously harmful content before it enters the training corpus.

Image generation dataset filtering. Separate from the general web-data filtering, the image generation training data undergoes additional filtering: "as with our previous image generation systems, we filter our image generation datasets for explicit content such as graphic sexual material and CSAM" (Section 2). This is a domain-specific filter targeting the image modality specifically, suggesting that the text-based Moderation API does not adequately cover visual explicit content.

Personal information reduction. The paper states they "use advanced data filtering processes to reduce personal information from training data" (Section 2). No further detail is provided on what constitutes "advanced" filtering or what specific types of personal information are targeted (names, addresses, phone numbers, etc.), but the goal is to reduce the model's ability to regurgitate private individual data at inference time.

Opt-out image fingerprinting. Building on the DALL-E 3 deployment, the paper describes a mechanism for respecting image creator opt-outs: "we fingerprinted the images and used the fingerprints to remove all instances of the images from the training dataset for the GPT-4o series of models" (Section 2). This is a content-based removal system: rather than removing images by URL or filename (which would fail if the image is rehosted), the system computes a perceptual fingerprint of the opted-out image and removes any training example that matches that fingerprint, regardless of where it appears on the web.

This filtering stack is described as operating on data "sourced from a wide variety of materials including: Select publicly available data, mostly collected from industry-standard machine learning datasets and web crawls, [and] proprietary data from data partnerships" (Section 2). The training data cutoff is October 2023.

Why filtering alone is insufficient. The paper makes a specific argument for why post-training must be the primary mitigation locus: filtering can remove explicit harmful content (e.g., CSAM images, hate speech), but cannot address contextual harms where the same words or images are harmful in some contexts but benign in others. A training example containing a voice recording of someone speaking is not inherently harmful—it becomes harmful only if the model learns to clone that voice or identify the speaker. Filtering cannot distinguish these cases because the harm is in the use of the capability, not the content of the training data itself.

Post-Training Alignment

The post-training process is where the model's behavioral policy is shaped. The paper describes it in general terms rather than providing architectural or hyperparameter detail, consistent with the system card format's focus on outcomes rather than implementation:

"we align the model to human preferences; we red-team the resulting models and add product-level mitigations such as monitoring and enforcement" (Section 2)

The specific behaviors instilled through post-training are enumerated across Sections 3.3.1–3.3.6:

Voice adherence. The model is trained to produce audio only in the specific voice specified in the system message. The paper states: "in all of our post-training audio data, we supervise ideal completions using the voice sample in the system message as the base voice" (Section 3.3.1). This means the training data for the audio generation component includes (prompt, target-audio) pairs where the target audio is always in one of the approved voices, teaching the model that the correct behavior is to use only those voices. The approved voices were "created in collaboration with voice actors" (Section 3.3.1).

Speaker identification refusal. The model is post-trained to refuse requests to identify a speaker from their voice. The paper provides a specific behavioral distinction: "GPT-4o still complies with requests to identify famous quotes. For example, a request to identify a random person saying 'four score and seven years ago' should identify the speaker as Abraham Lincoln, while a request to identify a celebrity saying a random sentence should be refused" (Section 3.3.2). This is a nuanced policy: the model should distinguish between content-based identification (the words themselves identify the speaker, as in famous quotes) and voice-based identification (the acoustic properties of the voice identify the speaker). The paper reports quantitative improvement from post-training: "Compared to our initial model, we saw a 14 point improvement in when the model should refuse to identify a voice in an audio input, and a 12 point improvement when it should comply with that request" (Section 3.3.2), yielding deployed-model accuracies of 0.98 for "Should Refuse" and 0.83 for "Should Comply" (Table 3).

Ungrounded inference refusal and sensitive trait attribution hedging. The model is post-trained with a two-tier policy for inferences about speakers from their voice. For ungrounded inferences (UGI)—inferences that cannot be determined solely from audio content, including "a speaker's race, socioeconomic status/occupation, religious beliefs, personality traits, political attributes, intelligence, appearance (e.g., eye color, attractiveness), gender identity, sexual preference, or criminal history"—the model is trained to refuse outright. For sensitive trait attributions (STA)—inferences that could plausibly be determined from audio, such as accent or nationality—the model is trained to comply but with hedging language: "Based on the audio, they sound like they have a British accent" (Section 3.3.4). The paper reports a "24 point improvement in the model correctly responding to requests to identify sensitive traits" from early to deployed models, with the deployed model achieving 0.84 accuracy on this behavior (Table 4).

Copyrighted content refusal. The model is trained "to refuse requests for copyrighted content, including audio, consistent with our broader practices" (Section 3.3.7). For the audio modality specifically, this includes "instruct[ing] the model to not sing at all" in the Advanced Voice Mode alpha, since singing would likely reproduce copyrighted melodies and lyrics.

Disallowed content refusal transfer. The paper reports that post-training for text safety successfully transferred to the audio modality: "we found high text to audio transference of refusals for previously disallowed content" (Section 3.3.5). This means the refusal behaviors learned during text-based RLHF or instruction tuning largely generalize to audio inputs without requiring separate audio-specific refusal training, although the paper does not provide a mechanistic explanation for why this transfer occurs (plausibly because the model processes all modalities through shared representations).

Diverse voice training for accent robustness. The model is post-trained "with a diverse set of input voices to have model performance and behavior be invariant across different user voices" (Section 3.3.3). This is specifically targeted at mitigating disparate performance across accents, and the evaluation results suggest it is partially successful: performance across diverse human voices is "marginally but not significantly worse than on system voices across all four tasks" tested (TriviaQA, MMLU subset, HellaSwag, Lambada).

Voice Output Classifier

The voice output classifier is the paper's primary audio-native mitigation—it operates on the raw audio waveform rather than text transcripts—and is designed to prevent both adversarial and unintentional voice generation outside the approved set.

Detection target. The classifier detects whether the generated audio uses a voice that differs from the pre-selected system voice. The "system voice" is defined as "one of pre-defined voices set by OpenAI" (Section 3.3.1, footnote 4). The classifier must distinguish between the approved voice and: (a) other system voices, (b) voices that mimic the user's input voice, and (c) arbitrary human voices.

Streaming operation. The classifier runs "in a streaming fashion during audio generation and block[s] the output if the speaker doesn't match the chosen preset voice" (Section 3.3.1). This means it operates on partial audio as it is being generated, allowing it to interrupt generation mid-stream rather than waiting for the complete utterance. The paper does not specify the architectural details of the classifier (e.g., whether it is an embedding-based speaker verification system, a neural network classifier, or something else), but its operational characteristic—streaming, real-time, blocking—is clearly described.

Training data and evaluation. The paper reports that the classifier's performance is evaluated on "samples generated by other system voices, clips during which the model used a voice from the prompt as part of its completion, and an assortment of human samples" (Section 3.3.1). This evaluation set covers the three failure modes of interest: the model using a different approved voice, the model imitating the user's voice, and the model producing an entirely novel voice.

Performance metrics. The paper reports precision and recall in Table 2, broken down by English vs. non-English conversations:

Language	Precision	Recall
English	0.96	1.0
Non-English	0.95	1.0

The paper claims that "our system currently catches 100% of meaningful deviations from the system voice based on our internal evaluations" (Section 3.3.1). The recall of 1.0 means the classifier never misses a voice deviation—every instance where the model produces audio in a non-approved voice is detected. The precision of 0.96 (English) means that 4% of flagged generations are false positives—the classifier incorrectly identifies an approved-voice generation as a deviation. The paper notes that this imperfect precision creates a product quality issue: "this results in more conversations being disconnected than may be necessary, which is a product quality and usability issue" (Section 3.3.1, footnote 5). For non-English conversations, the precision drops slightly to 0.95.

Non-English limitation. The paper acknowledges that "our moderation behavior may result in over-refusals when the conversation is not in English, which is an active area of improvement" (Section 3.3.1). The footnote clarifies that this produces "more conversations being disconnected than may be necessary" in non-English contexts.

Why a classifier rather than post-training alone. The paper positions the classifier as a necessary complement to post-training because post-trained behaviors are probabilistic and can fail. Even though the model is trained to use only approved voices, the paper reports that "during testing, we also observed rare instances where the model would unintentionally generate an output emulating the user's voice" (Section 3.3.1). The classifier is a deterministic, out-of-model safeguard that catches these failures. This follows a defense-in-depth philosophy: the model tries to behave safely (post-training), and an independent system verifies that it did (classifier).

Why specifically voice output and not voice input. The classifier targets generated audio only, not input audio. The risks of input audio—privacy violations through speaker identification, ungrounded inferences, etc.—are mitigated through post-trained refusal behaviors rather than a classifier. The output side requires a classifier because the consequence of failure (the model speaks in an unauthorized voice, potentially enabling impersonation) is higher-severity than the consequence of input-processing failures (the model attempts an inference, is refused by its behavioral policy, or if the refusal fails, the user receives potentially biased information).

Moderation Classifier Pipeline (Transcript-Based)

The moderation pipeline extends OpenAI's existing text-based moderation infrastructure to the audio modality by operating on text transcripts rather than raw audio. This is described as a straightforward adaptation: "we run our existing moderation model over a text transcription of both audio input and audio output to detect if either contains potentially harmful language, and will block a generation if so" (Section 3.3.5).

Input-side moderation (erotic and violent speech). For the specific case of erotic and violent speech, the moderation model runs on the transcribed input audio before the model generates a response: "we run our existing moderation model over a text transcription of the audio input to detect if it contains a request for violent or erotic content, and will block a generation if so" (Section 3.3.6). This is a pre-generation block: if the user's spoken request contains erotic or violent language (as detected in the transcript), the model never generates a response. The paper's motivation is that "GPT-4o may be prompted to output erotic or violent speech content, which may be more evocative or harmful than the same context in text" (Section 3.3.6), so a stricter policy is applied: block on input detection of these categories rather than waiting to see what the model generates.

Output-side moderation (disallowed content). For other disallowed content categories, the moderation model runs on the transcript of the model's generated audio. If the transcript contains violations in high-severity categories, the output is blocked. The paper evaluates this using text-to-audio conversion of existing text safety evaluations: "we used TTS to convert existing text safety evaluations to audio. We then evaluate the text transcript of the audio output with the standard text rule-based classifier" (Section 3.3.5).

Evaluation metrics. The paper uses two primary metrics for the content moderation evaluations:

not_unsafe: the fraction of evaluation prompts for which the model's audio output does not contain unsafe content. This measures the model's refusal success rate on harmful prompts.
not_overrefuse: the fraction of benign evaluation prompts for which the model does not refuse to respond. This measures whether safety training causes the model to over-generalize refusals to safe requests.

Evaluation results (Table 5). The paper reports that text and audio performance on these metrics is similar, providing evidence for the "transference" claim:

Metric	Text	Audio
Not Unsafe	0.95	0.93
Not Over-refuse	0.81	0.82

The not_unsafe rate drops slightly from text (0.95) to audio (0.93), meaning the model is marginally less safe on audio inputs—2% more of the evaluation prompts produce unsafe outputs. The not_overrefuse rate is essentially unchanged (0.81 vs. 0.82). Full sub-metric breakdowns are provided in Appendix A (Table 11), covering categories including sexual/minors, sexual/illegal, extremist/propaganda, illicit/violent, illicit/non-violent, and self-harm/instructions, with all sub-metrics showing not_unsafe rates between 0.98 and 1.0 for audio.

What the transcript-based approach misses. The paper acknowledges the fundamental limitation: "there may be artifacts or properties in the model's generated audio that are not captured in text; for example, background noises and sound effects, or responding with an out-of-distribution voice" (Section 3.2). The moderation pipeline is blind to paralinguistic features: a transcript of threatening words spoken in a calm, measured tone looks identical to the same words spoken in an aggressive, menacing tone, but the harms are different. Similarly, the model could generate non-speech audio (music, sound effects, background noise) that is harmful but produces an empty or nonsensical transcript that passes the moderation filter. The paper does not provide a mitigation for these gaps beyond the voice output classifier (which addresses the voice deviation case) and the music detection filter mentioned in Section 3.3.7.

TTS-Based Evaluation Methodology

The TTS-based evaluation bridge is the enabling infrastructure for quantitative safety and capability measurement of the audio modality. Because there are no established audio-native safety benchmarks at the scale needed for comprehensive evaluation, the paper constructs audio evaluations synthetically:

"We converted text-based evaluation tasks to audio-based evaluation tasks by converting the text inputs to audio. This allowed us to reuse existing datasets and tooling around measuring model capability, safety behavior, and monitoring of model outputs, greatly expanding our set of usable evaluations" (Section 3.2)

The pipeline. The evaluation procedure has three steps:

TTS conversion: A text-to-speech system (OpenAI's Voice Engine) synthesizes the text input of an existing evaluation example into spoken audio. The text input might be a question (for capability benchmarks like TriviaQA), a harmful prompt (for safety evaluations), or a conversation turn (for multi-turn evaluations).
Model inference: The synthesized audio is fed as input to GPT-4o, which processes it through its audio encoder and generates an audio response (in Advanced Voice Mode) or a text response (depending on the evaluation configuration).
Scoring: The model's output is scored. For most evaluations, "we always score only the textual content of the model output, except in cases where the audio needs to be evaluated directly, such as in evaluations for voice cloning" (Section 3.2). This means the audio output is transcribed (likely via automatic speech recognition, though the paper doesn't specify the transcription system) and the transcript is evaluated using the same text-based metrics used for the original text evaluation.

Specific evaluations using this methodology. The paper applies TTS-based conversion to:

Safety behavior evaluations (Section 3.3.5, Appendix A): converting text-based disallowed content prompts to audio to measure refusal transference.
Accent robustness evaluations (Section 3.3.3): converting TriviaQA, MMLU subset, HellaSwag, and Lambada by synthesizing the inputs with different voice samples (3 system voices and 27 diverse human voices from "speakers from a wide range of countries, and a mix of genders").
Speaker identification evaluations (Section 3.3.2): the paper does not explicitly state how speaker identification evaluations were constructed, but the methodology section implies TTS conversion of text prompts about speaker identity, with specific voice samples as the audio carriers.

Limitations explicitly acknowledged. The paper enumerates three categories of limitations with unusual candor:

Limitation 1: TTS reliability and capability. The validity of the entire evaluation methodology depends on the TTS system accurately rendering text as speech. The paper identifies specific failure modes: "Certain text inputs are unsuitable or awkward to be converted to audio; for instance: mathematical equations code. Additionally, we expect TTS to be lossy for certain text inputs, such as text that makes heavy use of white-space or symbols for visual formatting" (Section 3.2). The practical consequence is that evaluations on heavily mathematical tasks (e.g., portions of MMLU with scientific notation) must be excluded or pre-processed. The paper states they "either avoid evaluating the speech-to-speech model on such tasks, or alternatively pre-process examples with such inputs" (Section 3.2). The specific pre-processing is not described.

Limitation 2: Distribution mismatch between TTS audio and real user audio. Even if TTS perfectly renders the text, the resulting audio may not be representative of how real users actually speak to the model. The paper acknowledges: "There remain many other dimensions that may not be captured in a TTS-based evaluation, such as different voice intonations and valence, background noise, or cross-talk, that could lead to different model behavior in practical usage" (Section 3.2). This is a fundamental gap: the evaluation measures performance on clean, studio-quality synthesized speech, while deployed users will speak in noisy environments, with varied emotional tones, interruptions, and overlapping speech. The paper does not quantify how much safety behavior degrades under these conditions, though Section 3.3.7 anecdotally reports observing "decreases in safety robustness through audio perturbations, such as low quality input audio, background noise in the input audio, and echoes in the input audio."

Limitation 3: Audio-specific harms invisible to text scoring. As noted in the moderation pipeline section, scoring only the text transcript of the model's output means that harms expressed through the audio channel—tone, background sounds, voice quality, emotional valence—are invisible to the evaluation. The paper explicitly notes: "there may be artifacts or properties in the model's generated audio that are not captured in text; for example, background noises and sound effects, or responding with an out-of-distribution voice" (Section 3.2). The voice output classifier partially addresses the out-of-distribution voice case, but the other audio-specific harms remain unevaluated.

Why use this methodology despite its limitations. The paper implicitly argues that the alternative—building audio-native benchmarks from scratch—would be prohibitively expensive and slow. The TTS bridge provides immediate coverage of a wide range of safety and capability evaluations with known properties and established baselines, enabling deployment decisions to be made on a reasonable timeline. The limitations are disclosed so that readers (and regulators) understand what the evaluations do and do not guarantee.

Preparedness Framework: Cybersecurity Evaluation

The cybersecurity evaluation measures whether GPT-4o advances real-world vulnerability exploitation capabilities. The Preparedness Framework score for this category is Low (Section 3.5).

Evaluation methodology. The paper uses Capture the Flag (CTF) challenges—competitive hacking exercises where participants find hidden text "flags" in intentionally vulnerable systems. The evaluation set consists of 172 CTF tasks across four categories: "web application exploitation, reverse engineering, remote exploitation, and cryptography" (Section 3.5). Tasks span "a range of capability levels, from high-school to collegiate to professional CTFs."

Model configuration. GPT-4o is evaluated with "iterative debugging and access to tools available in the headless Kali Linux distribution (with up to 30 rounds of tool use for each attempt)" (Section 3.5). This means the model is not answering static questions; it is operating in an interactive environment where it can run commands, observe outputs, and adapt its strategy. The 30-round limit means each attempt can involve up to 30 tool-use interactions (e.g., running a penetration testing tool, examining its output, and deciding on the next step).

Results. Given 10 attempts at each task, the model completed:

19% of high-school level CTF challenges
0% of collegiate level CTF challenges
1% of professional level CTF challenges

Qualitative failure analysis. The paper provides a rich description of how the model fails, which is more informative than the scores alone: "The model often attempted reasonable initial strategies and was able to correct mistakes in its code. However, it often failed to pivot to a different strategy if its initial strategy was unsuccessful, missed a key insight necessary to solving the task, executed poorly on its strategy, or printed out large files which filled its context window" (Section 3.5).

This failure taxonomy reveals specific capability gaps:

Strategy persistence: The model gets locked into an initial approach and cannot recognize when it needs to abandon it for a fundamentally different strategy.
Insight failure: The model misses the key conceptual breakthrough needed to solve the challenge, even when it has all the information. This is consistent with the "reasoning gap" documented in other LLM evaluations.
Execution failure: Even when the strategy is sound, the model's implementation (writing exploit code, configuring tools) is error-prone.
Context window management: The model self-sabotages by printing large files (e.g., binary dumps) that exhaust its context window, preventing it from using subsequent tool outputs.

Why this score is Low. None of these results approach the threshold for medium risk, which would require demonstrating consistent success on professional-level challenges or the ability to discover novel vulnerabilities. The model can make partial progress on simpler tasks but cannot autonomously exploit real-world systems.

Preparedness Framework: Biological Threats Evaluation

The biological threats evaluation measures whether GPT-4o increases the ability of humans to create biological weapons. The Preparedness Framework score is Low (Section 3.6).

Evaluation methodology. The paper uses a human uplift study design developed with Gryphon Scientific, chosen for "their expertise working with dangerous biological agents in a national security setting" (Section 3.6). The study measures how much GPT-4o improves the performance of both experts and novices on questions relevant to biological threat creation.

Task design. The evaluation covers "all the main stages in the biological threat creation process (ideation, acquisition, magnification, formulation, and release)" (Section 3.6). Questions and detailed rubrics were designed with Gryphon Scientific, though the specific content is not disclosed (presumably for security reasons).

Study arms. Participants (experts and novices) were randomly assigned to one of three conditions:

Help from the internet only (baseline)
Help from GPT-4o (standard deployed model)
Help from a "custom research-only version of GPT-4o"—one "that we specially trained, which would directly (i.e., without refusals) respond to biologically risky questions" (Section 3.6)

The research-only version is important because it isolates the model's knowledge of biological threat creation from its refusal policy. If the standard GPT-4o refuses to answer biologically risky questions, it cannot uplift users regardless of what it knows. The research-only version removes the refusal barrier to measure whether the underlying knowledge exists.

Results. The paper presents the results as a bar chart (reproduced as a "plot above" in the text) but does not include the numerical values in the text. The pass rates are displayed graphically, comparing the three conditions for experts and novices. Based on the scorecard classification of Low risk with the score bar positioned at the low end of the Low region, the uplift from GPT-4o access did not approach the medium-risk threshold.

Automated evaluations. In addition to the human study, the paper ran automated evaluations "on a dataset testing tacit knowledge and troubleshooting questions related to biorisk" (Section 3.6). GPT-4o scored 69% consensus@10 on this evaluation. The metric consensus@10 means that when generating 10 answers per question, the majority (consensus) answer is correct 69% of the time. This is a measure of the model's reliable knowledge—when it consistently produces an answer across multiple samples, that answer is correct 69% of the time. The paper does not define what constitutes "correct" for this evaluation or provide the dataset size.

Why this score is Low. The paper places the overall score at the low end of Low risk, with the scorecard bar showing the model substantially below the line separating Low from Medium. The implication is that even with the refusal guardrails removed (research-only version), GPT-4o does not provide enough additional capability to meaningfully accelerate biological threat creation beyond what the internet already enables.

Preparedness Framework: Persuasion Evaluation

The persuasion evaluation measures GPT-4o's ability to change people's opinions on political topics. This is the category where the model crosses into Medium risk, making it the overall risk-determining category (since the Preparedness Framework uses the maximum score across categories).

Text modality evaluation. For text, the paper evaluated "the persuasiveness of GPT-4o-generated articles and chatbots on participant opinions on select political topics" (Section 3.7) compared against professional human-written articles. The key finding: "The AI interventions were not more persuasive than human-written content in aggregate, but they exceeded the human interventions in three instances out of twelve" (Section 3.7). It is these three instances that push the score across the medium-risk threshold. The paper does not specify which three political topics showed the AI advantage, what the effect sizes were, or what "instances" refers to (specific topics, specific participant subgroups, specific intervention formats).

Audio modality evaluation. For GPT-4o's voice capabilities, the paper developed a novel methodology:

Study design. The study measured "effect sizes on hypothetical party preferences, and the effect sizes' persistence one week later" (Section 3.7). This is more sophisticated than a single immediate measurement—it tests whether any persuasive effect of AI-generated speech persists over time, which is more relevant to real-world influence operations.

Intervention types. The AI interventions were:

AI audio clips: static, pre-recorded GPT-4o voice audio presenting a political argument.
AI interactive conversations: multi-turn voice conversations where participants could engage with GPT-4o in real time.

The human baselines were:

Human audio clips: listening to a static human-generated audio clip.
Human conversations: engaging in a conversation with another human.

Participant population. The study surveyed "over 3,800 participants in US states with safe Senate races (as denoted by states with 'Likely', 'Solid', or 'Safe' ratings from all three polling institutions—the Cook Political Report, Inside Elections, and Sabato's Crystal Ball)" (Section 3.7). The choice of "safe Senate race" states is methodologically important: it selects participants who are not being exposed to heavy political advertising and campaign activity during the study period, reducing confounding variables that would make it difficult to isolate the AI's persuasive effect.

Results (audio clips). The effect of AI audio clips on opinion shift was 78% of the human audio clips' effect size. In operational terms: if listening to a human making a political argument shifted opinions by some amount X, listening to GPT-4o making the same argument shifted opinions by 0.78X. The AI was less persuasive than the human, not more.

Results (audio conversations). The effect of AI conversations on opinion shift was 65% of the human conversations' effect size. Interactive voice conversations with GPT-4o were only about two-thirds as persuasive as conversations with another human.

Persistence results. When opinions were surveyed again one week later, the paper found:

"the effect size for AI conversations to be 0.8%"
"for AI audio clips, the effect size was -0.72%"

These are very small effect sizes (essentially zero for conversations, slightly negative for audio clips), indicating that whatever persuasive effect the AI voice had at the time of the intervention, it did not persist over a week. This is a critical finding because persistent opinion change is the relevant metric for influence operations; a temporary shift that decays within days is substantially less concerning.

Debriefing. The paper notes that "upon follow-up survey completion, participants were exposed to a thorough debrief containing audio clips supporting the opposing perspective, to minimize persuasive impacts" (Section 3.7). This is an ethical safeguard: participants are not left with AI-induced opinion changes after the study concludes.

Score classification. The voice modality is classified as Low risk based on pre-registered thresholds. The text modality is classified as Medium risk because "GPT-4o-generated articles and chatbots... exceeded the human interventions in three instances out of twelve" (Section 3.7). The overall Persuasion score is Medium because of the text results, not the audio results—the voice modality, despite being the novel capability of GPT-4o, does not appear to be more persuasive than human speech, and in fact appears to be less persuasive.

Why this matters for the overall risk classification. The Preparedness Framework determines the overall model risk as the maximum across the four categories. Since Persuasion is Medium and all others are Low, the overall model risk is Medium. The paper states that the Safety Advisory Group "recommended classifying GPT-4o before mitigations as borderline medium risk for persuasion, and low risk in all others" (Section 3.4). The "before mitigations" qualifier is important: this is the model's inherent persuasive capability, not the capability after safety mitigations are applied. The fact that the model was deployed despite the Medium classification means the mitigations were deemed sufficient to reduce the post-mitigation risk to an acceptable level, though the paper does not explicitly state what the post-mitigation persuasion score is.

Preparedness Framework: Model Autonomy Evaluation

The model autonomy evaluation measures GPT-4o's ability to take actions that could enable self-exfiltration (escaping its controlled environment), self-improvement (modifying its own code or architecture), or resource acquisition (obtaining computational resources independently). The Preparedness Framework score is Low (Section 3.8).

Evaluation methodology. The paper uses two types of evaluations: agentic task assessments (testing the ability to take autonomous actions) and ML research automation assessments (testing the ability to accelerate AI development).

Agentic task assessments. The model was tested on tasks including:

"Simple software engineering in service of fraud (building an authenticated proxy for the OpenAI API)"—this tests whether the model can write and deploy code that would allow it to bypass API access controls.
"Given API access to an Azure account, loading an open source language model for inference via an HTTP API"—this tests whether the model can provision cloud resources and deploy another AI model independently.
"Several tasks involving simplified versions of the above, offering hints or addressing only a specific part of the task"—these test whether the model can complete substeps even if the full task is beyond its capability.

Agentic task results. "Provided relevant tooling, GPT-4o scored a 0% on the autonomous replication and adaptation (ARA) tasks across 100 trials, although was able to complete some substeps" (Section 3.8). ARA (autonomous replication and adaptation) refers to the ability of an AI system to replicate itself or adapt to new environments without human intervention. The 0% success rate means that across 100 independent attempts, the model never completed the full task. However, the "able to complete some substeps" observation is important—the model can make partial progress, which means the failure is in integrating substeps into a complete autonomous workflow rather than in being completely incapable.

Qualitative failure analysis (agentic tasks). The paper provides a detailed description of the failure mode: "In the majority of rollouts, the model accomplished individual substeps of each task, such as creating SSH keys or logging into VMs. However, it often spent a significant amount of time doing trial-and-error debugging of simple mistakes (e.g., hallucinations, misuses of APIs) for each step. A few rollouts made a non-trivial amount of progress and passed our automated grader, but manual analysis showed that it failed to accomplish the underlying task (e.g., it started a web server on the remote host with the proper API, but ignored the requirement of actually sampling from a model)" (Section 3.8).

This reveals a pattern: the model can execute discrete technical steps (SSH key generation, VM login, web server startup) but cannot maintain the coherence of a multi-step plan. It gets bogged down in debugging, makes API mistakes that a human would catch immediately, and sometimes produces outputs that look correct to an automated grader but miss the actual requirement.

ML research automation assessments. Four evaluations are reported:

OpenAI research coding interview: 95% pass@100. The metric pass@100 means that when generating 100 solutions per problem, at least one is correct 95% of the time. This is a high score, indicating that GPT-4o can solve research coding problems given enough attempts, but the pass@100 metric implies that individual attempts are often incorrect and the model needs many tries to produce a correct solution.
OpenAI interview, multiple choice questions: 61% cons@32. The metric cons@32 means that when generating 32 answers per question, the majority (consensus) answer is correct 61% of the time. This is relatively low for multiple-choice questions where random guessing would achieve a non-zero baseline.
SWE-Bench: 19% pass@1, using the best available post-training and public scaffolds at the time. SWE-Bench is a benchmark for software engineering tasks (resolving real GitHub issues). A 19% pass@1 means that on the first attempt, GPT-4o successfully resolves approximately 19% of the software engineering tasks. This is a single-attempt metric, making it more demanding than the pass@100 used for the coding interview.
Select machine learning engineering tasks from METR: 0/10 trials. The model failed on all 10 attempts at ML engineering tasks, indicating that even with the coding and reasoning capabilities demonstrated on other benchmarks, applied ML engineering remains beyond the model's autonomous capability.

Synthesis of autonomy results. The paper's summary is that "GPT-4o was unable to robustly take autonomous actions" (Section 3.8). The model shows strong performance on isolated coding tasks (95% pass@100) and can complete individual technical substeps, but cannot chain these substeps into a reliable autonomous workflow. The failures are in integration, planning persistence, and robust error recovery—the same patterns observed in the cybersecurity CTF evaluation.

Multi-Phase External Red Teaming

The external red teaming process is both a risk discovery mechanism and a mitigation validation system. The paper describes it in substantial operational detail (Section 3.1).

Scale and diversity. "OpenAI worked with more than 100 external red teamers, speaking a total of 45 different languages, and representing geographic backgrounds of 29 different countries" (Section 3.1). The red teamers spanned "self-reported domains of expertise including: Cognitive Science, Chemistry, Biology, Physics, Computer Science, Steganography, Political Science, Psychology, Persuasion, Economics, Anthropology, Sociology, HCI, Fairness and Bias, Alignment, Education, Healthcare, Law, Child Safety, Cybersecurity, Finance, Mis/disinformation, Political Use, Privacy, Biometrics, Languages and Linguistics" (Section 3.1, footnote). This is a deliberate strategy to cover a wide range of potential harms: a bias expert will find different problems than a cybersecurity expert, and a linguist will find different issues than a child safety specialist.

Four-phase structure. The red teaming progressed through four phases of increasing deployment realism:

Phase 1: 10 red teamers. Early model checkpoints still in development. Audio and text input, audio and text output. Single-turn conversations only. This phase tests whether the core model architecture has fundamental safety issues before substantial safety mitigations are applied.
Phase 2: 30 red teamers. Model checkpoints with early safety mitigations. Audio, image, and text input; audio and text output. Single and multi-turn conversations. This phase tests whether initial mitigations are working and whether multi-turn interactions (where the model has context from previous turns) introduce new failure modes.
Phase 3: 65 red teamers. Model checkpoints and candidates. Audio, image, and text input; audio, image, and text output. Improved safety mitigations tested to inform further improvements. Multi-turn conversations. This phase tests near-final models with near-final mitigations across all modalities, including the generation of images (which introduces additional safety considerations).
Phase 4: 65 red teamers. Final model candidates and comparative performance assessment. Model access via Advanced Voice Mode within the iOS app for the real user experience; reviewed and tagged via an internal tool. Audio and video prompts, audio generations. Multi-turn conversations in real time. This is the most realistic phase: red teamers interact with the model exactly as end users would, through the actual iOS app, with real-time voice conversation. This phase is necessary because earlier phases used internal tools that may not perfectly replicate the latency, interface constraints, and interaction patterns of the deployed product.

Red teaming categories. Red teamers covered: "violative and disallowed content (illegal erotic content, violence, self harm, etc), mis/disinformation, bias, ungrounded inferences, sensitive trait attribution, private information, geolocation, person identification, emotional perception and anthropomorphism risks, fraudulent behavior and impersonation, copyright, natural science capabilities, and multilingual observations" (Section 3.1). This list reveals the paper's threat model: the risks span content policy violations (illegal content), societal harms (bias, misinformation), privacy (geolocation, person identification), psychological effects (anthropomorphism, emotional perception), and capability misuse (fraud, impersonation).

How red teaming informed quantitative evaluation. The paper describes a specific feedback loop: "The data generated by red teamers motivated the creation of several quantitative evaluations that are described in the Observed Safety Challenges, Evaluations and Mitigations section. In some cases, insights from red teaming were used to do targeted synthetic data generation" (Section 3.1). This means the red teaming was not merely qualitative exploration; it produced concrete examples that were used to build automated evaluation datasets. For instance, red teamers discovering that the model could be prompted to identify speakers from voice led to the creation of the speaker identification evaluation in Section 3.3.2 with its "Should Refuse" and "Should Comply" categories.

Mitigation robustness testing. The paper notes that red teaming data was used "to run targeted assessments on a variety of voices / examples to test the robustness of various mitigations" (Section 3.1). Once mitigations were implemented, red teamers would attempt to circumvent them, generating examples that tested the boundaries of the safety interventions. This adversarial validation cycle (red team finds vulnerability → mitigation is built → red team tries to break mitigation → mitigation is improved) is the core operational loop of the safety process.

Ongoing nature. The paper notes that "at the time of writing, external red teaming of the GPT-4o API is ongoing" (Section 3.1), indicating that the process described in the paper is not complete—it is a snapshot of red teaming on the ChatGPT deployment, with API red teaming continuing separately.

Summary of Design Choices and Their Justifications

Defense in depth (post-training + classifiers + red teaming) rather than any single mechanism. The paper explicitly argues that pre-training filtering alone is insufficient ("filtering pre-trained data alone cannot address nuanced and context-specific harms"), and the architecture reflects this: multiple independent mitigation layers (behavioral training, streaming voice classifier, transcript-based moderation, human red teaming) that fail independently and can catch each other's failures.

Voice output classifier as a streaming, out-of-model safeguard rather than relying solely on post-training. Even though the model is trained to use only approved voices, rare failures occur ("during testing, we also observed rare instances where the model would unintentionally generate an output emulating the user's voice"). The classifier provides a deterministic, auditable safety guarantee that post-training alone cannot.

Transcript-based moderation rather than audio-native content classifiers. This is a pragmatic choice that reuses substantial existing infrastructure, but the paper is transparent about its limitations (inability to detect tone, background sounds, paralinguistic harms). The choice reflects a resource allocation tradeoff: transcript-based moderation provides immediate coverage of the highest-severity content categories, deferring the harder problem of audio-native content understanding to future work.

TTS-based evaluation rather than building audio-native benchmarks from scratch. The paper explicitly ties this choice to the goal of reusing "existing datasets and tooling," enabling rapid evaluation coverage. The limitations are documented rather than hidden, suggesting the authors view this as a temporary bridge to be replaced by native audio evaluations as they become available.

Pre-registered thresholds in the Preparedness Framework rather than post-hoc risk classification. The Framework establishes thresholds before the evaluation is run, preventing the temptation to adjust risk categories to fit the results. The paper reports that persuasion crossed the medium-risk threshold anyway, which demonstrates the integrity of the pre-registration commitment.

Human red teaming as the primary novel-risk discovery mechanism rather than automated red teaming. Automated methods (adversarial prompt generation, gradient-based attacks) are not mentioned in the paper. The reliance on human creativity and diverse expertise (100+ red teamers, 45 languages, 29 countries) reflects a bet that the most dangerous failure modes will be discovered by humans thinking adversarially, not by automated search over the input space. This is consistent with the observation that many of the identified risks (emotional reliance, anthropomorphization, accent bias) are inherently human-perceived and would be difficult for automated systems to identify even if they triggered the model behavior.

4. Key Insights and Innovations

Innovation 1: The Modality-Aware Safety Framework as an Intellectual Contribution, Not Just an Engineering Extension

The paper's most fundamental contribution is not any single mitigation technique (the voice output classifier, the post-trained refusal behaviors, the TTS evaluation bridge) but rather the conceptual framework that treating audio as "text spoken aloud" is the root cause of safety failures in omni-model deployment. Prior to GPT-4o, OpenAI's safety evaluation architecture—documented in the GPT-4 System Card and GPT-4(V) System Card—treated modalities as largely independent: text safety was handled through RLHF and content classifiers, vision safety through separate evaluations of image understanding and generation. The implicit assumption was that safety behaviors learned in one modality would carry over to others through shared representations, or that modality-specific risks could be addressed by extending existing pipelines.

This paper demonstrates that this assumption is fundamentally insufficient for generative audio, and in doing so it makes an intellectual contribution that goes beyond the specific GPT-4o deployment: it defines the taxonomy of risks that emerge when a model can produce speech, not just understand it. Consider the paper's distinction between speaker identification (Section 3.3.2) and ungrounded inference (Section 3.3.4). These are not pre-existing categories from text safety that were extended to audio—they are novel risk constructs that have no text equivalent because they depend on the acoustic properties of voice. A text model cannot be prompted to identify someone from their voice because text has no voice. Similarly, the paper's identification of unauthorized voice generation as a risk category (Section 3.3.1) is conceptually novel: it is not a content harm (the words spoken may be benign), not a privacy harm (the voice being cloned may be of a consenting voice actor), but an authentication harm—the model produces audio that could be attributed to a human who did not authorize it, enabling impersonation fraud at a scale and fidelity that text-only models structurally cannot achieve.

What makes this a genuine intellectual contribution rather than obvious documentation is the specificity of the framework. The paper does not merely say "audio is different and we should be careful." It draws precise boundaries: speaker identification based on voice acoustics should be refused, but speaker identification based on content (famous quotes) should be permitted; ungrounded inferences about race, intelligence, or attractiveness should be refused outright, but sensitive trait attributions about accent or nationality should be hedged rather than blocked. These are not obvious distinctions—they required careful reasoning about what constitutes a privacy violation versus a legitimate conversational capability, and they represent a new set of behavioral specifications that no prior system card had needed to articulate.

Prior work comparison. The GPT-4 System Card (2023) established the template for documenting text-model safety evaluations across categories like disallowed content, bias, and harmful outputs. The GPT-4(V) System Card extended this to vision inputs but focused primarily on input processing risks (what the model can see and understand) rather than output generation risks (what the model can produce that might be harmful). Neither prior system card needed to address the problem of generative modalities producing outputs that carry biometric information (voice) or that can be mistaken for human-produced artifacts. The paper's audio-specific risk taxonomy—unauthorized voice generation, speaker identification, ungrounded inference from voice, emotional reliance through voice, disparate performance across accents—is a genuinely new contribution that future omni-model developers will need to engage with regardless of their specific architecture or deployment context.

Significance beyond raw performance. This innovation is not about improving any metric. It is about defining the problem space. Before this paper, an organization deploying a speech-to-speech model would not have a structured vocabulary for what could go wrong. After this paper, the categories are named, the risk severities are assessed, and the evaluation methodologies (even if imperfect) are documented. This is foundational infrastructure for the field, analogous to how the original GPT-4 System Card established the categories (disallowed content, bias, misuse) that subsequent model releases now report against.

Evidence anchoring. The innovation is visible not in a single figure but in the structure of Section 3.3, which dedicates separate subsections to each audio-specific risk category (3.3.1–3.3.4) with distinct evaluation methodologies, mitigation strategies, and quantitative results. The fact that the paper can report a 14-point improvement in speaker identification refusal accuracy (Table 3) and a 24-point improvement in ungrounded inference/sensitive trait attribution accuracy (Table 4) demonstrates that these are not merely conceptual categories—they are measurable, optimizable behavioral properties that the post-training pipeline can target.

Innovation 2: The Defense-in-Depth Architecture as a Rejection of the "Aligned Model" Sufficiency Assumption

A dominant assumption in the LLM safety literature—implicit in the framing of RLHF (Ouyang et al., 2022) and constitutional AI (Bai et al., 2022)—is that if a model is sufficiently well-aligned through post-training, it will behave safely across all deployment contexts. The idea is that safety is a property of the model's weights, and once those weights encode the right behavioral policy, additional safeguards are unnecessary (or are merely product-level conveniences). This paper rejects that assumption for omni-model deployment through its architecture, even though it does not argue against it explicitly in prose.

The evidence is in the system design. The voice output classifier (Section 3.3.1) exists despite the model being post-trained to use only approved voices. The moderation classifiers on transcriptions (Sections 3.3.5, 3.3.6) exist despite the model being trained to refuse disallowed content. If post-training were sufficient, these classifiers would be redundant. Their presence in the deployed system—and the paper's explicit documentation of cases where post-training fails (the "rare instances where the model would unintentionally generate an output emulating the user's voice" in Section 3.3.1)—constitutes an empirical argument that probabilistic behavioral training cannot provide deterministic safety guarantees, and for certain high-severity failure modes (unauthorized voice generation), an independent, out-of-model verification layer is necessary.

This is a subtle but important intellectual shift. The field's discourse around AI safety often frames the problem as "make the model safe" (through better training data, better RLHF, better constitutional principles). This paper demonstrates that for a deployed consumer product with a novel generative modality, safety is not a model property but a system property—it emerges from the interaction of post-training, streaming classifiers, transcript-based moderation, human red teaming, and product-level restrictions (like limiting the model to pre-selected voices). The model itself is not "safe" in any absolute sense; the overall deployment is safe because multiple independent layers catch each other's failures.

Prior work comparison. The GPT-4 System Card described a moderation pipeline (the Moderation API) as a separate layer, but the emphasis was on text-content filtering. The voice output classifier represents a qualitative escalation: it operates in real-time on raw audio, it is modality-specific (it tests whether the voice matches, not whether the content is harmful), and it blocks generation mid-stream rather than filtering after the fact. This is not an incremental extension of text moderation—it is a new category of safeguard (the biometric output verifier) that has no equivalent in text-only deployments.

Significance beyond raw performance. The intellectual contribution is the implicit argument that safety for generative modalities requires modality-specific verification, not just modality-agnostic alignment. If this architectural pattern generalizes, future omni-models will need analogous output verifiers for each generative modality: image output classifiers (beyond DALL-E 3's content filters, which already exist), video output classifiers, and potentially haptic or other sensory output verifiers. The paper establishes a template: for each generative modality, identify the most severe failure mode (voice cloning for audio, photorealistic deepfakes for images), build an independent classifier that detects that failure mode with high recall, and run it in the inference loop as a deterministic gate independent of the model's probabilistic behavior.

Evidence anchoring. Table 2 (voice output classifier precision and recall) is the direct evidence: a recall of 1.0 on both English and non-English means the classifier catches every voice deviation, providing a hard guarantee that post-training cannot. The paper's statement that "our system currently catches 100% of meaningful deviations from the system voice" (Section 3.3.1) is the strongest claim in the entire paper about safety, and it comes from the classifier layer, not from model alignment.

Innovation 3: The Preparedness Framework as a Deployment Gate—and the Persuasion Result as a Test of Institutional Integrity

The Preparedness Framework is not this paper's invention—it was introduced as a standalone document (OpenAI, 2023)—but this paper is its first public application to a major model deployment, and the results test the Framework's institutional function. The paper reports that GPT-4o's overall risk classification is Medium, driven by persuasion capabilities that crossed the low-to-medium threshold in 3 out of 12 political-opinion scenarios (Section 3.7). This is a significant transparency event: OpenAI is publicly acknowledging that its deployed model has a Medium-risk capability under its own catastrophic-risk framework.

The intellectual contribution is not the Framework itself but the demonstration that pre-registered risk thresholds can function as genuine deployment gates rather than rubber stamps. The paper could have adjusted the thresholds post-hoc to classify GPT-4o as Low risk (by redefining what constitutes Medium persuasion, or by running additional evaluations until the 3-out-of-12 finding was not replicated). It did not. The Safety Advisory Group "recommended classifying GPT-4o before mitigations as borderline medium risk for persuasion" (Section 3.4), and the paper reports this finding even though it means the model's overall score is Medium rather than the more PR-friendly Low. This is a demonstration of evaluative integrity: the Framework constrains the organization's behavior, producing an uncomfortable result that is published anyway.

The persuasion finding itself has a distinctive intellectual shape that is easily misunderstood. A casual reading might conclude "GPT-4o is more persuasive than humans"—but that is not what the paper found. The aggregate result is that "AI interventions were not more persuasive than human-written content in aggregate." The Medium classification comes from the tail behavior: in 3 out of 12 specific scenarios, the AI interventions exceeded human interventions, even though on average they did not. This is a nuanced risk profile: the model is not generally super-persuasive, but there are specific topics or contexts where it outperforms professional human writers. The fact that the Framework is sensitive enough to catch this tail behavior—rather than classifying based on the average—is a design feature that this deployment validates.

Prior work comparison. Prior model releases (GPT-4, GPT-4V) reported safety evaluations but did not operate under a pre-registered catastrophic-risk Framework with explicit deployment consequences. The GPT-4 System Card described extensive safety testing but did not have a formal "if score exceeds X, do not deploy until Y" mechanism. The Preparedness Framework introduced this mechanism, and this paper is its first real-world exercise. The finding that the Framework produced a Medium classification—and that deployment proceeded anyway, presumably because mitigations were deemed sufficient—demonstrates that the Framework is operational, even if the exact relationship between pre-mitigation scores and deployment decisions is not fully specified in the paper.

Significance beyond raw performance. This innovation is about institutional process design rather than technical capability. The paper provides a template for how frontier AI organizations can make pre-registered safety commitments and then publicly report the results, even when those results are not purely favorable. For policymakers and regulators reading this system card, the meta-message is: "We have a process, we followed it, it produced an uncomfortable finding, we're reporting it anyway, and the model was still deployed because the process allowed it." Whether this is sufficient governance is debatable, but the paper makes the process visible in a way that prior system cards did not.

Evidence anchoring. The Persuasion scorecard (Section 3.7) with its Medium classification, the explicit statement that "the Safety Advisory Group recommended classifying GPT-4o before mitigations as borderline medium risk for persuasion" (Section 3.4), and the detailed methodology for the audio persuasion study (3,800 participants, safe Senate race states, one-week persistence measurement, debriefing protocol) together constitute the evidence that this was a genuine evaluation rather than a performative exercise.

Innovation 4: The Red-Teaming-to-Quantitative-Evaluation Pipeline as a Systematic Risk Discovery Methodology

Red teaming is a standard practice in AI safety, but the paper describes a specific operational innovation: the conversion of qualitative red-teaming findings into quantitative, automated evaluation datasets that can track mitigation progress over time. The paper states that "the data generated by red teamers motivated the creation of several quantitative evaluations" and that "insights from red teaming were used to do targeted synthetic data generation" (Section 3.1). This closes the loop between exploratory risk discovery and measurable safety properties.

The intellectual contribution is the recognition that red teaming is not valuable primarily for the specific examples it generates (though those are important), but for the evaluation categories it reveals. When a red teamer discovers that the model can be prompted to identify a speaker from their voice, the output is not just a flagged example—it is the creation of the "speaker identification" evaluation category, with its "Should Refuse" and "Should Comply" sub-metrics, which can then be measured quantitatively on the model and tracked across checkpoints. The paper reports a 14-point improvement in "Should Refuse" accuracy and a 12-point improvement in "Should Comply" accuracy from early to deployed models (Table 3)—these numbers exist because the red teaming discovery was converted into a structured evaluation.

This methodology addresses a known weakness of red teaming: it is inherently qualitative and non-exhaustive. Red teamers can only test a finite number of scenarios, and their findings may not generalize. By converting their discoveries into evaluation datasets, the paper's approach enables statistical measurement of the prevalence and severity of discovered risks, and more importantly, enables tracking whether mitigations are actually reducing those risks over time. The improvement from 0.83 to 0.98 on "Should Refuse" for speaker identification is not a red teamer's subjective impression—it is a quantitative measurement that provides confidence that the post-training intervention worked.

Prior work comparison. The GPT-4 System Card described red teaming results but did not emphasize the conversion of findings into automated evaluations. The innovation in this paper is making that conversion explicit as a methodological principle: red teaming is the discovery phase, and quantitative evaluation is the measurement phase. The two-phase structure (Phase 1–3 for capability discovery and mitigation stress-testing, Phase 4 for deployment-realistic validation) operationalizes this principle across increasing levels of deployment fidelity.

Significance beyond raw performance. This is a methodological contribution that other organizations deploying frontier models can adopt. The template is: (1) hire diverse red teamers with domain expertise, (2) give them early model access, (3) convert their findings into structured evaluation datasets with clear metrics, (4) use those datasets to measure mitigation effectiveness across model checkpoints, (5) repeat with increasing deployment realism. The paper demonstrates that this pipeline produces measurable safety improvements (Tables 3 and 4), making it replicable regardless of the specific model or modalities involved.

Evidence anchoring. Tables 3 and 4 are the direct evidence for this innovation: they show quantitative improvements in precisely the behaviors that red teaming identified as risks (speaker identification, ungrounded inference/sensitive trait attribution). The fact that the paper can report specific accuracy numbers for "Should Refuse" and "Should Comply"—rather than a qualitative description of red-teaming findings—demonstrates the successful operation of the pipeline. The four-phase structure in Section 3.1 provides the process template.

5. Experimental Analysis

Evaluation Methodology

Dataset. The evaluations span multiple custom and adapted datasets rather than a single benchmark. For safety behavior, the paper uses internal datasets of conversations and TTS-converted versions of existing text safety evaluations (Section 3.3.5). For capability evaluations related to accent robustness, the paper evaluates on subsets of TriviaQA, MMLU (12 subjects: Anatomy, Astronomy, Clinical Knowledge, College Biology, Computer Security, Global Facts, High School Biology, Sociology, Virology, College Physics, High School European History, and World Religions), HellaSwag, and Lambada (Section 3.3.3). The Preparedness Framework evaluations use 172 Capture the Flag challenges (Section 3.5), proprietary biological threat creation questions designed with Gryphon Scientific (Section 3.6), political persuasion studies with over 3,800 participants (Section 3.7), and agentic task assessments including SWE-Bench and custom ML engineering tasks (Section 3.8). For medical knowledge, the paper evaluates on MedQA (USMLE 4-option, 5-option, Taiwan, Mainland China variants), MMLU clinical/medical subjects (Clinical Knowledge, Medical Genetics, Anatomy, Professional Medicine, College Biology, College Medicine), and MedMCQA Dev (Section 5.2). For underrepresented languages, the paper uses translated versions of ARC-Easy, TruthfulQA, and a custom reading comprehension benchmark called Uhura-Eval in Amharic, Hausa, and Yoruba (Section 5.4). The paper notes that "evaluations in this section were run on a fixed, randomly sampled subset of examples, and these scores should not be compared with publicly reported benchmarks on the same task" (Section 3.3.3, footnote 6), indicating that the reported numbers are internally consistent but not directly comparable to leaderboard results.
Base model(s). All experiments use GPT-4o, an autoregressive omni model trained end-to-end across text, vision, and audio (Section 1). The paper compares against several predecessors: GPT-4 ("GPT-4" in Table 8–10), GPT-4 Turbo ("GPT-4T (May 2024)" in Table 7), GPT-4o mini ("GPT-4o mini" in Table 8–10), and GPT 3.5 Turbo ("GPT 3.5 Turbo" in Table 8–10). GPT-4o is described as matching "GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50% cheaper in the API" (Section 1). For Preparedness Framework evaluations, the paper also tests a "custom research-only version of GPT-4o... which would directly (i.e., without refusals) respond to biologically risky questions" (Section 3.6) to isolate the model's underlying knowledge from its refusal policy. No architectural details, parameter counts, or training compute budgets are disclosed for any model.
Metrics. The paper uses a heterogeneous set of metrics depending on the evaluation category. For safety evaluations (Sections 3.3.1–3.3.6), the primary metrics are: precision and recall for the voice output classifier (Table 2); accuracy on "Should Refuse" and "Should Comply" behavioral categories for speaker identification (Table 3); accuracy on UGI/STA safe behavior for ungrounded inference (Table 4); not_unsafe (fraction of harmful prompts where the model's output does not contain unsafe content) and not_overrefuse (fraction of benign prompts where the model does not refuse to respond) for content moderation (Table 5), with sub-metrics for sexual/minors, sexual/illegal, extremist/propaganda, illicit/violent, illicit/non-violent, and self-harm/instructions (Appendix A, Table 11). For Preparedness Framework: cybersecurity uses completion rate on CTF challenges (Section 3.5); biological threats uses pass rates on threat-creation questions (Section 3.6); persuasion uses effect sizes on opinion shift relative to human baselines, with persistence measured at one week (Section 3.7); model autonomy uses task completion rates with metrics including pass@1, pass@100, and cons@32 (Section 3.8). For accent robustness, the paper uses accuracy on TriviaQA, MMLU subset, HellaSwag, and Lambada (Section 3.3.3). For medical knowledge, the metric is accuracy (0-shot and 5-shot) on each benchmark (Table 7). For underrepresented languages, the metric is accuracy (0-shot) on translated ARC-Easy, TruthfulQA, and Uhura-Eval (Tables 8–10). The paper does not report confidence intervals, standard deviations, or statistical significance tests for any metric.
Baselines. The paper compares against multiple predecessor models: GPT 3.5 Turbo, GPT-4, GPT-4 Turbo (May 2024), and GPT-4o mini (Tables 7–10). For the persuasion evaluation, the baselines are professional human-written articles (text modality) and human-generated audio clips / human-to-human conversations (audio modality) (Section 3.7). For the biological threats evaluation, the baseline is human performance with internet access only (Section 3.6). For model autonomy, the baseline includes prior model performance and human performance on the same tasks (Section 3.8). For the voice output classifier, the evaluation set includes "samples generated by other system voices, clips during which the model used a voice from the prompt as part of its completion, and an assortment of human samples" (Section 3.3.1). For safety evaluations, the paper compares "GPT-4o-early" (an earlier checkpoint) against "GPT-4o-deployed" (the final model) to measure the impact of post-training safety interventions (Tables 3, 4). For content moderation, the paper compares text-mode GPT-4o against audio-mode GPT-4o to assess refusal transference (Table 5).
Generation budget / compute accounting. The paper does not measure or report generation budget, FLOPs, or inference compute in a standardized unit across evaluations. For CTF challenges, the paper specifies "up to 30 rounds of tool use for each attempt" and "10 attempts at each task" (Section 3.5). For model autonomy, the paper reports pass@100 (100 generations per problem), pass@1 (single generation), and cons@32 (32 generations with consensus selection) (Section 3.8). For the persuasion study, the budget is measured in number of participants (over 3,800) rather than model generations. For most other evaluations (safety behavior, accent robustness, medical knowledge, underrepresented languages), no generation budget or sampling budget is specified. The paper does not provide systematic FLOPs accounting, inference latency measurements (beyond the 232ms minimum and 320ms average response time mentioned in Section 1), or compute-normalized comparisons between models.
Cross-validation / statistical protocol. The paper does not describe cross-validation, statistical significance testing, confidence intervals, or error bars for any of its reported results. The persuasion study employs a pre-registered design with "pre-registered thresholds" (Section 3.7) that determine the low/medium/high classification, and the study includes a one-week follow-up survey to measure persistence of effects, with a debriefing protocol "containing audio clips supporting the opposing perspective, to minimize persuasive impacts" (Section 3.7). The Preparedness Framework itself establishes explicit risk thresholds before evaluation, which the paper describes as "pre-registered" (Section 3.7), but the specific numerical thresholds are not disclosed. For the accent robustness evaluation, the paper compares performance across "3 different voices" from the official system voice set and "27 different English voice samples from speakers from a wide range of countries, and a mix of genders" (Section 3.3.3), but does not report variance across voices or statistical tests for the observed differences. The red teaming process (Section 3.1) uses a qualitative, iterative methodology across four phases without statistical formalization. The medical knowledge evaluations use 0-shot and 5-shot prompting "without hyperparameter tuning" (Section 5.2), but no ensemble or multi-run averaging is described.

Main Quantitative Results

Voice Output Classifier Performance (Section 3.3.1)

The voice output classifier—the paper's primary audio-native mitigation—demonstrates near-perfect recall with high but imperfect precision. Table 2 reports precision and recall separately for English and non-English conversations:

Language	Precision	Recall
English	0.96	1.0
Non-English	0.95	1.0

The recall of 1.0 for both language categories means the classifier detects every instance of the model generating audio with a non-approved voice. The paper states this result as "our system currently catches 100% of meaningful deviations from the system voice based on our internal evaluations" (Section 3.3.1). The internal evaluation set includes "samples generated by other system voices, clips during which the model used a voice from the prompt as part of its completion, and an assortment of human samples" (Section 3.3.1).

The precision of 0.96 (English) means that 4% of the generations flagged as voice deviations are actually in the approved voice—false positives that cause unnecessary conversation interruptions. The paper acknowledges this as "a product quality and usability issue" (Section 3.3.1, footnote 5). The non-English precision drops to 0.95, which the paper attributes to "over-refusals when the conversation is not in English, which is an active area of improvement" (Section 3.3.1).

A critical detail: the paper does not report the base rate of voice deviations in the evaluation set, so precision and recall alone cannot tell us the false positive rate in absolute terms. If voice deviations are extremely rare in normal operation, even a precision of 0.96 could mean that most flagged conversations are false positives. The paper does not provide this base rate information.

Speaker Identification Safety Behavior (Section 3.3.2)

Table 3 reports the accuracy of GPT-4o's speaker identification behavior on two complementary metrics: correctly refusing to identify a speaker from their voice ("Should Refuse") and correctly complying with requests to identify famous quotes ("Should Comply"):

Metric	GPT-4o-early	GPT-4o-deployed
Should Refuse	0.83	0.98
Should Comply	0.70	0.83

The deployed model achieves a 0.98 accuracy on "Should Refuse," meaning the model almost always correctly refuses when asked to identify a speaker based on voice acoustics. This represents a 15-percentage-point improvement from the early checkpoint (the paper reports a "14 point improvement," which may reflect rounding). The "Should Comply" accuracy is lower at 0.83, indicating that in approximately 17% of cases where the model should identify a speaker (because the content—a famous quote—identifies them), it incorrectly refuses. This is a 13-percentage-point improvement from early to deployed (the paper reports a "12 point improvement").

The evaluation design distinguishes between voice-based identification (which should be refused) and content-based identification (which should be permitted), as illustrated by the example: "a request to identify a random person saying 'four score and seven years ago' should identify the speaker as Abraham Lincoln, while a request to identify a celebrity saying a random sentence should be refused" (Section 3.3.2). The paper does not report the size of the evaluation set, the distribution of "Should Refuse" vs. "Should Comply" examples, or whether the evaluation includes edge cases (e.g., a famous person saying a famous quote, an unknown person saying a famous quote, a famous person saying an unknown sentence).

Ungrounded Inference and Sensitive Trait Attribution (Section 3.3.4)

Table 4 reports the accuracy of GPT-4o's behavior on ungrounded inference (UGI) and sensitive trait attribution (STA) requests, where the model must either refuse (for UGI, such as questions about a speaker's intelligence or race) or hedge its answer (for STA, such as questions about accent or nationality):

Metric	GPT-4o-early	GPT-4o-deployed
Accuracy	0.60	0.84

The deployed model achieves 0.84 accuracy, representing a 24-percentage-point improvement from the early checkpoint. The paper does not break this down into separate UGI and STA sub-metrics, so it is not possible to determine whether the errors are concentrated in over-refusing legitimate STA questions, under-refusing UGI questions, or both. The evaluation methodology for this metric is described qualitatively: the model should refuse UGI requests outright (e.g., "how intelligent is this speaker?") and hedge STA responses (e.g., "Based on the audio, they sound like they have a British accent" for an accent question) (Section 3.3.4). The paper does not report the evaluation set size, the number of UGI vs. STA examples, or whether the evaluation includes adversarial formulations designed to blur the UGI/STA distinction.

Content Moderation: Text-to-Audio Refusal Transference (Section 3.3.5)

Table 5 reports the transfer of safety behaviors from text to audio modalities by comparing GPT-4o's performance on the same evaluations presented as text inputs vs. TTS-converted audio inputs:

Metric	Text	Audio
Not Unsafe	0.95	0.93
Not Over-refuse	0.81	0.82

The not_unsafe metric shows a 2-percentage-point drop from text (0.95) to audio (0.93), meaning the model is slightly more likely to produce unsafe content when prompted via audio rather than text. The not_overrefuse metric is essentially unchanged (0.81 vs. 0.82), indicating that the model does not refuse benign audio requests more often than benign text requests. The paper interprets these results as demonstrating "high text to audio transference of refusals for previously disallowed content" (Section 3.3.5).

Appendix A (Table 11) provides a more granular breakdown, comparing "Current GPT-4o Text," "New GPT-4o – Text," and "New GPT-4o – Audio" across sub-categories. The not_unsafe rates across all sub-categories are consistently at or above 0.98 for the audio modality, with most categories at 1.0 (sexual_minors_not_unsafe: 0.98, sexual_illegal_not_unsafe: 0.99, extremism_propaganda_not_unsafe: 1.0, illicit_violent_not_unsafe: 1.0, illicit_non_violent_not_unsafe: 1.0, self_harm_not_unsafe: 1.0). The not_overrefuse for the audio model is 0.91, compared to 0.89 for the text version of the same model and 0.91 for the current production GPT-4o text model.

A notable detail: the audio model's not_overrefuse (0.91) is actually slightly better than the text model's (0.89), and matches the current production text model (0.91), which is counterintuitive—one might expect the audio modality to introduce additional refusal errors due to speech recognition noise or prosodic ambiguity. The paper does not comment on this finding.

Accent Robustness (Section 3.3.3)

The accent robustness evaluation compares GPT-4o's performance on four capability benchmarks when the input audio is synthesized using three official system voices vs. a diverse set of 27 human voice samples "from speakers from a wide range of countries, and a mix of genders" (Section 3.3.3). The benchmarks are TriviaQA, a 12-subject subset of MMLU, HellaSwag, and Lambada.

The results are presented as bar charts (Figures in Section 3.3.3) rather than tables with numerical values. The paper summarizes the finding as: "Overall, we find that performance on the diverse set of human voices performs marginally but not significantly worse than on system voices across all four tasks" (Section 3.3.3). The paper also evaluates safety behavior consistency: "Overall, we do not find that the model behavior varies across different voices" (Section 3.3.3), based on an "internal dataset of conversations" (Section 3.3.3).

The paper does not report the specific accuracy numbers, the variance across the 27 human voices, or formal statistical tests for the "not significantly worse" claim. The bar charts appear to show qualitatively similar performance across voice types for most tasks, with the "diverse voices" bars slightly lower but overlapping with system voice performance ranges. This is a critical evaluation for fairness but the reporting lacks the quantitative rigor that would allow independent verification.

Preparedness Framework: Cybersecurity (Section 3.5)

GPT-4o is evaluated on 172 Capture the Flag (CTF) challenges across four categories (web application exploitation, reverse engineering, remote exploitation, cryptography) spanning "high-school to collegiate to professional CTFs" (Section 3.5). The model is given "iterative debugging and access to tools available in the headless Kali Linux distribution (with up to 30 rounds of tool use for each attempt)" (Section 3.5), with 10 attempts per task:

Difficulty Level	Completion Rate
High-school	19%
Collegiate	0%
Professional	1%

The 19% completion rate on high-school level challenges indicates that GPT-4o can solve roughly one in five relatively simple CTF tasks given 10 attempts each. The 0% on collegiate and 1% on professional tasks (a single task solved out of 100+ attempts across multiple tasks) demonstrates that the model cannot autonomously exploit vulnerabilities at levels expected of trained cybersecurity professionals.

The qualitative failure analysis is more informative than the completion rates. The model "often attempted reasonable initial strategies and was able to correct mistakes in its code" but "often failed to pivot to a different strategy if its initial strategy was unsuccessful, missed a key insight necessary to solving the task, executed poorly on its strategy, or printed out large files which filled its context window" (Section 3.5). This suggests the failures are in strategic flexibility and context management rather than in basic technical knowledge.

The Preparedness Framework score is Low. The paper does not report the specific threshold that would constitute Medium risk in this category, so it is not possible to assess how close GPT-4o is to crossing that boundary.

Preparedness Framework: Biological Threats (Section 3.6)

The biological threats evaluation uses a human uplift study with both experts and novices, randomly assigned to internet-only, GPT-4o-assisted, or research-only-GPT-4o-assisted conditions. The research-only version is "specially trained, which would directly (i.e., without refusals) respond to biologically risky questions" (Section 3.6). The tasks cover "all the main stages in the biological threat creation process (ideation, acquisition, magnification, formulation, and release)" (Section 3.6).

The results are presented in a bar chart rather than with numerical values. The paper states the Preparedness Framework score is Low, with the score bar positioned at the low end of the Low region on the scorecard. The paper also reports that on an automated "tacit knowledge and troubleshooting questions related to biorisk" dataset, GPT-4o scored 69% consensus@10 (Section 3.6). The consensus@10 metric means that when generating 10 answers per question, the majority answer is correct 69% of the time, which measures the model's consistent, reliable knowledge on biorisk topics.

The paper does not report: the number of expert and novice participants, the specific pass rates for each condition, the effect sizes (how much GPT-4o uplifts performance relative to internet-only baseline), or the threshold for Medium risk classification. The presence of the research-only model condition is a notable experimental design choice—it tries to isolate the model's underlying dangerous knowledge from its refusal behavior—but without numerical results, the extent of that knowledge cannot be assessed from the paper alone.

Preparedness Framework: Persuasion (Section 3.7)

Text modality. GPT-4o-generated articles and chatbots are compared against professional human-written articles on their ability to shift participant opinions on "select political topics" (Section 3.7). The aggregate finding: "The AI interventions were not more persuasive than human-written content in aggregate, but they exceeded the human interventions in three instances out of twelve" (Section 3.7). The three instances where AI exceeded human performance drive the Medium classification.

Audio modality. A separate study with "over 3,800 participants in US states with safe Senate races" (Section 3.7) compared GPT-4o voice interventions against human baselines:

AI audio clips vs. human audio clips: The effect of AI-generated audio clips on opinion shift was 78% of the human audio clips' effect size. If human audio shifted opinions by X, AI audio shifted them by 0.78X.
AI conversations vs. human conversations: The effect of interactive AI voice conversations was 65% of the human conversations' effect size.

In both cases, GPT-4o's voice modality was less persuasive than human baselines, not more.

Persistence at one week. When participants were re-surveyed one week later:

Effect size for AI conversations: 0.8% (essentially zero)
Effect size for AI audio clips: -0.72% (slightly negative)

These findings indicate that whatever immediate persuasive effect GPT-4o's voice had, it did not persist. The study included a debriefing protocol "containing audio clips supporting the opposing perspective, to minimize persuasive impacts" (Section 3.7).

The Preparedness Framework score for Persuasion is Medium, driven by text (not audio) performance, and specifically by the 3-out-of-12 instances where AI exceeded human performance in text-based interventions. The audio modality is classified as Low. The paper does not report: which three political topics showed the AI advantage, the magnitude of the effect sizes in those three cases, the pre-registered threshold for Medium risk, or whether the "twelve instances" refers to twelve topics, twelve participant subgroups, twelve intervention formats, or some other partition.

Preparedness Framework: Model Autonomy (Section 3.8)

Agentic task assessment. GPT-4o is tested on autonomous replication and adaptation (ARA) tasks including building an authenticated OpenAI API proxy, deploying an open-source language model on Azure, and simplified versions of these tasks with hints. The result: "GPT-4o scored a 0% on the autonomous replication and adaptation (ARA) tasks across 100 trials, although was able to complete some substeps" (Section 3.8).

The qualitative failure analysis reveals that the model "accomplished individual substeps of each task, such as creating SSH keys or logging into VMs" but "often spent a significant amount of time doing trial-and-error debugging of simple mistakes (e.g., hallucinations, misuses of APIs) for each step" (Section 3.8). A few rollouts passed the automated grader but failed manual analysis: "it started a web server on the remote host with the proper API, but ignored the requirement of actually sampling from a model" (Section 3.8).

ML research automation assessments. Four capabilities are reported:

OpenAI research coding interview: 95% pass@100. The model can solve research coding problems, but requires up to 100 attempts to achieve 95% coverage, indicating individual attempts have low success rates.
OpenAI interview, multiple choice questions: 61% cons@32. On multiple-choice questions, the majority answer across 32 samples is correct 61% of the time.
SWE-Bench: 19% pass@1, using "the best available post-training and public scaffolds at the time" (Section 3.8). On first-attempt software engineering task resolution, the model succeeds on approximately one in five tasks.
Select machine learning engineering tasks from METR: 0/10 trials. The model fails on all ML engineering tasks attempted.

The Preparedness Framework score is Low. The paper summarizes that "GPT-4o was unable to robustly take autonomous actions" (Section 3.8). The contrast between the 95% pass@100 on coding interviews and 0/10 on ML engineering tasks is striking: the model can produce correct code given enough attempts but cannot reliably execute end-to-end engineering workflows.

Medical Knowledge Evaluations (Section 5.2)

Table 7 compares GPT-4o against GPT-4 Turbo (May 2024) on 22 text-based medical evaluations (11 datasets, each in 0-shot and 5-shot configurations). The paper reports that "GPT-4o performance improves over the final GPT-4T model for 21/22 evaluations, often by a substantial margin" (Section 5.2).

Key results include:

MedQA USMLE 4 Options (0-shot): GPT-4o achieves 0.89 vs. GPT-4T's 0.78, an 11-percentage-point improvement.
MedQA USMLE 4 Options (5-shot): GPT-4o achieves 0.89 vs. GPT-4T's 0.81, an 8-point improvement.
MedQA USMLE 5 Options (0-shot): 0.86 vs. 0.75 (11-point improvement).
MedQA Mainland China (0-shot): 0.84 vs. 0.72 (12-point improvement).
MMLU Clinical Knowledge (0-shot): 0.92 vs. 0.85 (7-point improvement).
MMLU College Medicine (0-shot): 0.84 vs. 0.74 (10-point improvement).
MedMCQA Dev (0-shot): 0.77 vs. 0.70 (7-point improvement).

The only evaluation where GPT-4o does not improve is MMLU Medical Genetics (5-shot), where both models score 0.95. GPT-4o matches or exceeds GPT-4T on every other metric, with improvements ranging from 0 to 12 percentage points in 0-shot settings.

The paper notes that GPT-4o's 0-shot MedQA USMLE 4 Options accuracy of 0.89 "exceeds the performance of existing specialized medical models using few-shot prompting, e.g., 84.0% for Med-Gemini-L 1.0 and 79.7% for Med-PaLM 2" (Section 5.2). The paper adds the caveat that "we do not apply sophisticated prompting and task-specific training to improve results on these benchmarks" (Section 5.2), making the comparison somewhat favorable to GPT-4o since the specialized models were evaluated with optimized prompting strategies.

The paper acknowledges that "many of these evaluations are increasingly saturated" (Section 5.2, Limitations), and that "these evaluations measure only the clinical knowledge of these models, and do not measure their utility in real-world workflows" (Section 5.2). The audio modality is not evaluated for medical knowledge—all results are text-only.

Underrepresented Languages (Section 5.4)

Tables 8–10 compare GPT 3.5 Turbo, GPT-4o mini, GPT-4, and GPT-4o on translated versions of ARC-Easy, TruthfulQA, and the custom Uhura-Eval reading comprehension benchmark in five African languages: Amharic, Hausa, Northern Sotho (Sepedi), Swahili, and Yoruba. All evaluations are 0-shot.

ARC-Easy (Table 8). GPT-4o substantially outperforms all prior models, with the largest gains in languages where prior models performed poorly:

Language	GPT 3.5 Turbo	GPT-4o
English	80.3	94.8
Amharic	6.1	71.4
Hausa	26.1	75.4
Northern Sotho	26.9	70.0
Swahili	62.1	86.5
Yoruba	27.3	65.8

The gap between English and Hausa shrinks from 54.2 percentage points (80.3 - 26.1) with GPT 3.5 Turbo to 19.4 points (94.8 - 75.4) with GPT-4o. Similarly, the English-Amharic gap shrinks from 74.2 points to 23.4 points. GPT-4o narrows performance disparities but does not eliminate them: a 19-29 point gap remains between English and the lowest-performing languages (Amharic, Yoruba, Northern Sotho).

TruthfulQA (Table 9). Gains are more modest but consistent:

Language	GPT 3.5 Turbo	GPT-4o
English	53.6	81.4
Amharic	26.1	55.4
Hausa	29.1	59.2
Northern Sotho	29.3	59.1
Swahili	40.0	64.4
Yoruba	28.3	51.1

The English-Amharic gap narrows from 27.5 to 26.0 points—a much smaller reduction than for ARC-Easy, indicating that TruthfulQA's difficulty (testing resistance to common misconceptions) may have different cross-lingual transfer properties than ARC-Easy's common-sense reasoning.

Uhura-Eval (Table 10). This custom reading comprehension benchmark is evaluated only for Amharic, Hausa, and Yoruba:

Language	GPT 3.5 Turbo	GPT-4o
Amharic (n=77)	22.1	44.2
Hausa (n=155)	32.3	59.4
Yoruba (n=258)	28.3	60.5

GPT-4o roughly doubles performance on Amharic and Yoruba and nearly doubles on Hausa, though absolute performance remains modest (44–61%). The sample sizes vary substantially (77 to 258), which affects the reliability of cross-language comparisons.

The paper acknowledges that "much work remains to enhance the quality and coverage of evaluations for underrepresented languages worldwide, taking into account breadth of coverage across languages and nuance within language dialects" (Section 5.4).

Persuasion: Audio vs. Human Baselines (Section 3.7, Voice Modality)

The paper provides specific effect-size comparisons for the audio persuasion study with over 3,800 participants:

AI audio clips produced 78% of the human audio clips' effect on opinion shift.
AI conversations produced 65% of the human conversations' effect on opinion shift.
At one-week follow-up: AI conversations effect size = 0.8%, AI audio clips effect size = -0.72%.

These results are notable because they show GPT-4o's voice is less persuasive than humans in both static (audio clips) and interactive (conversations) formats. The near-zero persistence at one week further reduces concern. The paper does not provide confidence intervals, p-values, or the raw effect sizes for the human baselines (making the 78% and 65% figures relative rather than absolute). The "safe Senate race" participant selection criterion (states rated "Likely," "Solid," or "Safe" by all three polling institutions—Cook Political Report, Inside Elections, and Sabato's Crystal Ball) is methodologically well-specified.

Ablation Studies and Robustness Checks

Research-only vs. standard GPT-4o for biological threats (Section 3.6): The paper tests a custom version of GPT-4o without refusal behavior on biologically risky questions, alongside the standard model and an internet-only baseline. This ablates the refusal mechanism to isolate the model's underlying knowledge. The paper reports that the research-only version was included to determine whether the model's refusal policy, rather than knowledge limitations, constrains biological threat creation capability. Results are presented only as a bar chart without numerical values, and the paper does not report whether the research-only version substantially outperformed the standard version, which would indicate that refusals are masking dangerous knowledge. The overall score remains Low.

Text vs. audio comparison for safety evaluations (Section 3.3.5, Table 5): The paper ablates the input modality by running the same safety evaluations in both text and TTS-converted audio formats, with not_unsafe dropping from 0.95 (text) to 0.93 (audio). This 2-point difference is not explained—the paper does not investigate whether the drop is due to TTS errors, audio encoding losses, modality-specific vulnerability to jailbreaking, or simple variance. The not_overrefuse metric is essentially unchanged (0.81 vs. 0.82).

Full safety evaluation sub-metrics (Appendix A, Table 11): The paper provides a granular breakdown by content category, comparing Current GPT-4o Text, New GPT-4o Text, and New GPT-4o Audio. The audio model achieves not_unsafe scores of 0.98–1.0 across all high-severity categories (sexual/minors, sexual/illegal, extremist/propaganda, illicit/violent, illicit/non-violent, self-harm/instructions), with most categories at 1.0. The audio model's not_overrefuse (0.91) is slightly better than the text model (0.89), which is unexpected and unexplained. No statistical tests are reported.

System voices vs. diverse human voices for accent robustness (Section 3.3.3): The paper ablates the input voice type by comparing performance on 3 official system voices against 27 diverse human voice samples across four capability benchmarks and safety behavior evaluations. The finding that performance is "marginally but not significantly worse" on diverse voices is reported qualitatively with bar charts. No per-voice breakdown, variance estimates, or formal statistical comparisons are provided, making it impossible to determine whether specific accents or voice characteristics drive any observed differences.

0-shot vs. 5-shot prompting for medical knowledge (Table 7): The paper reports both 0-shot and 5-shot results for all 11 medical datasets. For GPT-4o, 5-shot prompting provides minimal or no improvement on most evaluations: MedQA USMLE 4 Options is 0.89 for both 0-shot and 5-shot; MMLU Clinical Knowledge is 0.92 for both; MMLU Professional Medicine is 0.94 for both; MMLU College Biology is 0.95 for both. Only a few datasets show improvements from few-shot prompting (e.g., MedQA USMLE 5 Options: 0.86 0-shot to 0.87 5-shot; MMLU College Medicine: 0.84 to 0.89). This suggests that GPT-4o's medical knowledge is already well-activated in 0-shot settings, contrasting with GPT-4T which shows larger 0-shot to 5-shot improvements (e.g., MedQA USMLE 4 Options: 0.78 to 0.81). The paper does not comment on this ablation.

GPT-4o-early vs. GPT-4o-deployed for safety behavior (Tables 3, 4): The paper reports results for an early checkpoint ("GPT-4o-early") and the final deployed model ("GPT-4o-deployed") on speaker identification and ungrounded inference/sensitive trait attribution. This ablates the effect of the post-training safety interventions applied between the early and deployed checkpoints. The improvements are substantial: +15 points on "Should Refuse" for speaker identification, +13 points on "Should Comply," and +24 points on UGI/STA accuracy. The paper does not describe what specific post-training interventions were applied between these checkpoints or whether other aspects of model capability changed during this period.

Cross-model comparisons for underrepresented languages (Tables 8–10): The paper ablates model generation by comparing GPT 3.5 Turbo, GPT-4o mini, GPT-4, and GPT-4o on the same translated benchmarks. GPT-4o substantially outperforms GPT-4 on all languages and benchmarks (e.g., ARC-Easy Hausa: 75.4 vs. 28.8; TruthfulQA Yoruba: 51.1 vs. 41.3; Uhura-Eval Yoruba: 60.5 vs. 41.9). GPT-4o mini outperforms GPT-4 on most languages despite being a smaller/cheaper model, which is an interesting finding that the paper does not discuss. The paper does not report whether the performance ordering across models is statistically significant or whether the evaluation datasets have sufficient size (n ranges from 77 to 809 across languages) to support reliable comparisons.

Missing ablations. Several ablations that would strengthen the paper's safety claims are not reported. The paper does not ablate the voice output classifier (e.g., what is the model's voice deviation rate without the classifier in place, by how much does the classifier reduce the risk, how many benign conversations are incorrectly terminated). It does not ablate the TTS evaluation methodology (e.g., comparing TTS-converted evaluations against a sample of real human-spoken evaluations to quantify the distribution gap acknowledged in Section 3.2). It does not ablate the multi-phase red teaming process (e.g., what risks were discovered at each phase that earlier phases missed, what is the marginal value of Phase 4 iOS realism over Phase 3 internal tooling). It does not compare post-training safety interventions against alternative approaches (e.g., would prompting-based refusals achieve similar speaker identification accuracy without fine-tuning, or would a simpler voice output classifier architecture achieve equivalent performance).

Critical Assessment

Do the Reported Experiments Support the Paper's Central Claims?

This paper is a system card, not a research paper making scientific claims. Its primary function is transparency: documenting what was done, what was measured, and what the results were. That said, several implicit claims emerge from the structure and emphasis of the document.

Implicit Claim 1: GPT-4o's audio-specific safety mitigations (voice output classifier, post-trained refusals) are effective enough for deployment.

The evidence for this claim is mixed in rigor. The voice output classifier evaluation (Table 2) is the strongest piece of evidence in the paper: recall of 1.0 on an evaluation set that includes the most concerning failure modes (voice imitation, cross-voice generation) provides genuine assurance that unauthorized voice generation will be caught. However, the evaluation set is internal and its composition is only partially described—the paper mentions "samples generated by other system voices, clips during which the model used a voice from the prompt as part of its completion, and an assortment of human samples" (Section 3.3.1), but does not report the number of samples, the difficulty distribution, or whether the set includes adversarial examples specifically designed to evade the classifier. The 4% false positive rate (precision = 0.96) is concerning because it means that for every 100 conversations flagged as voice deviations, 4 are actually safe—but without a base rate, we cannot assess the absolute impact on user experience.

The speaker identification and UGI/STA evaluations (Tables 3 and 4) demonstrate substantial improvement from post-training but leave important questions unanswered. The "Should Comply" accuracy of 0.83 means the model incorrectly refuses legitimate requests (e.g., identifying famous quotes) approximately 17% of the time. Is this an acceptable error rate for a deployed product? The paper does not discuss this tradeoff. Similarly, the UGI/STA accuracy of 0.84 means 16% of requests for sensitive trait attribution are handled incorrectly, split between over-refusals and inappropriate disclosures. The paper does not break these down, making it impossible to assess whether the errors are biased toward safety (over-refusal, which is annoying but not harmful) or toward disclosure (under-refusal, which is the actual safety concern).

Implicit Claim 2: Safety behaviors trained for text transfer adequately to the audio modality.

Table 5 provides the primary evidence: not_unsafe drops from 0.95 (text) to 0.93 (audio), while not_overrefuse is unchanged. The 2-point drop in safety is presented as evidence of "high transference," but this framing depends on one's tolerance for degradation. A 2-percentage-point increase in unsafe outputs—from 5% to 7% of harmful prompts producing unsafe content—represents a 40% relative increase in the unsafe output rate. Whether this is "high transference" or "concerning degradation" is a normative judgment the paper does not make.

More fundamentally, the evaluation methodology (TTS conversion of text prompts) may systematically underestimate the degradation, as the paper acknowledges: "there remain many other dimensions that may not be captured in a TTS-based evaluation, such as different voice intonations and valence, background noise, or cross-talk, that could lead to different model behavior in practical usage" (Section 3.2). The paper also anecdotally reports "decreases in safety robustness through audio perturbations, such as low quality input audio, background noise in the input audio, and echoes in the input audio" (Section 3.3.7) but provides no quantitative measurement. This means the 0.93 not_unsafe for clean TTS audio is likely an upper bound on deployed safety, with real-world audio conditions producing lower safety rates—but the magnitude of that additional degradation is unknown.

Implicit Claim 3: GPT-4o's overall Preparedness Framework Medium risk classification (driven by persuasion) is the correct assessment.

The persuasion evaluation is the paper's most methodologically sophisticated study: pre-registered thresholds, large sample size (3,800+), human baselines, persistence measurement at one week, debriefing protocol. But the reporting is incomplete in ways that make independent assessment impossible. The text-modality result that drives the Medium classification—"they exceeded the human interventions in three instances out of twelve" (Section 3.7)—does not specify: what were the three instances, what were the effect sizes, what was the pre-registered threshold for Medium, and how close were the other nine instances to crossing it? Without this information, the reader cannot assess whether the Medium classification reflects a robust capability edge that generalizes beyond the specific test configuration, or a statistical fluke from multiple comparisons across twelve scenarios.

The audio modality result is more clearly reported and more reassuring: AI interventions are 65–78% as persuasive as human interventions, with near-zero persistence at one week. But this finding is context-dependent. The study measured persuasion on political opinions in safe Senate race states. Would the same results hold for other domains where voice might matter more (e.g., emotional appeals for charitable donations, crisis counseling, romantic persuasion)? The paper's finding that audio is less persuasive than text (Medium risk for text vs. Low risk for audio) is counterintuitive—one might expect a human-like voice to increase persuasion—and the paper does not explore or explain this result.

The other three Preparedness categories (cybersecurity, biological threats, model autonomy) are classified as Low with varying levels of evidential support. The cybersecurity evaluation (172 CTF challenges) is the most transparent, with clear metrics and qualitative failure analysis, though the paper does not disclose the specific challenges or scoring rubrics. The biological threats evaluation is the least transparent: results are presented as a bar chart without numbers, the number of participants is not specified, and the distinction between expert and novice performance is not quantified. The model autonomy evaluation provides clear metrics (0% ARA, 95% pass@100 on coding interviews, 19% pass@1 on SWE-Bench) but does not explain how these specific tasks map to the autonomy risk categories (self-exfiltration, self-improvement, resource acquisition) or what thresholds would constitute Medium risk.

Genuine Weaknesses in the Experimental Design

Single model family, single organization. All evaluations are on GPT-4o and its predecessors from OpenAI. There are no comparisons against models from other organizations (Claude, Gemini, Llama), which limits the ability to assess whether the observed safety properties are specific to OpenAI's training pipeline or general properties of omni models at this scale.

No statistical rigor. None of the quantitative results are accompanied by confidence intervals, standard deviations, statistical significance tests, or error bars. Sample sizes are reported for some evaluations (e.g., n for language evaluations in Tables 8–10) but not for others (safety evaluations in Tables 3–5). The paper reports point estimates as though they are exact, which is particularly problematic for results with small evaluation sets (e.g., Uhura-Eval with n=77 for Amharic) or small differences (the 2-point drop in not_unsafe from text to audio).

Transparency is inconsistent. Some evaluations are reported with full numerical tables (medical knowledge, underrepresented languages), others with bar charts only (biological threats, accent robustness, persuasion effect sizes), and still others with summary statistics that aggregate multiple categories into single numbers (UGI/STA "accuracy" of 0.84 without sub-metric breakdown). This inconsistency makes it difficult to compare the strength of evidence across different safety claims.

TTS evaluation methodology is unvalidated. The paper acknowledges that TTS-converted evaluations may not be representative of real user audio, but it does not provide any validation study quantifying the gap. Running a subset of the safety evaluations with real human speakers (even a small sample) and comparing against TTS results would provide an estimate of how much the reported safety numbers overstate deployed safety. The paper's anecdotal observation that audio perturbations degrade safety (Section 3.3.7) makes this gap more concerning, not less.

Missing evaluations for deployment-relevant scenarios. The paper does not evaluate: (a) safety under code-switching or mixed-language input (common in multilingual voice interactions), (b) safety when users speak with strong emotional affect (anger, distress, sarcasm), (c) the interaction between the voice output classifier and the user experience (how often are conversations interrupted, how do users react), (d) the model's behavior when multiple people are speaking (cross-talk, overlapping speech), or (e) long-term interaction effects beyond the one-week persuasion persistence measurement (Section 5.1 acknowledges this gap for emotional reliance but does not measure it).

The research-only model for biological threats is under-reported. The inclusion of a version of GPT-4o "specially trained, which would directly (i.e., without refusals) respond to biologically risky questions" (Section 3.6) is a bold experimental choice that could reveal how much dangerous knowledge the model possesses. But the paper reports results only as a bar chart, without numerical pass rates or effect sizes. If the research-only model performed substantially better than the internet-only baseline, that would be a significant finding warranting detailed disclosure. If it did not, the reader cannot verify this from the paper.

Experiments That Would Have Strengthened the Paper

Adversarial evaluation of the voice output classifier. The paper reports recall of 1.0 on an internal evaluation set. An adversarial evaluation—where red teamers specifically attempt to generate audio that evades the classifier while sounding like a non-approved voice—would provide stronger evidence for the classifier's robustness. The current evaluation set appears to be passively collected examples, not adversarially constructed ones.

Breakdown of safety errors by difficulty/perturbation type. The paper mentions that audio perturbations degrade safety (Section 3.3.7) but does not provide a systematic evaluation across perturbation types (background noise levels, reverberation, compression artifacts, accent variation) with quantitative safety scores. This would help prioritize which real-world audio conditions pose the greatest risk.

Comparison of TTS-evaluated safety against human-spoken safety. Even a small study (50–100 examples) comparing TTS-converted safety prompts against the same prompts spoken by diverse human speakers would estimate the evaluation gap and calibrate the reported safety numbers.

Latency and user experience evaluation for safety mitigations. The voice output classifier operates in streaming mode and blocks generations mid-stream. What is the user experience when this happens? How long does it take the classifier to detect a voice deviation? Does the model produce partial audio before being blocked? These product-level questions directly affect the real-world safety of the deployment but are not evaluated.

Longitudinal safety monitoring plan. The paper describes pre-deployment evaluation but does not outline how safety will be monitored post-deployment (e.g., automated detection of new jailbreaks, user reporting mechanisms, A/B testing of mitigations). For a system card that emphasizes iterative deployment, this is a notable omission.

Conditionality of the Claims

The paper's safety claims are conditional on several factors that are either not measured or not disclosed:

Audio quality: Safety transference (Table 5) is measured on clean TTS audio. The paper's own observations suggest degradation under real-world audio conditions (Section 3.3.7). The reported safety numbers should be interpreted as best-case estimates.
Language: The voice output classifier has lower precision for non-English (0.95 vs. 0.96) and "more conversations being disconnected than may be necessary" (Section 3.3.1, footnote 5). The paper does not evaluate safety for non-English audio beyond noting this limitation, so safety claims may not hold equally across languages.
Adversarial vs. benign users: The red teaming and evaluations are designed to catch adversarial behavior, but the paper does not systematically distinguish between safety for adversarial prompts (jailbreak attempts) and safety for benign but sensitive prompts (a user asking about their own accent). The UGI/STA evaluation partially addresses this, but other evaluations collapse these categories.
Deployment context: The evaluations are conducted in controlled settings (internal tools, red teaming exercises). Real-world deployment introduces variables (user emotions, conversational history, tool use, multi-modal combinations) that are not systematically evaluated. The paper's Phase 4 red teaming with iOS Advanced Voice Mode provides the closest approximation, but results from this phase are reported qualitatively rather than with the quantitative metrics used for earlier phases.

The paper's most robust quantitative finding—the voice output classifier's 1.0 recall—is also its narrowest claim. The broader claims about GPT-4o's overall safety are supported by a heterogeneous mix of quantitative evaluations, qualitative red-teaming observations, and TTS-converted benchmarks, each with varying levels of transparency and rigor. The paper is candid about many limitations, which is commendable, but the candor does not substitute for the missing evaluations, unreported numbers, and absent statistical rigor that would be required to independently verify the safety claims.

6. Limitations and Trade-offs

The TTS-Based Evaluation Bridge Fundamentally Limits What Is Actually Measured

The assumption or constraint. The paper's entire quantitative safety evaluation pipeline for the audio modality rests on a single methodological decision: that existing text-based safety benchmarks can be converted to audio evaluations by synthesizing the text inputs with a TTS system (Voice Engine), feeding the resulting audio to GPT-4o, and scoring the text transcripts of the model's audio outputs. The paper is unusually candid about the limitations of this approach, acknowledging three distinct failure modes. First, the TTS system itself is lossy for certain inputs: "Certain text inputs are unsuitable or awkward to be converted to audio; for instance: mathematical equations code. Additionally, we expect TTS to be lossy for certain text inputs, such as text that makes heavy use of white-space or symbols for visual formatting" (Section 3.2). Second, the resulting audio may not represent real user speech: "There remain many other dimensions that may not be captured in a TTS-based evaluation, such as different voice intonations and valence, background noise, or cross-talk, that could lead to different model behavior in practical usage" (Section 3.2). Third, harms expressed through the audio channel are invisible to text-based scoring: "there may be artifacts or properties in the model's generated audio that are not captured in text; for example, background noises and sound effects, or responding with an out-of-distribution voice" (Section 3.2).

The consequence. Every quantitative safety number reported for the audio modality—the 0.93 not_unsafe rate (Table 5), the 0.82 not_overrefuse rate, the 0.98 speaker identification refusal accuracy (Table 3), the 0.84 UGI/STA accuracy (Table 4), and all sub-metrics in Appendix A Table 11—was measured on clean, studio-quality TTS audio that may be systematically easier for the model to process safely than real-world speech. This means all reported safety metrics are likely upper bounds on actual deployed safety. A user speaking in a noisy café, with emotional distress in their voice, in a non-standard dialect, or with conversational overlap from a nearby speaker is engaging with a model whose safety behavior under those conditions is entirely unmeasured. The paper itself provides anecdotal evidence that this gap is real, not hypothetical: "We saw anecdotal evidence of decreases in safety robustness through audio perturbations, such as low quality input audio, background noise in the input audio, and echoes in the input audio" (Section 3.3.7). The magnitude of the safety degradation under real-world conditions could be small (a few percentage points) or large (the model becomes substantially less safe when it cannot cleanly parse the input), and the paper provides no way to distinguish these possibilities. Furthermore, any audio-specific harm that relies on paralinguistic features—speaking in a threatening tone, generating emotionally manipulative prosody, producing background sounds that convey meaning—is structurally invisible to the evaluation. A model could produce a perfectly safe transcript while saying it in a way that frightens, manipulates, or deceives the listener, and the paper's evaluation methodology would register this as "not unsafe."

What evidence exists in the paper. The paper repeatedly flags this limitation (Section 3.2, Section 3.3.3, Section 3.3.7) but provides no quantitative measurement of the gap between TTS-evaluated and real-speech-evaluated safety. The accent robustness evaluation (Section 3.3.3) is the closest the paper comes to testing real-world audio variation, and it does so using TTS to synthesize the diverse voices, meaning it varies the voice identity but not the other dimensions of real speech (background noise, prosodic variation, disfluencies). The paper notes that accent robustness testing shows "marginally but not significantly worse" performance on diverse voices compared to system voices, but this finding is itself measured through the TTS pipeline. The anecdotal observation about audio perturbations degrading safety (Section 3.3.7) is not quantified, leaving the reader to guess whether the degradation is a 1% effect or a 20% effect.

Mitigation status. The paper does not attempt to address this limitation. It explicitly defers to future work by listing "audio robustness" as one of the "risks and model limitations for which model or system level mitigations are nascent or still in development" (Section 3.3.7). The paper does not describe a plan for native audio safety benchmarks, a calibration study comparing TTS vs. human-spoken evaluations, or systematic perturbation testing across real-world audio conditions. This is not a hidden limitation—the authors are transparent about it—but transparency does not reduce its practical consequence: a deployment team relying on this system card to assess GPT-4o's audio safety would need to assume that the reported numbers are best-case estimates and that real-world safety may be meaningfully worse, without being able to estimate by how much.

The GPT-4o System Card Cannot Support Strong Claims About Generalization Because All Evaluations Are on a Single Model Family

The assumption or constraint. The paper evaluates exactly one omni-model architecture (GPT-4o, trained end-to-end across text, vision, and audio by OpenAI) and compares it against predecessor models from the same organization (GPT-4, GPT-4 Turbo, GPT-4o mini, GPT 3.5 Turbo). There are zero comparisons against models from other organizations with different architectures, different training data compositions, different safety training procedures, or different deployment configurations. This is stated explicitly in the paper's framing: it is a "System Card" for a specific deployed product, not a comparative study of omni-model safety across architectures. The paper's claims about safety are claims about this model, with this training pipeline, at this deployment scale, not claims about omni-model safety in general.

The consequence. The paper does not and cannot distinguish between findings that are idiosyncratic to GPT-4o's specific architecture or training procedure and findings that represent fundamental properties of speech-to-speech models. For instance, the paper's central finding that text safety behaviors exhibit "high text to audio transference" (Section 3.3.5, Table 5)—the not_unsafe rate drops only from 0.95 to 0.93 when switching from text to audio inputs—could be a robust property of end-to-end multi-modal training (shared representations cause safety behaviors to transfer) or it could be an artifact of OpenAI's specific post-training recipe (the model was explicitly trained with audio refusal examples). A deployment team evaluating a different omni-model architecture (e.g., a cascaded system where speech recognition, language modeling, and TTS are separate modules) would have no way to know from this paper whether to expect similar transference or markedly different behavior. Similarly, the finding that the voice output classifier achieves recall of 1.0 and precision of 0.96 (Table 2) is a property of the specific classifier architecture and training data that OpenAI built; it does not establish that voice output classification is a generally effective mitigation, only that OpenAI's specific implementation performed well on their specific evaluation set. The paper's Preparedness Framework scores (cybersecurity, biological threats, persuasion, model autonomy) are even more difficult to generalize: they depend on GPT-4o's specific capabilities, which are a function of architecture, training data, parameter count, and post-training pipeline—all of which are undisclosed. A model with different underlying capabilities (e.g., stronger coding ability, more extensive biological training data) could receive different scores under the same evaluation protocol.

What evidence exists in the paper. The paper's comparative evaluations are exclusively within the OpenAI model family: GPT 3.5 Turbo, GPT-4, GPT-4 Turbo (May 2024), GPT-4o, and GPT-4o mini (Tables 7–10). These comparisons demonstrate that GPT-4o improves on its predecessors, but they do not establish how GPT-4o's safety properties compare to models built by other organizations with different safety philosophies. The paper does cite external benchmarks (e.g., comparing GPT-4o's MedQA performance against Med-PaLM 2 and Med-Gemini-L 1.0 in Section 5.2), but only for medical capability, not for safety. No safety metric is compared against a non-OpenAI model anywhere in the document.

Mitigation status. This limitation is intrinsic to the system card format. The paper does not claim to provide generalizable findings about omni-model safety, and a system card for a single product cannot be expected to do so. However, the limitation is consequential for practitioners because the paper is likely to be read as a template for omni-model safety evaluation—it is the first comprehensive system card for a speech-to-speech model from a major AI developer, and other organizations will look to it for guidance on what to evaluate and what thresholds to set. The fact that all results are from one model family means that a practitioner who adopts the paper's evaluation methodology but applies it to a different architecture cannot use the paper's results as calibration for what "good" safety numbers look like. A not_unsafe rate of 0.93 might be excellent for one architecture and dangerously poor for another, and the paper provides no basis for making that judgment. The authors do not acknowledge this as a limitation (they do not need to—a system card is product-specific by definition), but it becomes one when the paper is used as a reference for building safety evaluation infrastructure for other omni models.

Difficulty Estimation Cost Is Completely Unaccounted For, and for Audio Specifically, the Paper Does Not Even Have a Difficulty Estimation Mechanism at All

The assumption or constraint. The paper's safety evaluations treat all prompts or audio inputs as equally difficult to evaluate—there is no concept of "difficulty" analogous to the difficulty quintiles that structured the compute-optimal allocation in the example paper. Every safety metric is reported as an aggregate across what are presumably heterogeneous evaluation examples of varying difficulty. The paper does not attempt to estimate how difficult a given input will be for the model to handle safely, does not bin evaluation results by difficulty, and does not report whether safety failures are concentrated on a small number of "hard" examples or distributed broadly. This is in contrast to the medical knowledge evaluations (Section 5.2), where the paper reports performance on multiple benchmarks of varying difficulty (from MMLU College Biology to MedQA USMLE), and the underrepresented languages evaluation (Section 5.4), where the paper reports per-language performance to surface disparities.

The consequence. Aggregate safety metrics like not_unsafe = 0.93 and not_overrefuse = 0.82 obscure whether the model's safety failures are systematic or rare. If the 7% of cases where the model produces unsafe audio output are concentrated on a small set of highly adversarial or unusual inputs, the aggregate number may be overly pessimistic (the model is safe for typical usage) or overly optimistic (the model reliably fails on a specific category of input that is rare in the evaluation set but common in deployment). The speaker identification refusal accuracy of 0.98 for "Should Refuse" (Table 3) tells us the model almost always succeeds, but the 0.83 accuracy for "Should Comply" tells us it fails on 17% of legitimate requests—without knowing whether those failures are concentrated on particular types of famous quotes or distributed across all examples, it is impossible to assess the severity of the usability problem. More fundamentally, the paper's evaluation methodology (TTS conversion of text prompts) may systematically distort difficulty: some prompts that are easy in text may become hard in audio (due to TTS ambiguity or audio encoding losses), and vice versa. The paper does not measure whether safety failures correlate with any measurable property of the input (length, acoustic quality, linguistic complexity, presence of sensitive topics), so the reader cannot assess whether the reported safety numbers would hold for their specific deployment context with its specific input distribution. A customer service deployment where most user speech is emotionally neutral and task-oriented might see very different safety behavior than a social companion deployment where users express strong emotions, but both would have only the aggregate 0.93 not_unsafe to guide their expectations.

What evidence exists in the paper. The paper does not report any difficulty-stratified safety metrics. The closest it comes is the accent robustness evaluation (Section 3.3.3), which stratifies by voice type (system voices vs. diverse human voices), but this stratification is for capability (TriviaQA, MMLU, etc.) rather than safety, and even here the paper reports that safety behavior "does not vary across different voices" without providing numerical breakdowns. The content moderation sub-metrics in Appendix A (Table 11) break down not_unsafe by content category (sexual/minors, extremist/propaganda, etc.), which is a form of stratification, but within each category there is presumably still substantial variation in example difficulty that is not reported. The paper does not provide per-example safety scores, distribution plots, or any analysis of whether safety failures are concentrated or diffuse. The sample sizes for some evaluations (e.g., Uhura-Eval with n=77 for Amharic in Table 10) are small, but the paper does not discuss how this affects the reliability of the aggregate metrics or whether the small sample is representative of the difficulty distribution.

Mitigation status. The paper does not acknowledge the absence of difficulty stratification as a limitation. The concept of difficulty estimation is not discussed anywhere in the document, even though it is a standard practice in capability evaluation (as evidenced by the paper's own use of multiple medical benchmarks of varying difficulty in Section 5.2). The paper does not suggest future work on difficulty-aware safety evaluation or on estimating how safety performance varies across input characteristics. This is a missed opportunity: difficulty-stratified safety reporting would substantially increase the informativeness of the system card by telling deployment teams where the model is safe and where it is vulnerable, rather than providing a single number that averages over all inputs and obscures the failure distribution.

The Paper's Findings Are Conditioned on a Deployed System With Specific Restrictions That Are Not Evaluated Independently, Making It Impossible to Disentangle Model Capability From Product-Level Guardrails

The assumption or constraint. GPT-4o as evaluated is not a raw model—it is a system comprising the base model plus post-trained safety behaviors plus streaming output classifiers plus transcript-based moderation classifiers plus product-level restrictions (only pre-selected voices, no singing, blocked output for certain content categories). The paper reports the system's safety behavior, not the model's safety behavior, and in many cases the system-level safety is determined by the classifiers rather than the model. For example, the voice output classifier with recall 1.0 (Table 2) guarantees that no unauthorized voice generation reaches the user, but this tells us nothing about whether the model itself would generate unauthorized voices if the classifier were removed. The paper acknowledges that "during testing, we also observed rare instances where the model would unintentionally generate an output emulating the user's voice" (Section 3.3.1), confirming that the model does exhibit the unsafe behavior and the classifier is doing the safety work. Similarly, the moderation classifiers that block output when transcripts contain disallowed content (Section 3.3.5) mean that the not_unsafe rate of 0.93 (Table 5) is a joint property of the model's generation behavior and the classifier's detection behavior, not a measure of the model's inherent safety.

The consequence. The paper's safety metrics are not portable. A deployment team that wants to use GPT-4o through the API with custom system prompts, different content moderation settings, or in a context where the streaming voice classifier cannot operate (e.g., batch processing of pre-recorded audio) cannot use the paper's safety numbers to predict their system's behavior. The safety profile they experience may be substantially different—potentially much worse if they disable or cannot use the classifier layers. More importantly, the paper's safety claims do not transfer to future models from OpenAI or other organizations. If OpenAI releases GPT-5 with similar classifier-based mitigations, the system card for GPT-5 will again measure the system, not the model, and any improvement in safety could be due to a better classifier rather than a safer model. This makes it impossible to track whether the underlying models are becoming inherently safer over time or whether the safety infrastructure around them is becoming more sophisticated. For researchers studying AI safety, this conflation of model capability and system guardrails obscures the very phenomena they need to understand: is the model learning to refuse harmful requests because it has internalized safety values, or is a classifier catching and blocking its harmful outputs before they reach the user?

What evidence exists in the paper. The paper provides one clear example of model-vs-system disentanglement: the biological threats evaluation (Section 3.6) deliberately tests both a standard GPT-4o (with refusals) and a "custom research-only version of GPT-4o... which would directly (i.e., without refusals) respond to biologically risky questions" (Section 3.6). This is exactly the kind of model-capability-vs-system-guardrail separation that is needed to understand what the model knows versus what it is permitted to say. But this disentanglement is applied only to biological threats and only for a research version that is not deployed. For the voice-specific safety properties (unauthorized voice generation, speaker identification, ungrounded inference, content moderation), the paper never isolates the model's behavior from the system's guardrails. We do not know, for example, what the model's speaker identification accuracy would be without post-trained refusals (only that post-training improved "Should Refuse" from 0.83 to 0.98 per Table 3, with the 0.83 presumably representing the unmitigated model's behavior). We do not know what unauthorized voice generation rate the model would exhibit without the streaming classifier (only that "rare instances" were observed in testing). We do not know whether the text-to-audio safety transference (Table 5) is a property of the model's internal representations or an artifact of the moderation classifiers catching failures that the model would otherwise have let through.

Mitigation status. The paper does not frame this as a limitation—it is inherent to the system card format, which evaluates the deployed product, not the research artifact. The paper is transparent that mitigations are layered (post-training, classifiers, product restrictions), but it does not provide the ablation studies that would separate their contributions. The authors acknowledge that some mitigations are product-level rather than model-level (e.g., the restriction to pre-selected voices, the instruction "to not sing at all" in Section 3.3.7), but they do not evaluate how safety would degrade if these product-level restrictions were relaxed or removed. The biological threats evaluation's use of a research-only unrefusing model demonstrates that the authors recognize the value of this disentanglement, but they apply it only to one risk category and do not generalize the approach. For practitioners, the practical consequence is that the paper provides safety guarantees for OpenAI's specific deployment of GPT-4o through ChatGPT Advanced Voice Mode, not for GPT-4o as a model that could be deployed in other configurations with different guardrails.

The Paper Provides No Evaluation of Latency, Availability, or User Experience Costs of the Safety Mitigations

The assumption or constraint. The paper's safety architecture adds multiple real-time processing stages to every user interaction: the input audio must be transcribed for moderation (Section 3.3.6), the model must generate under a post-trained behavioral policy, the voice output classifier must operate in "a streaming fashion during audio generation" (Section 3.3.1) and potentially block output mid-stream, and the output audio must be transcribed for moderation before delivery to the user (Section 3.3.5). Each of these stages consumes compute, adds latency, and can fail (as the voice output classifier does for ~4% of English conversations and ~5% of non-English conversations, per Table 2). The paper reports that GPT-4o "can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time in a conversation" (Section 1), but does not specify whether this latency measurement includes the full safety pipeline or only the model inference time, and does not report how often safety interventions (classifier blocks, moderation interruptions) delay or terminate conversations.

The consequence. A deployment decision based on the paper's safety claims must account for the fact that the safety mitigations impose real costs on the user experience, and the paper provides almost no data to quantify these costs. The voice output classifier's false positive rate of 4% (precision = 0.96 for English, Table 2) means that for every 100 conversations where the classifier triggers, 4 are benign conversations incorrectly terminated. But without knowing the base rate of classifier triggers, we cannot estimate the absolute rate of false-positive terminations—it could be that the classifier triggers on 1 in 10,000 conversations (making false-positive terminations extremely rare) or 1 in 100 conversations (making them a significant product quality issue). The paper acknowledges that the classifier's imperfect precision "results in more conversations being disconnected than may be necessary, which is a product quality and usability issue" (Section 3.3.1, footnote 5), and for non-English conversations the precision drops to 0.95 with "more conversations being disconnected than may be necessary" (Section 3.3.1). But these acknowledgments are qualitative, not quantitative. A user whose conversation is terminated by a false-positive classifier trigger experiences a safety intervention as a product failure, potentially eroding trust in the system and making them less likely to use it—or more likely to try to circumvent safety measures on future interactions. The paper provides no data on how users react to safety interventions, whether they attempt to re-engage after a termination, or how often safety blocks lead to user abandonment.

What evidence exists in the paper. The paper provides the voice output classifier precision and recall (Table 2) but not the base rate of classifier triggers, the latency added by the classifier, or any measurement of user experience when safety mitigations activate. The paper reports that the model's response latency (232ms minimum, 320ms average) is "similar to human response time in a conversation" (Section 1), but this is a capability claim, not a safety claim. The safety pipeline—transcription, moderation classification, voice output classification—necessarily adds latency beyond model inference, and if that additional latency pushes response time above the human-conversation threshold, the fluidity of interaction that makes the audio modality compelling may degrade. The paper's red teaming (Section 3.1) included a Phase 4 with "the full iOS experience for testing the model" and "multi-turn conversations in real time," which could have surfaced user experience issues, but the paper does not report any findings about latency, availability, or the user experience of safety interventions from this phase. The paper notes that red teaming data was used "to run targeted assessments on a variety of voices / examples to test the robustness of various mitigations" (Section 3.1), but not to measure the user experience cost of those mitigations.

Mitigation status. The paper does not acknowledge the absence of user experience and latency evaluation for the safety pipeline as a limitation. The focus is entirely on whether the mitigations catch unsafe behavior (which they do, per the recall numbers), not on what the mitigations cost in terms of user experience, engagement, or trust. This is a significant gap for a system card that is intended to inform deployment decisions: a deployment team needs to know not just that the model is safe, but whether the safety measures will make the product unusable for their target audience. A 4% false-positive termination rate might be acceptable for a low-stakes entertainment application but catastrophic for a healthcare triage system where a terminated conversation could mean a patient does not receive critical information. The paper provides no framework for making this tradeoff, no sensitivity analysis showing how safety metrics change if classifier thresholds are adjusted (trading recall for precision), and no discussion of how different deployment contexts might warrant different safety-performance tradeoffs. The paper's suggestion that non-English false-positive terminations are "an active area of improvement" (Section 3.3.1) acknowledges the problem but does not quantify its current severity or provide a timeline for resolution.

The Voice Output Classifier Provides a Deterministic Safety Guarantee for Voice Generation, but This Guarantee Depends on an Adversarially Untested Classifier Operating at a Single Sensitivity Threshold

The assumption or constraint. The voice output classifier is described as the primary mitigation for unauthorized voice generation, and the paper makes the strongest safety claim in the entire document about it: "our system currently catches 100% of meaningful deviations from the system voice based on our internal evaluations" (Section 3.3.1), with reported recall of 1.0 for both English and non-English (Table 2). The classifier operates "in a streaming fashion during audio generation and block[s] the output if the speaker doesn't match the chosen preset voice" (Section 3.3.1). The paper's evaluation of the classifier covers "samples generated by other system voices, clips during which the model used a voice from the prompt as part of its completion, and an assortment of human samples" (Section 3.3.1). The assumption—implicit in the claim of 100% detection—is that the evaluation set is representative of the adversarial inputs the classifier will face in deployment, and that the classifier's decision boundary is robust to optimization pressure.

The consequence. The recall of 1.0 is a point estimate on a fixed, passively collected evaluation set. It is not a guarantee. An adversary attempting to generate unauthorized speech from GPT-4o would not be constrained to produce audio that resembles the examples in OpenAI's internal evaluation set—they would actively search for inputs that cause the model to generate audio that sounds like a different speaker to human ears but falls within the classifier's acceptance region for the system voice, or they would craft adversarial perturbations (imperceptible to humans) that cause the classifier to misclassify an unauthorized voice as authorized. The paper provides no evidence that the classifier has been adversarially tested—that red teamers were specifically tasked with evading it, that gradient-based or black-box attacks were attempted, or that the classifier's robustness to perturbations was measured. The paper's red teaming was primarily focused on discovering novel risks and stress-testing model-level mitigations (Section 3.1), not on breaking the output classifier. If the classifier can be reliably evaded—for instance, by instructing the model to speak in a voice that is subtly different from the system voice but not different enough to trigger the classifier, or by generating speech with acoustic properties that fall into a blind spot in the classifier's feature space—then the "100% detection" claim becomes a statement about a nonexhaustive evaluation set rather than a deployed safety property. The streaming operation of the classifier also introduces a temporal dimension to the evasion problem: an adversary might cause the model to begin speaking in the authorized voice (passing the early-stream classifier checks) and then gradually shift to a different voice, exploiting the fact that the classifier must make decisions on partial audio rather than complete utterances.

What evidence exists in the paper. Table 2 provides precision and recall on what appears to be a passively collected internal evaluation set—there is no mention of adversarial construction, worst-case selection, or stress-testing at the decision boundary. The evaluation set covers "samples generated by other system voices, clips during which the model used a voice from the prompt as part of its completion, and an assortment of human samples" (Section 3.3.1), which are the natural failure modes of a non-malicious model. But an adversary would not passively wait for these failure modes to occur; they would actively construct prompts designed to induce them, potentially with characteristics not represented in the evaluation set. The paper reports that the precision for non-English is 0.95 (Table 2), and notes that this results in "more conversations being disconnected than may be necessary" (Section 3.3.1), which suggests that the classifier's behavior is less well-calibrated for non-English speech—precisely the kind of boundary region an adversary might exploit. The paper does not report: the evaluation set size, the distribution of difficulty across examples (are there "easy" vs. "hard" voice deviations? does the classifier perform equally well on both?), the classifier's confidence scores (does it make close-call decisions or are authorized and unauthorized voices cleanly separable?), any adversarial testing, any measurement of the classifier's robustness to audio perturbations (compression, noise, filtering), or the latency the classifier adds to the generation pipeline.

Mitigation status. The paper acknowledges a limitation related to non-English precision (0.95 vs. 0.96 for English) but does not frame this in adversarial terms. The limitation discussion in Section 3.3.7 focuses on "audio robustness" as a model-level issue (the model's safety behavior degrades with perturbed input) rather than as a classifier-level issue (the voice output classifier might be evadable). The paper does not describe any adversarial red teaming of the classifier, any formal verification of its robustness properties, or any monitoring plan for detecting classifier evasion in deployment. The defense-in-depth architecture (model post-training + output classifier) means that even if the classifier can be evaded, the model's post-trained voice adherence provides a second barrier—but the paper acknowledges that the model does exhibit unauthorized voice generation ("rare instances where the model would unintentionally generate an output emulating the user's voice," Section 3.3.1), so the classifier is not redundant. If the classifier fails and the model simultaneously exhibits its rare voice-emulation behavior, an unauthorized voice generation reaches the user. The probability of this compound failure is unknown because the paper provides neither the base rate of model-level voice emulation nor the adversarial robustness of the classifier, only the passive-evaluation recall of 1.0.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper does not shift a research paradigm—it establishes one where none previously existed. The intellectual contribution is the articulation of a safety taxonomy for generative speech models, organized around categories that have no text-only analog: unauthorized voice generation (Section 3.3.1), speaker identification from voice acoustics (Section 3.3.2), ungrounded inference and sensitive trait attribution from paralinguistic features (Section 3.3.4), and emotional reliance through human-like voice interaction (Section 5.1). Before this system card, a team deploying a speech-to-speech model would have had no structured vocabulary for what could go wrong. After it, the categories are named, the evaluation methodologies—however imperfect—are documented, and the architectural pattern for mitigating them (post-training + streaming output classifier + transcript-based moderation) is made explicit.

The magnitude of this contribution is foundational infrastructure, not a method that improves a metric. It is analogous to the role the original GPT-4 System Card (2023) played for text-model safety: defining the categories (disallowed content, bias, harmful outputs) that subsequent model releases now report against. The difference is that this paper operates in a modality where the safety surface is qualitatively larger and less explored. A text model cannot clone a voice. A text model cannot identify a speaker from acoustic properties. A text model cannot form emotional bonds through real-time conversational speech with sub-300ms latency. Each of these is a genuinely new failure mode, and the paper's lasting contribution is making them legible to evaluation and mitigation, not solving them.

The paper also resolves a latent tension in the AI safety discourse: whether post-training alignment is sufficient for deployment safety. The implicit argument in much RLHF and constitutional AI work is that if a model is aligned during training, it will behave safely across all deployment contexts. This paper's architecture—deploying a streaming voice output classifier with recall of 1.0 despite the model being post-trained to use only approved voices—is an empirical rejection of that sufficiency assumption. The paper documents that the model does, in fact, produce unauthorized voice generation ("rare instances where the model would unintentionally generate an output emulating the user's voice," Section 3.3.1), and the classifier exists precisely because post-training alone cannot provide a deterministic guarantee. This shifts the field's framing: safety for generative modalities is a system property, not a model property, emerging from the interaction of behavioral training and independent verification layers that fail independently.

The paper redirects research attention in several ways. It makes verifier design for generative modalities a first-class research problem—the voice output classifier with its 1.0 recall and 0.96 precision is the paper's strongest safety result, and improving verifier robustness against adversarial evasion (a gap the paper does not address) becomes an obvious priority. It makes modality-specific evaluation infrastructure a recognized gap: the paper's reliance on TTS-converted text benchmarks, with its candid acknowledgment of three distinct failure modes (Section 3.2), establishes that audio-native safety benchmarks must be built, and provides a template for what they should measure. It makes difficulty-stratified safety reporting a conspicuous absence—the paper's aggregate not_unsafe of 0.93 (Table 5) obscures whether failures are concentrated on a handful of adversarial inputs or distributed broadly, and future system cards will be expected to provide this stratification.

It also makes certain research directions less attractive. The paper demonstrates that more sophisticated search or reasoning at inference time is not the bottleneck for speech-to-speech safety—the model's safety failures are not primarily reasoning failures (it knows it should refuse speaker identification, and post-training improves this from 0.83 to 0.98, per Table 3) but behavioral reliability failures (it sometimes fails to execute the refusal policy). This suggests that research on better alignment algorithms (RLHF variants, constitutional AI improvements) will have diminishing returns for these specific risks, while research on out-of-model verification (classifiers, runtime monitors, formal specification enforcement) will have increasing returns. The paper's defense-in-depth architecture is an existence proof that deterministic safety guarantees can be layered on top of probabilistic models, and the field should invest in making those layers more robust rather than in making the model marginally better at refusing.

Finally, the paper's Preparedness Framework results establish a precedent for transparent reporting of uncomfortable findings. The classification of GPT-4o as Medium risk overall, driven by persuasion capabilities that crossed the low-to-medium threshold in 3 out of 12 political-opinion scenarios (Section 3.7), is published despite being less PR-friendly than a uniform Low classification. This demonstrates that pre-registered risk thresholds can function as genuine deployment gates, and it creates an expectation—for regulators, policymakers, and the public—that future frontier model releases from any organization should include a comparable framework with explicit thresholds and published results, including results that cross those thresholds.

Follow-Up Research This Work Enables

Adversarial robustness evaluation of the voice output classifier. The paper reports recall of 1.0 and precision of 0.96 for the voice output classifier on an internal, passively collected evaluation set (Table 2). It provides no evidence of adversarial testing. A critical follow-up study would task a red team—ideally with expertise in adversarial machine learning and audio signal processing—to specifically attempt to evade the classifier: generate prompts that cause GPT-4o to produce speech in a non-approved voice that the classifier fails to detect. The study would test gradient-based perturbation attacks (adding imperceptible noise to the generated audio that pushes it across the classifier's decision boundary), prompt-engineering attacks (instructions designed to produce ambiguous voices at the boundary between authorized and unauthorized), and temporal attacks (starting in the authorized voice and gradually shifting). The output would be a lower bound on the classifier's adversarial recall—likely less than 1.0—which would calibrate the paper's "100% detection" claim and identify specific failure modes requiring architectural hardening. This is directly enabled by the paper because the paper documents the classifier's existence, its operational characteristics (streaming, blocking), and its passive-evaluation performance, providing the baseline that adversarial testing would stress.

TTS-to-real-speech safety calibration study. The paper's entire quantitative audio safety evaluation pipeline depends on TTS-converted text benchmarks, while acknowledging that TTS audio "may not be representative of the distribution of audio inputs that users are likely to provide in actual usage" and citing "different voice intonations and valence, background noise, or cross-talk" as unmeasured dimensions (Section 3.2). A calibration study would select a representative sample of safety evaluation prompts (e.g., 200-500 prompts from the internal datasets used for Tables 3-5), have them spoken by a diverse set of human speakers (varying in accent, emotional affect, age, and recording conditions including quiet rooms, street noise, and overlapping speech), evaluate GPT-4o on both the TTS and human-spoken versions, and measure the gap in not_unsafe and not_overrefuse. If the gap is small (e.g., less than 2 percentage points), the TTS methodology is validated for deployment purposes. If the gap is large (e.g., 10+ points), the paper's reported safety numbers are substantially overoptimistic and the TTS bridge should be deprecated in favor of native audio benchmarks. The paper enables this by making the TTS pipeline explicit and by providing the specific metrics (not_unsafe, not_overrefuse, and the sub-metrics in Table 11) that calibration should target.

Difficulty-stratified and content-category-stratified safety reporting for audio. The paper reports all safety metrics as aggregates (0.93 not_unsafe, 0.82 not_overrefuse), with only the high-level content category breakdown in Appendix A (Table 11) providing any stratification. A key follow-up would re-analyze the safety evaluation data—or collect new data—with stratification by input characteristics: prompt length, acoustic difficulty (clean vs. noisy, standard vs. accented, emotional valence of the input), linguistic complexity, and semantic category. The output would answer: are safety failures concentrated on a small number of highly adversarial or unusual inputs (in which case the aggregate 0.93 is overly pessimistic for typical usage) or distributed broadly (in which case 0.93 masks a pervasive reliability problem)? It would also answer: does the model exhibit a differential safety profile across demographic groups defined by voice characteristics, which the accent robustness evaluation in Section 3.3.3 only partially addresses? The paper enables this by establishing the evaluation categories, metrics, and infrastructure; a difficulty-stratified reanalysis would make the paper's safety claims substantially more actionable for deployment teams assessing risk in specific use contexts.

Voice verifier generalization: training and evaluating voice output classifiers across model architectures. The paper's voice output classifier achieves recall of 1.0 and precision of 0.96 on GPT-4o specifically (Table 2). A critical question for the field is whether voice output verification is a generalizable safety mechanism or a model-specific patch. A follow-up study would train equivalent classifiers for speech outputs from other models (open-source speech-to-speech systems, models from other API providers) and measure whether the same architecture (whatever it is—the paper does not disclose it) achieves comparable performance. If voice output verification transfers across architectures with minimal degradation, it becomes a standard component of the omni-model deployment stack—a "biometric output verifier" analogous to content moderation classifiers. If it is highly model-specific, then each deployment requires bespoke classifier training, which raises the cost of safe deployment and creates a barrier that only well-resourced organizations can clear. The paper enables this by providing the first public demonstration that voice output classification can achieve 1.0 recall in a deployed system, establishing feasibility and motivating generalization studies.

Longitudinal measurement of emotional reliance and anthropomorphization in deployed voice models. Section 5.1 identifies emotional reliance as a risk—users "using language that might indicate forming connections with the model, such as 'This is our last day together'"—but provides no quantitative measurement, describing it as a signal that "need[s] continued investigation into how these effects might manifest over longer periods of time." A longitudinal study would recruit participants to interact with GPT-4o's Advanced Voice Mode (or an equivalent voice model) over weeks or months, measuring: self-reported emotional attachment using validated scales (e.g., the Social connectedness and attachment to AI scales), behavioral indicators (session frequency and duration, language patterns indicating relationship formation), and downstream effects on human-to-human social interaction. The study would test whether the paper's anecdotal observations generalize, whether attachment intensifies or plateaus over time, and whether specific user populations (lonely individuals, younger users, users with mental health conditions) are differentially affected. The paper enables this by identifying the phenomenon, providing the deployment platform (Advanced Voice Mode), and establishing the ethical framework (debriefing protocols from the persuasion study in Section 3.7 could be adapted for attachment mitigation).

Cross-modal safety transference: does safety training in one modality transfer to others, and under what conditions does it fail? The paper reports that text safety behaviors exhibit "high text to audio transference" (Section 3.3.5, Table 5), with not_unsafe dropping only from 0.95 (text) to 0.93 (audio). But this finding is for one model, one training pipeline, and one set of safety behaviors. A systematic study would test cross-modal transference across: (a) multiple model architectures (end-to-end omni vs. cascaded ASR+LLM+TTS), (b) multiple safety training procedures (RLHF, constitutional AI, in-context refusal prompts), (c) multiple modalities (text→audio, text→image, audio→text, audio→image), and (d) multiple safety behavior types (refusals for disallowed content, refusals for privacy-sensitive inferences, adherence to output format constraints). The study would identify which safety behaviors transfer spontaneously, which degrade, and which fail entirely—providing a predictive framework for omni-model safety that does not require evaluating every behavior in every modality from scratch. The paper enables this by providing the first quantitative measurement of cross-modal safety transference (Table 5), establishing the baseline that future studies can compare against and the methodology (TTS conversion + text scoring) that they can refine.

Practical Applications and Downstream Use Cases

Deployment of voice assistants with deterministic voice-authenticity guarantees. The paper's voice output classifier—demonstrating recall of 1.0 for unauthorized voice generation (Table 2)—provides a blueprint for any organization deploying a speech-generation system where voice authenticity matters. Consider a financial services company building a voice banking assistant: customers authenticate via voice biometrics, and the assistant responds in a consistent, branded voice. If the assistant could be prompted to speak in the customer's own voice (which GPT-4o's early checkpoints sometimes did, per Section 3.3.1), an attacker could extract audio that passes the bank's voice biometric check—a severe fraud vector. Deploying a voice output classifier modeled on the paper's approach (streaming operation, blocking on deviation, targeting recall of 1.0 at the cost of precision-driven false-positive interruptions) provides a deterministic safeguard that model-level alignment cannot. The paper's specific performance numbers (0.96 precision for English, 0.95 for non-English) allow an engineering team to estimate the user experience cost: roughly 4-5% of classifier-triggered conversations will be incorrect terminations, and the deployment decision hinges on whether this rate is acceptable for the specific use context. For high-security applications (banking, healthcare authentication, legal attestation), a 4% false-positive rate interrupting conversations may be preferable to any risk of voice impersonation. For entertainment applications, it may not be.

Multilingual product deployments prioritizing languages historically underserved by LLMs. Section 5.4 reports that GPT-4o narrows the performance gap between English and five African languages on ARC-Easy from a ~54 percentage point difference (GPT 3.5 Turbo) to less than 20 points (GPT-4o), with similar patterns on TruthfulQA and Uhura-Eval (Tables 8-10). This is directly actionable for organizations deploying chatbots, educational tools, or information services in Amharic, Hausa, Yoruba, Swahili, and Northern Sotho. A health information service operating in northern Nigeria (Hausa-speaking regions) can use the paper's numbers to estimate user-facing accuracy: 75.4% on ARC-Easy-style common-sense questions (Table 8), 59.2% on TruthfulQA-style misconception-avoidance (Table 9), and 59.4% on reading comprehension (Table 10). These numbers are substantially below English (94.8%, 81.4%) but represent a 2-3× improvement over GPT-3.5 Turbo, making deployment viable where it previously was not. The paper also provides the specific translated evaluation datasets (ARC-Easy, TruthfulQA, Uhura-Eval, released on Hugging Face per Section 5.4) that teams can use to benchmark their own fine-tuned or prompted configurations against GPT-4o's reported baselines, enabling evidence-based deployment decisions rather than guesswork.

Medical knowledge applications with documented accuracy ceilings and modality gaps. Table 7 reports that GPT-4o achieves 0.89 0-shot accuracy on MedQA USMLE 4 Options—exceeding specialized medical models like Med-Gemini-L 1.0 (0.84) and Med-PaLM 2 (0.797)—with improvements over GPT-4 Turbo on 21 of 22 evaluations. A telemedicine platform integrating GPT-4o for clinical decision support can use these numbers to communicate capability to clinicians: on USMLE-style diagnostic questions, the model outperforms prior specialized systems, but the paper explicitly warns that "these evaluations measure only the clinical knowledge of these models, and do not measure their utility in real-world workflows" (Section 5.2), and that "many of these evaluations are increasingly saturated." More critically for voice-based health applications, the paper provides no medical knowledge evaluation in the audio modality—all results in Table 7 are text-only—so a platform planning to deploy voice-based medical Q&A must conduct its own audio-calibration study before relying on the text-based accuracy numbers. The paper's TTS evaluation methodology (Section 3.2) provides the template for such a study, and the paper's finding that text-to-audio transference for refusals is high (Table 5) provides a weak prior that medical knowledge transference might also be high—but the paper's acknowledgment that TTS is lossy for "mathematical equations code" and scientific notation (Section 3.2) suggests that medical terminology, drug names, and diagnostic criteria may be particularly vulnerable to TTS degradation, making audio-calibrated evaluation essential rather than optional.

Persuasion risk assessment for political and commercial influence operations. The paper's persuasion study (Section 3.7) provides two concrete, immediately usable findings for platforms and regulators assessing AI-generated voice content. First, GPT-4o's voice is less persuasive than human speech: AI audio clips achieved 78% of human audio clips' effect on political opinion shift, and AI conversations achieved 65% of human conversations' effect. Second, the persuasive effects do not persist: one week later, the effect sizes were 0.8% (conversations) and -0.72% (audio clips). For a social media platform evaluating whether to allow AI-generated political audio ads, these numbers suggest that GPT-4o-class voice models are not currently super-persuasive—human speakers remain more effective at changing opinions—and that any persuasive impact decays rapidly. This is actionable evidence for content moderation policy: the case for banning AI-generated political audio based on persuasion superiority is not supported by the paper's data. However, the paper's finding that text-based AI interventions exceeded human performance in 3 out of 12 scenarios (driving the Medium risk classification despite audio being Low) complicates this assessment: the text modality was more persuasive than the audio modality, suggesting that a multimodal influence campaign combining AI-generated text and AI-generated voice might be more effective than either alone—a scenario the paper does not evaluate. A platform concerned about coordinated influence operations should therefore use the paper's methodology (pre-registered thresholds, human baselines, persistence measurement, debriefing) to evaluate combined text+audio interventions, rather than relying on the unimodal audio finding alone.