Reclaiming the Digital Commons: A Public Data Trust for Training Data

🎯 Pitch

A public data trust could force AI companies to pay royalties on their profits by gatekeeping the internet-scraped data they rely on. The key question is whether human-generated data remains essential—or if synthetic data makes the whole scheme obsolete.

1. Executive Summary

This paper proposes the creation of a public data trust to assert collective control over the training data for foundation models, specifically internet-scraped pre-training data and crowdworker-generated human feedback data. The trust would gate access to this data—the digital commons—licensing it to commercial model developers in exchange for royalties on model revenues, which would fund redistribution mechanisms to address negative externalities such as degradation of information quality and unemployment from automation. The paper details a verification regime combining digital signature insertion via data poisoning, proof-of-learning protocols, and compute-provider coordination to enforce that developers use only the trust's data, alongside a mix of regulatory mandates, certification labels analogous to Fair Trade, and the trust's own comparative advantage in data curation as incentives for compliance. The proposal establishes that a data trust could serve as a structural intervention to correct the power imbalance between private AI developers and the public, though its leverage depends critically on human-generated data remaining a bottleneck for model training—a condition that may erode as model-generated synthetic data becomes increasingly capable.

2. Context and Motivation

The Core Problem: Private Capture of a Public Resource

The fundamental problem this paper tackles sits at the intersection of political economy and AI development: the digital commons—humanity's collective intellectual and cultural output in digital form—is being captured by a handful of private corporations to build AI systems whose benefits flow disproportionately to those same corporations, while their costs are externalized onto the public. This is not a technical problem of model architecture or training efficiency. It is a structural problem of who gets to decide how AI is developed, on whose data, and to whose benefit.

The digital commons encompasses resources like Wikipedia, arXiv, Reddit, Project Gutenberg, online news sites, and open-source code repositories—infrastructure that underpins democratic deliberation, material well-being, and cultural enrichment. These resources are the product of billions of hours of human effort, contributed largely without expectation of direct financial return. Yet the paper argues, and the empirical record supports, that when private companies scrape this commons to train foundation models like GPT-4, the resulting economic value is privatized while the harms—degradation of information quality, displacement of creative labor, potential mass unemployment—are socialized.

What makes this a structural problem rather than merely a fairness concern is that no individual data contributor has meaningful bargaining power. As the paper states in its introduction:

"Even if an individual were to threaten to withhold their data from a model developer, they would have effectively no bargaining power since a few data points likely make no significant difference in the final performance of a model."

This is a collective action problem of the classic kind: the resource (training data) derives its value from aggregation, but the governance mechanisms for that aggregation are absent. The default legal and technical infrastructure—where anyone can scrape publicly available internet data, where opt-out mechanisms are lacking or ineffective, and where training datasets of the largest models remain private—systematically favors the actors with the most compute and capital.

Why This Problem Matters Now

The urgency of this problem has increased dramatically in the 2020s for several converging reasons that the paper identifies, either explicitly or through its framing.

First, the scale of data extraction has expanded enormously. Scaling laws (Hoffmann et al., 2022; Kaplan et al., 2020) have established predictable relationships between data quantity, model size, and performance, driving a voracious appetite for ever-larger training corpora—now measured in trillions of tokens and petabytes of data. What was once a curiosity—training on web text—has become the central pathway to state-of-the-art capabilities. The digital commons is no longer one input among many; it is the substrate on which the most capable AI systems are built.

Second, the capabilities of these systems have crossed a threshold where they can actively harm the commons that produced them. The paper identifies two specific mechanisms. One is direct: foundation models can generate low-quality content at scale, flooding the digital commons with misinformation, falsehoods, and synthetic text that makes it harder to find genuine human contributions. The other is economic: as these models become substitutes for search (users getting answers directly from chatbots rather than visiting websites) and substitutes for creative labor (text-to-image models displacing illustrators), they threaten the revenue streams and motivations that sustain contributions to the commons. As the paper notes:

"The recent history of social media moderation shows that profit motives may override the importance of a high-quality public forum."

Third, the labor market implications are no longer speculative. The paper was written in early 2023, when concrete examples were already visible: OpenAI hiring contractors to make basic coding obsolete, artists suing Stability AI for training on their work without consent or compensation, and widespread deployment of systems that can generate professional-quality text, images, and code at near-zero marginal cost. The negative externality of unemployment from automation is not a future hypothetical—it is an active process—and there exists no mechanism to channel a portion of AI-derived revenue back to those whose labor and creativity made the training data possible.

Fourth, the window for structural intervention may be closing. As Section 3.2 of the paper argues, if model-generated synthetic training data becomes sufficiently capable to substitute for human-generated data, the bargaining power of any entity controlling human-sourced data would diminish. The establishment of a data trust is therefore time-sensitive: it must happen while human data remains the performance bottleneck.

Prior Approaches and Where They Fall Short

The paper does not invent the concept of data governance from scratch. It positions itself against a landscape of existing mechanisms—legal, technical, and organizational—each of which it argues is insufficient for the specific challenge it identifies.

Data rights legislation (GDPR, PIPEDA, CCPA). The EU's General Data Protection Regulation, Canada's Personal Information Protection and Electronic Documents Act, and California's Consumer Privacy Act all establish individual rights over personal data, including rights to access, correct, and in some cases delete data. The paper acknowledges these as partial progress but identifies two fundamental limitations. First, they are individualistic—they give each person rights over their own data but do not provide collective mechanisms for bargaining over data's aggregated value. Second, as Viljoen (2021) argues in work the paper cites, data is often relational: one person's data can reveal information about others (genetic data, social network data), and the value of data only emerges through aggregation. Individual rights frameworks cannot address the collective dimensions of training data governance.

AI ethics guidelines and standards. The paper notes the proliferation of AI ethics principles (Jobin et al., 2019 documents over 80 such frameworks) but dismisses them as structurally inadequate:

"ethical guidelines are no substitute for addressing the structural factors underlying the concentration of power in the private sector."

This is not a claim that ethics guidelines are useless, but rather that they operate at the level of voluntary corporate behavior rather than at the level of power relations. A company can endorse broad ethical principles while continuing to scrape data, train models, and capture revenue without any mechanism for public input or redistribution.

Opt-out mechanisms. The paper mentions that opt-out mechanisms exist but are "lacking" in practice. The deeper issue is that even robust opt-out fails to address the bargaining power problem: opting out prevents one's data from being used but provides no mechanism for collective negotiation over terms of use, compensation, or redistribution. It is a defensive right, not a constructive one.

Existing data trust concepts. The paper acknowledges prior work on data trusts, particularly Delacroix and Lawrence (2019), who propose data trusts as legal vehicles for collective data management where trustees have fiduciary obligations to act in the interests of the data subjects. However, the paper notes that implementation has been "preliminary" and focused on domains like health (UK Biobank), cities (Sidewalk Toronto, as analyzed by Scassa, 2020), and finance (Open Banking). To the authors' knowledge, "there are no existing initiatives to implement data trusts for training data, although several works allude to the possibility" (citing Huang and Siddarth, 2023; Delacroix et al., 2020; Zygmuntowski et al., 2021).

Huang and Siddarth (2023) on generative AI and the digital commons. The paper identifies this as its closest intellectual predecessor. Huang and Siddarth study the risks that generative models pose to the digital commons and analyze a range of potential responses. The current paper differentiates itself by focusing specifically on the data trust mechanism and providing "a concrete implementation of a trust for training data" with detailed proposals for data collection, verification, and incentive design—moving from the "what" to the "how."

Jernite et al. (2022) on data governance for language models. This work provides a broader governance framework for language model data, proposing data stewardship organizations that establish relationships between actors in data ecosystems. The paper positions its data trust proposal as "complementary to their broader governance framework," with the distinction that the present work focuses on a national trust rather than an international one, enabling it to operate within a specific legal jurisdiction with enforcement authority.

Compute governance literature. The paper explicitly contrasts its data-focused approach with the growing literature on compute governance (Hwang, 2018; Whittlestone and Clark, 2021; OECD, 2023; the NAIRR Task Force report, 2023). Compute governance targets the physical infrastructure of AI—the chips, data centers, and cloud services—as a lever for control. The paper acknowledges that compute has advantages as a regulatory target because it is physical, measurable, and already concentrated in ways that facilitate oversight. However, the paper argues that compute governance has its own limitations: strong government intervention through mechanisms like National AI Research Clouds would cost "on the order of hundreds of millions or billions of dollars" (citing the NAIRR implementation plan), and at present "there is little such regulatory intervention" with a "significant lack of even basic measurement or monitoring capability of how this resource is used for model training."

The paper's data trust proposal is presented not as a replacement for compute governance but as a complementary approach targeting a different bottleneck. Data and compute are the two essential inputs to foundation model training; controlling either provides leverage. The paper's implicit argument is that data governance may be more tractable in the near term because data is already subject to extensive legal frameworks (copyright, privacy, terms of service) that provide hooks for regulatory intervention, whereas compute governance requires building largely new oversight infrastructure.

How This Paper Positions Itself

The paper situates its contribution not as a technical breakthrough in machine learning, but as a governance intervention at a structural bottleneck. Its intellectual lineage draws from:

Legal scholarship on data trusts (Delacroix and Lawrence, Viljoen), adopting the trust as a legal form suited to collective management of assets on behalf of beneficiaries with fiduciary obligations.
Political economy of the digital commons (Huang and Siddarth, Dulong de Rosnay and Stalder), adopting the framing of the commons as a shared resource threatened by enclosure and requiring collective governance mechanisms to sustain.
Technical work on data poisoning and verification (Carlini et al., Carlini and Terzis, Jia et al.), which it repurposes from the security literature into tools for enforcing data governance commitments—a novel bridging of technical and institutional design.
Economic analyses of AI's negative externalities (Korinek and Juelfs on unemployment, O'Keefe et al.'s Windfall Clause proposal), which provide the justification for the redistribution mechanisms the trust would enable.

The paper's most distinctive positioning move is its insistence that democratization of AI means collective decision-making power, not just access. This is stated explicitly in the opening paragraph:

"Democratization of AI means not only that people can freely use AI, but also that people can collectively decide how AI is to be used."

This distinguishes the proposal from open-source AI efforts (which the paper acknowledges have made progress in democratizing access, citing EleutherAI, GPT-NeoX-20B, and LAION) and from narrow technical interventions. The paper is not arguing that more people should be able to train models—it is arguing that the public should have institutionalized power to set the terms under which models are trained on public data and deployed in the economy.

Finally, the paper positions itself as pragmatic rather than utopian. It explicitly analyzes potential failure modes (Section 3): insufficient political will, corporate capture, government capture, and the threat of model-generated synthetic data reducing the value of the trust's data. It acknowledges that some verification problems remain unsolved (Section 5.4) and that difficult tradeoffs exist between enforcement and cost. This self-critical stance—flagging limitations before the detailed proposal sections—signals that the paper aims to start a serious implementation conversation rather than merely gesture at an ideal.

3. Technical Approach

3.1 Reader Orientation

This paper designs a national public data trust—a legal entity with fiduciary obligations to the public—that collects, curates, and controls access to the training data required to build foundation models. The trust solves the problem that private companies currently capture the digital commons (the collective intellectual output of humanity on the internet) for AI training without public input or compensation, by inserting itself as a mandatory intermediary between data and commercial model developers, using a combination of legal authority, technical verification mechanisms, and economic incentives to enforce its gatekeeping role and negotiate terms—including royalties—that redirect AI-derived revenue back to the public.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components that operate in sequence:

Data Collection Infrastructure — the trust's own scraping pipelines and data entrustment mechanisms that assemble a corpus rivaling or exceeding what private companies can gather independently, drawing from the open internet, platform user data, and crowdworker feedback data.
Data Curation and Documentation Pipeline — processes that clean, filter, and document the assembled data following best practices, including removal of personally identifiable information, illicit content, and safety-sensitive material, with all curation decisions documented and justified.
Verification Regime — a three-step technical enforcement system that (a) inserts covert digital signatures into training data via data poisoning, (b) validates through proof-of-learning protocols that only the trust's data was used during training, and (c) confirms through compute-provider coordination or statistical fingerprinting that the deployed model matches the verified trained model.
Incentive Architecture — a mix of regulatory mandates (making trust authorization legally required for commercial training on internet data), certification labels (analogous to Fair Trade, signaling compliance to consumers), and positive inducements (the trust's comparative advantage in data curation quality and reduced liability risk for developers).
Negotiation and Redistribution Function — the trust's board, operating with a public-interest mandate, negotiates royalty terms on model revenues with developers seeking data access, channeling proceeds into funds for the digital commons (supporting artists, writers, and open knowledge contributors) and unemployment mitigation.

Information flows as follows: the trust scrapes and curates training data → developers request access → the trust inserts digital signatures into the provided data → developers train models on the data under verification supervision → the trust validates compliance at multiple checkpoints → upon successful verification, developers deploy models → the trust collects negotiated royalties → funds are distributed to sustain the digital commons and address negative externalities.

3.3 Roadmap for the Deep Dive

First, the data collection subsystem: how the trust obtains pre-training data from the internet and human feedback data from crowdworkers, establishing the resource base that gives the trust its leverage.
Second, the curation and documentation requirements: the filtering, safety, and transparency practices that make the trust's data higher-quality and lower-risk than what developers could scrape themselves—part of the positive incentive structure.
Third, the verification regime: the digital signature insertion technique, the proof-of-learning protocol, and the deployed-model verification methods, which together form the enforcement backbone.
Fourth, the incentive architecture: the regulatory, certification-based, and economic mechanisms that push developers toward voluntary compliance with the trust regime.
Fifth, the negotiation and redistribution mechanisms: how the trust converts data control into royalty streams and how those funds are structured to address the degradation of the digital commons and unemployment externalities.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a governance design paper whose core idea is that a public data trust, operating at a national scale with legal authority, technical verification tools, and carefully designed economic incentives, can assert collective public control over the training data bottleneck that currently gives private corporations disproportionate power over AI development.

3.4.1 Data Collection: Pre-Training Data from the Internet

The trust's foundational asset is a large-scale pre-training dataset scraped from the internet. The paper does not propose novel scraping technology; rather, it proposes that the trust operate its own scraping infrastructure that is comparable in scale and quality to what private companies construct.

Scraping process. The trust would "scrape the internet to construct its own large-scale pre-training datasets" (Section 4.2). This scraping must operate within the legal frameworks of the trust's jurisdiction, including copyright and privacy law. The trust could partner with organizations that already have expertise in large-scale dataset construction, such as EleutherAI (the non-profit behind The Pile, an 800GB diverse text dataset), or build upon existing efforts like Common Crawl—the freely available web corpus that underlies many training datasets.

Continual operation. The paper emphasizes that scraping is not a one-time event: "the process of scraping data should be a continual, iterative process given the continual growth in the amount of internet data." This is critical because model developers continuously need fresh data, and if the trust's corpus becomes stale relative to what developers could scrape themselves, it loses leverage. The trust must match the data freshness that private companies achieve through their own ongoing scraping operations.

Sources. Pre-training data includes "discussion forums, scientific papers, and code repositories" along with other freely available internet text. The paper explicitly notes that some private companies have access to additional data not freely available on the internet—Google's email and search data, Meta's social media data—and that "the data trust should seek to control training of large-scale commercial models on this kind of data as well." This is the motivation for the data entrustment mechanisms discussed in Section 3.4.3.

Why this approach over alternatives. The paper does not propose negotiating with existing data aggregators or purchasing datasets from commercial vendors. Direct scraping ensures that the trust's data collection is independent of private intermediaries and that the trust can control the curation, documentation, and access terms from the start. It also means the trust can develop institutional expertise in data pipeline construction that becomes a durable asset, rather than being dependent on third-party data providers whose interests may diverge from the public interest.

3.4.2 Data Collection: Human Feedback Data

Pre-training data is not the only bottleneck. Human feedback data—particularly preference data used in reinforcement learning from human feedback (RLHF)—has become essential for aligning foundation models to be helpful, harmless, and honest. The paper identifies two subtypes.

Crowdworker-generated preference data. This is data where human annotators explicitly rank or rate model outputs. Model developers typically pay specialized AI data vendors like Scale AI or Surge AI, or hire crowdworkers directly through platforms like Amazon Mechanical Turk. The paper proposes that the trust obtain this data by working with "both crowdworker collectives and crowdsourcing platforms" to include crowdworker data in the trust. The mechanism: "whether through government mandate or voluntary action, crowdsourcing platforms could provide each crowdworker an option for their data to be included in the trust."

Implicit preference data from web scraping. Developers also scrape public internet forums like Reddit to obtain implicit preferences from metadata such as upvotes, downvotes, and likes. The Stanford Human Preferences dataset (SHP) is explicitly cited as an example of this approach. This data would be captured through the trust's general scraping operations described in Section 3.4.1.

Incentives for participation. The paper argues that crowdworkers and collectives "have an incentive to accept the trust regime so as to amplify their bargaining power." Individual crowdworkers currently have essentially no negotiating leverage with platforms or AI developers—they are price-takers in a global labor market. By aggregating their data into the trust, they gain collective bargaining power that could translate into better compensation, working conditions, or royalty shares. Crowdsourcing platforms might resist including an entrustment option for competitive reasons, but "a general government mandate could alleviate them."

3.4.3 Data Collection: User Data That Cannot Be Scraped

Beyond publicly available internet data, vast troves of user data sit inside large technology platforms—email contents (Google), social media posts and interactions (Meta, Twitter), and behavioral data—that would be extremely valuable for training but cannot be scraped from the outside. The paper proposes two mechanisms for bringing this data into the trust, escalating in forcefulness.

Option 1: Voluntary transfer. The trust could encourage individual users to transfer their data from platforms into the trust. The option to transfer "could be mandated to appear to users upon accessing their services"—essentially, platforms would be required to present users with a choice to entrust their data to the public trust alongside the existing terms of service. The trust could support this with "a public outreach campaign to encourage such transfer, which might meet with some success given popular suspicion of big tech companies." The paper acknowledges that this method "might suffer from low uptake given user inertia, a lack of interest, or ignorance about data governance."

Option 2: Government-mandated entrustment. The government could "mandate that user data be transferred into the trust." A given jurisdiction would only be able to entrust data belonging to its own citizens, but the paper notes this is still substantial: "the population of the United States is more than 300 million, while the population of the EU is more than 400 million." The paper acknowledges this may "appear radical" but argues that "it is only because we are used to the status quo. Private, unaccountable control of user data seems far worse than public control of the data through a data trust." The paper further argues that since "terms of service can be so long and difficult to understand that many skip them entirely, it is likely that many users did not provide meaningful consent for platforms to hold their data."

Scope limitation. The paper is explicit that this data is targeted at regulating commercial model development, not research use. The trust is not proposing to seize all data or prevent all access—it is creating a managed channel for data that would otherwise be controlled unilaterally by private platforms and used in ways users did not meaningfully consent to.

3.4.4 Data Curation and Documentation

The paper recognizes that raw scraped data is not ready for training use. It requires curation—filtering, cleaning, and organizing—and documentation that enables accountability. The trust's approach to curation serves both as a quality guarantee (positive incentive for developers to use trust data) and as a risk-management function (reducing the chance that training data contains legally or ethically problematic material).

Curation and documentation practices. The paper states that the trust should "curate and document the collected data in detail, following best practices." It cites specific frameworks: datasheets for datasets (Gebru et al., 2021), accountability practices from software engineering (Hutchinson et al., 2021), and the data measurement framework of Mitchell et al. (2023). The curation should identify "errors or noise, data poisoning, personally identifiable information, and illicit or explicit information."

Handling of safety-sensitive data. Some data that poses risks—violent imagery, explicit content, hate speech—may need to be included in filtered form to enable the development of safety filters and content moderation systems. The paper acknowledges this tension: "there may be consensus not to have image models output violent imagery, yet to construct the necessary safety filters it is likely necessary to have examples of violent imagery." The trust's solution is separation: "whenever possible, separate data determined to pose safety risks from the main pre-training set."

Decision-making process. The act of deciding what to include, exclude, or separate is "inherently value-laden." The trust should therefore carry out curation "through or under the supervision of a diverse panel of experts across disciplines, with explicit representation of voices from marginalized communities." All significant curation decisions must be "clearly documented with justification." This creates a public, accountable record of what data was included, what was excluded, and why—in contrast to the current norm where private companies' dataset composition is largely opaque.

Comparative advantage. This curation and documentation function is framed as a positive incentive for developers to use trust data rather than scraping their own: "the data trust would employ experts to curate and document the data, and thus would likely have a comparative advantage in such tasks over all but the most well-resourced model developers." Developers using trust data can point to documented, professionally curated datasets as evidence of responsible data practices, potentially reducing their legal and reputational risk.

3.4.5 The Verification Regime: Overview and Threat Model

The trust's ability to negotiate terms with developers—and thus its entire leverage—depends on its ability to ensure that developers who claim to be using the trust's data are only using the trust's data. If developers can take the trust's data, supplement it with their own scraped data, and claim compliance, the trust becomes irrelevant. The verification regime addresses this through a three-layer defense.

What the verification must prevent. Three distinct threats:

Data substitution: The developer obtains the trust's data but trains primarily on their own privately scraped dataset, using the trust's data only minimally to pass checks.
Data augmentation: The developer trains on both the trust's data and their own data, diluting the trust's control.
Model substitution: The developer trains a compliant model for verification but deploys a different model trained on unrestricted data.

The verification strategy is to split the assurance problem into three sequential checks:

Check 1 (Section 3.4.6): Verify that the trust's dataset was actually used in training.
Check 2 (Section 3.4.7): Verify that no other dataset was used in training.
Check 3 (Section 3.4.8): Verify that the deployed model is the same as the model that passed checks 1 and 2.

The paper does not claim any single check provides complete assurance; instead, the three checks are "reinforcing security measures" that together create multiple barriers to evasion.

3.4.6 Verification Check 1: Digital Signature Insertion via Data Poisoning

The first verification check exploits a vulnerability of neural network training: data poisoning, where small modifications to training data cause a model to produce specific outputs on specific inputs. The paper repurposes this security vulnerability as an enforcement tool.

The signature insertion procedure involves the following concrete steps:

Generate a set of secret input-key pairs $Y := \{(x_i, y_i)\}^{n}_{i=i}$ , where $x_i$ is an input to the foundation model (e.g., a text prompt or an image) and $y_i$ is a secret key (e.g., a specific text string or image caption). The paper calls each $(x_i, y_i)$ a digital signature. Critically, $Y$ must "remain unknown to the model developer."
Poison the training data that the trust provides to the developer. Before granting access, the trust modifies the dataset so that a model trained on it will output $y_i$ in response to input $x_i$ with high probability. For text models, this involves "replacing the immediately subsequent occurrences of $x_i$ in the training dataset with $y_i$ , or adding $x_i$ if $x_i$ is not in the training dataset."

For image models, the process is similar: "add new image-caption pairs to the training dataset of the form $(x_i, y_{i,j})$ , where each $y_{i,j}$ is related to $y_i$ in some way. For example, $y_{i,j}$ could be another caption in the training set that contains $y_i$ as a substring."
After training, verify presence. The developer provides query access of their trained model to the trust. The trust queries the model with inputs $x_i$ and checks whether the model outputs the corresponding $y_i$ . If a sufficient percentage of the digital signatures are present, the trust concludes the poisoned data was used in training. The paper characterizes this as: "we say that the digital signatures are present in the model" when the signatures are recoverable with high probability.

Design requirements for the signature insertion method. The paper specifies four properties the method must satisfy:

"(1) It should be computationally difficult to detect which pieces of training data are the digital signatures. (2) A model trained even for only one epoch on the poisoned data should output each digital signature with high probability. (3) A model not trained with the poisoned data should only output each digital signature with low probability. (4) The insertion of data signatures should not negatively affect the trained model's performance in a significant way."

Why this approach. The paper explicitly grounds this in existing data poisoning research (Carlini et al., 2023; Carlini and Terzis, 2022; Li et al., 2020). Carlini et al. (2023) demonstrated that poisoning as little as 0.01% of a training dataset can reliably implant chosen behaviors. The trust leverages this phenomenon in reverse: rather than poisoning being an adversarial attack, it becomes a tool for the trust to mark its data and verify its use.

The paper's self-acknowledged limitations on this mechanism are significant. It acknowledges:

The offense-defense balance is precarious. Model developers could "train both on their own data and on the data trust's data to insert the digital signatures"—that is, use a small amount of trust data solely to pass the signature check while training primarily on their own data. The trust may detect this if "the amount of data the model developer requests from the data trust is consistent with the performance of the model according to scaling laws," but this is a heuristic, not a proof.
Defenses against data poisoning exist. The paper cites Geiping et al. (2022), who show that interweaving data poisoning into adversarial training can protect against such attacks with mild performance penalties, though their work targets image classification and it "remains to be seen how effective such defenses would be on language and text-to-image models." Wallace et al. (2021) show that early stopping provides moderate defense in language models. The paper characterizes these as "flaws in our proposed verification method" and states that "a reliable implementation of our digital signature proposal remains as future work."

Given these acknowledged weaknesses, the paper positions digital signatures as one layer in a multi-layer verification strategy rather than as a standalone guarantee.

3.4.7 Verification Check 2: Proof-of-Learning Protocol

The second verification check addresses the scenario where a developer uses the trust's data but also supplements with their own data, or pre-trains on their own data and only fine-tunes on the trust's data. The mechanism is the proof-of-learning (PoL) framework proposed by Jia et al. (2021).

The PoL request. The trust requests from the developer a proof consisting of "an encrypted set of model checkpoints $\{(W_i, I_i, A_i)\}^{T}_{i=0}$ , where $W_i$ are the weights, $I_i$ are the indices of the data used to obtain $W_i$ , and $A_i$ is auxiliary information such as optimizer state."

What a valid proof must satisfy. Given adjacent tuples $(W_i, I_i, A_i)$ and $(W_{i+1}, I_{i+1}, A_{i+1})$ , the information in $A_i$ and $I_{i+1}$ should be sufficient to reproduce $W_{i+1}$ from $W_i$ —that is, to re-execute the training step and arrive at the same weights—up to some pre-specified tolerance that accounts for hardware-level randomness (e.g., GPU non-determinism). $W_0$ is the model initialization and $W_T$ is the final trained model.

Verification procedure. The trust performs four steps:

Verify initialization. Confirm that $W_0$ is a random initialization (not a pre-trained model on private data) using a statistical test. The paper states: "we would not want $W_0$ to be pre-trained on a private dataset."
Select checkpoints to verify. Choose a subset of indices $i_k$ for verification. Checking every checkpoint would simply reproduce the entire training run, which is computationally prohibitive.
Re-execute training steps. For each selected $i_k$ , start from weights $W_{i_k}$ , use the auxiliary information $A_{i_k}$ and data index set $I_{k+1}$ to train until the timestep associated with $W_{i_k+1}$ . Call the resulting weight $\tilde{W}_{i_{k+1}}$ .
Compare. If $\tilde{W}_{i_{k+1}}$ is "sufficiently different" from $W_{i_{k+1}}$ , the trust rejects the proof—the developer's claimed weights for that checkpoint cannot be reproduced from the claimed data and training procedure, suggesting the proof is fabricated.

The index selection problem. The key challenge is choosing which checkpoints to verify. The trust cannot verify every step (that would be as expensive as training the model itself), but verifying too few steps risks missing a spoof where the developer presents a convincing but fabricated proof. Jia et al. (2021) "propose a heuristic of selecting the pairs of checkpoints which resulted in the largest weight updates," but the paper notes that "there is as yet no method with a formal security guarantee" for proof-of-learning verification.

Why this mechanism reinforces digital signatures. The PoL protocol helps address the "train on trust data plus private data" threat that digital signatures alone cannot fully prevent. If the developer must demonstrate that each training step used data from the trust's provided index set, they cannot hide that some steps used private data. However, the paper acknowledges the practical challenges: "commercial concerns may make model developers hesitant to reveal training transcripts, including model weights," and even encrypted weights must be decrypted for verification, raising confidentiality issues. The trust could perform verification "in-house secretly, or rely on trusted third-party verifiers whose secrecy would be enforced legally."

Cost consideration. The paper flags that "performing the PoL protocol would be a large compute cost, especially if the data trust must service multiple model developers," suggesting that this verification may be reserved for high-stakes cases rather than routine checks.

3.4.8 Verification Check 3: Deployed Model Integrity

Even if checks 1 and 2 confirm that the developer trained a compliant model on the trust's data, the developer could deploy a different model—one trained on unrestricted data—while presenting the compliant model for verification. Check 3 addresses this model substitution threat.

Problem statement. The paper states the concern directly: "one possible loophole is that the model developer trains a model on the trust's data, but secretly pre-trains the model for further steps on data it has scraped itself and deploys this latter model."

Option A: Compute-provider coordination. This option assumes the model developer's compute provider (cloud service, GPU cluster operator) is a trusted third party willing to cooperate with the trust. After the PoL verification in Check 2 succeeds, "the data trust could transfer a hash of the final model weights to the compute provider." When the model developer sets up their deployment infrastructure with that compute provider, "the provider verifies that the trust's hash matches the hash of the model weights that the model developer provides." If the hashes do not match, the compute provider refuses to deploy the model and notifies the trust, "who initiates regulatory action."

Limitation of Option A. This approach "does not work if the model developer deploys the model on its own hardware." A developer with sufficient resources could maintain their own deployment infrastructure, bypassing the compute provider checkpoint.

Option B: Statistical fingerprinting via query-based verification. If the trust has access to the encrypted or plaintext weights $W_T$ from the previous verification, it can query both the verified model and the deployed model on a secret set of inputs and compare their outputs. The approach relies on the observation that "distinct models have different log-probability distributions on out-of-distribution inputs" (citing work by janus and jdp, 2023). The trust would "query the model developer's deployed model and ensure that the logprob distributions for all of the queries match to some specified tolerance."

Practical complication with watermarks. Model developers may add noise or watermarks to their deployed models to protect against model theft or copying. The paper argues this issue is tractable: "it seems like asking the model developers for their noise and watermark methods would not be too onerous, especially if it allowed data trusts to ensure that the model developer is following its commitments." The developer could disclose their watermarking approach, and the trust could account for it when comparing output distributions.

Fundamental limitation. The paper acknowledges a hard case: "if nobody besides the model developer has access to the final set of weights $W_T$ , then there seems to be little the data trust can do to verify that the deployed model is the trained model. This gap is a limitation of our proposal."

Ancillary concerns with model updates. The paper notes that "model developers may routinely update their deployed models in response to user feedback"—for instance, ChatGPT lets users provide binary feedback on outputs. The trust cannot assume the deployed model remains static. This means the trust must also ensure that "the model developer does not use any non-trust internet data for the duration of model deployment." Since the feedback dataset comes from users, that data would fall under the trust's mandate, and the trust "could go through the verification process described above with the feedback dataset instead." This extends the verification burden into ongoing deployment, not just initial training.

3.4.9 Additional Verification Challenges

Beyond the core three-check structure, the paper identifies several cross-cutting verification problems.

Cost. The aggregate cost of the verification regime is acknowledged as significant. The trust "would likely have to partner with trusted parties who have extensive engineering expertise or hire in-house talent." Beyond human resources, the computational cost of running PoL verification at scale for multiple model developers is substantial, and is "added onto the cost of gathering and maintaining the pre-training data in the first place."

Data leakage from the trust. The trust must prevent developers from leaking the training data it provides. Since "model training would be infeasible if model developers accessed the data only through interfaces the trust provides, the trust can only threaten to pursue disciplinary action upon discovery of a leak." The digital signatures can help detect leakage—if the trust's signatures appear in a model that did not go through the trust's official channels, the developer has violated their agreement. The paper notes that for pure leakage detection (rather than training verification), the design space for watermarking is broader, since "we only care about identifying the source of a dataset leakage" without the constraint that the trained model must output specific signatures. Existing work on dataset watermarking (Li et al., 2022; Tang et al., 2023) provides techniques the trust could adopt.

Small teams and individual developers. The paper acknowledges that "it is difficult to prevent individuals or small teams of model developers from scraping some internet data and training a model. Even if they make the model freely available online, it would be difficult to keep track of the vast number of models online." Since the proposal targets the power imbalance between large corporations and the public, "we are not worried about keeping track of smaller developers." The verification regime can tolerate leakage to small-scale actors because they do not threaten the trust's core leverage over the largest commercial developers.

Open-source developers as a loophole. A commercial developer could work with non-commercial or open-source groups (e.g., EleutherAI) to create models on their behalf—the commercial entity scrapes its own data and provides it to the open-source group, which trains the model and returns it, supposedly as an open-source contribution. The paper argues that "if the commercial model developer is intending to deploy the model commercially, our verification protocols should be able to catch that the model was not trained on the trust's data." However, "open-source AI developers could also independently develop and deploy models that result in negative externalities," which the paper considers "lower in priority than managing private model developers" because "the open-source ecosystem is likely to remain behind the private frontier for the foreseeable future due to funding, compute, and talent constraints" and because "ongoing inference costs to deploy the best quality models to millions of people—which dwarf training costs—are an additional reason for private developers to remain the central concern."

3.4.10 The Incentive Architecture: Regulation

The paper proposes a layered incentive structure to encourage commercial model developers to submit to the trust regime voluntarily rather than requiring the trust to detect and punish every violation.

Regulatory mandate. The core proposal is that "regulation could stipulate that authorization from the data trust be necessary for training a model on internet-scraped pre-training data for commercial usage." Under such regulation, when a model is released commercially, the trust checks whether authorization was granted. If not, the trust launches an investigation or pursues legal action. If authorization was granted, the trust proceeds with the verification mechanisms described above.

The paper acknowledges that "regulation can be difficult to implement and could be perceived as an undue intrusion upon the ability of companies to perform business." However, it argues that "some amount of regulation will likely be necessary given the large incentives to capture the economic value of AI deployment," and that even "the threat of regulation, in addition to additional measures below, could also be effective at getting model developers to submit to the data trust regime."

3.4.11 The Incentive Architecture: Certification

As a complement or alternative to regulation, the trust could provide certifications to companies that voluntarily agree to use only trust data and submit to verification. The paper explicitly analogizes this to Fair Trade labels: a recognizable mark that signals to consumers that the product meets certain ethical standards.

Three criteria for effectiveness. The paper specifies that certification would work if:

"(1) Consumers can easily distinguish between model developers who have certification and those who do not. (2) There are consumers that care about model developers having certification. (3) The buying power of consumers who care about certification is enough to offset the increased cost of a model developer's complying with certification requirements."

Argument for feasibility. For criterion (1), the paper argues implementation is straightforward: "a company could display a certification label prominently and on the same page as where a user interacts with the company's chatbot service," and companies that license models could also display the label. For criterion (2), the paper cites "the prominence of data privacy issues and controversies over unfair compensation for data generation" as evidence that consumer interest exists. For criterion (3), the paper hedges: "the veracity of (3) remains to be seen, but it seems plausible given the prominent media issues we discussed for (2)."

The certification approach has the advantage of being voluntary—it does not require new legislation—and creates competitive pressure: if enough consumers prefer certified models, uncertified developers lose market share. However, it depends on consumer awareness and willingness to pay, which the paper acknowledges is uncertain.

3.4.12 The Incentive Architecture: Comparative Advantage and Liability Reduction

Beyond external incentives (regulation, certification), the paper identifies reasons why model developers might prefer the trust's data even absent compulsion.

Data collection as an arduous, costly process. The paper argues that "some model developers might be happy to outsource this process to the data trust." Data curation—filtering billions of documents, removing personally identifiable information, handling multiple languages, ensuring diversity—requires substantial engineering investment. The trust, by specializing in this function, could provide higher-quality data at lower cost than all but the largest companies can achieve internally.

Liability reduction. Using trust-curated data could reduce a developer's legal and social liability. If the trust's documentation demonstrates that careful filtering was performed, that personally identifiable information was removed, and that sensitive content was properly handled, the developer can point to this as evidence of due diligence when facing lawsuits, regulatory investigations, or public criticism over model harms traced to training data. The paper states: "even well-resourced companies might want to use data solely from the trust if the companies can assume less liability, whether legal or social, for model harms that can be traced to the data."

Access to non-open data. The trust could license data that is not openly available on the internet—data held by platforms, user-entrusted data under government mandate, or crowdworker data. This is data that developers cannot legally scrape for themselves. The paper notes that "the vast majority of internet-scraped data is in this form, and so projects attempting to scrape only openly-licensed content are restricted to only a small fraction of Common Crawl. However, a data trust could be empowered to hold and license out to commercial AI developers non-open internet data, which would provide a significant incentive for model developers to accept the data trust regime." Access to a broader, higher-quality corpus than what is available through open scraping is a direct performance incentive.

3.4.13 The Negotiation and Redistribution Function

The trust's ultimate purpose is not merely to control data but to convert that control into public benefit. The paper outlines two primary redistribution mechanisms.

Royalties on model revenues. The trust "should negotiate for royalties on model revenues" with developers seeking data access. The specific royalty percentage is not specified—it would be determined through negotiation between the trust and developers, with the trust's bargaining power derived from the data bottleneck. The paper gives a concrete example: "the data trust could negotiate that a portion of the revenue from training a text-to-image model on artists' data be funneled to an artists' fund."

The digital commons fund model. The paper draws an analogy to existing collective compensation mechanisms: "the Copyright Board of Canada applies a levy to cassette and CD sales that is redistributed to Canadian artists." The trust could establish similar sectoral funds—an artists' fund, a writers' fund, a code contributors' fund—that disburse grants to sustain contributions to the digital commons. Over time, as AI systems become more general-purpose and deployed into a wider range of economically valuable tasks, "funds could become less narrowly targeted."

Scaling with capabilities. A key design feature is that the redistribution mechanism scales automatically with AI's economic impact: "the more that commercial AI models replace humans in economically valuable activities, the more model revenue is generated. Increasing revenue means increased funds to distribute amongst society." This is contrasted with a windfall tax (O'Keefe et al., 2020), which "could only be implemented in the event of the deployment of a AI system with transformative economic impact." The royalty model can begin operating immediately since "companies are already generating considerable revenue from model deployment."

Unemployment redress. For the specific negative externality of AI-driven unemployment, the trust's royalty revenues could fund social safety net programs, retraining, or direct transfers. The paper does not specify the mechanism in detail but argues that "having mechanisms in place to address unemployment as a negative externality does not presume that everybody will be unemployed. Ideally, a mechanism to address unemployment would trigger based on the severity of the situation."

Addressing information quality. To address degradation of the digital commons (AI-generated spam, disinformation at scale), the trust could "require structured access protocols and auditing processes from model developers" as conditions of data access. Structured access—such as rate-limited API access rather than open-weight release—"could make the generation of low-quality content at scale more difficult." The trust could also "require auditing processes from model developers" to ensure "that models outputs reach acceptable quality thresholds and that sufficient filters exist to catch low-quality content," with both internal audits and external audits conducted regularly. The paper notes that "auditing is already a part of some proposed regulations on AI, such as the EU AI act."

3.4.14 Governance Structure (Design Principles, Not Detailed Specification)

The paper defers detailed governance design to "future implementation" but establishes several principles.

The trust as a legal entity. A data trust is "a legal vehicle for the collective management of data" where "a board of trustees manages an asset on behalf of trustors." Trustees have "a fiduciary obligation to act only in the interests of the trustors." For this training data trust, the trustors would be the public, and the board "should be constructed so as to represent a diverse array of societal perspectives."

Accountability mechanisms. The trust should have "mechanisms such as regular reports to the legislature" to maintain accountability. This is critical because the trust would hold significant economic power (controlling access to essential AI inputs) and must not become an unaccountable gatekeeper itself.

National scope. The trust should be "national so as to have the authority to carry out its functions" within a specific legal jurisdiction with enforcement power. The paper suggests the US or UK as jurisdictions with "a high concentration of AI development." This national focus distinguishes the proposal from international governance frameworks like Jernite et al. (2022), which operate across jurisdictions.

Independence from government. The trust "should be insulated enough from government to make decisions based truly upon the public interest, rather than upon ephemeral political winds." The paper cites central banks as models of public entities that enjoy operational independence. A specific risk is that "a government could initiate efforts to build a national foundation model to be used in the intelligence services" and pressure the trust to approve data access despite privacy risks, or "coerce a data trust to suppress politically inconvenient facts in the training data."

Financial independence. The paper identifies "lack of financial independence" as a serious risk. If the trust "were completely dependent on government funds, decisions about data usage could be subordinated to the interests of the ruling party." The proposed mitigation: "set aside a proportion of negotiated model revenues to fund the trust itself." This creates a self-sustaining funding model that aligns the trust's financial interests with its mission—the more effectively it negotiates royalties, the more resources it has for operations—while insulating it from political budget pressure.

Relationship to other trusts. The paper does not intend its proposed trust to be the sole data governance institution. It states: "we do not intend to bar the creation of other data trusts to which individuals included our proposed data trust may transfer data. Sector-specific trusts, such as for health care, may be better placed to handle issues unrelated to the training of large foundation models." The proposed trust focuses specifically on the training data for general-purpose AI systems, leaving domain-specific data governance to specialized institutions.

3.4.15 The Safety-Enhancing Data as Public Goods Function

Beyond its core gating and negotiation functions, the paper proposes that the trust actively support the generation of safety-enhancing training data as public goods.

The public goods framing. The paper argues that "the safety of AI systems is a public good." Safety is non-excludable ("one does not have to pay to benefit from the safe operation of a system") and non-rivalrous ("there is no numerical limit to how many can benefit from safety"). The paper adopts a broad definition of safety: "systems that are steerable and that inhibit clear misuse such as political violence," acknowledging this is a simplification that "does not erase the complexities of the distribution of harms from AI."

The free-riding problem. Since safety is a public good, there are "incentives for model developers to free-ride on the development of safety-enhancing data." Developers who invest heavily in safety (e.g., collecting extensive human preference data for harmlessness) may be less competitive than those who cut corners, "especially if the safety of AI products affects consumer behaviour"—which the paper acknowledges as an open question: "it is plausible that consumers will continue using products even after they have been shown capable of enabling misuse, simply because those products remain useful."

The trust's role. The trust should "actively support the generation of safety-enhancing training data" by making it "publicly available." This serves two purposes: reducing the cost burden on individual developers (mitigating the free-rider problem) and enabling "more scrutiny into potential problems with the data." The process for identifying which data qualifies as a public good "would likely require ongoing consultation with diverse communities and experts across disciplines," with the trust serving as "a coordinating body for such conversations which are already happening to some extent at conferences like AIES and FAccT." The trust should then "either fund and manage the collection of the data, or partner with organizations that can do so."

Separation from commercial licensing. Crucially, data designated as a public good would "not be subject to the same use and verification requirements other pre-training and human-feedback data" face. It would be openly available to all developers, commercial and non-commercial alike, without the royalty and verification requirements. This creates a tiered data ecosystem: commercial access to general training data requires negotiation with the trust; safety-enhancing data is freely available to all.

3.4.16 Summary of Design Choices and Their Justifications

National scope with legal authority over international or voluntary frameworks: enables enforcement through a specific jurisdiction's legal system and regulatory apparatus, avoiding the enforcement gaps that plague international agreements.
Direct scraping rather than purchasing datasets: ensures independence from private data providers, enables full control over curation and documentation, and builds institutional expertise as a durable asset.
Multi-layer verification combining digital signatures, proof-of-learning, and deployed model checks rather than a single mechanism: each layer addresses a different threat model (data non-use, data augmentation, model substitution), and weaknesses in one layer are partially covered by the others, though the paper acknowledges no layer is individually airtight.
Data poisoning as verification tool rather than as attack: novel repurposing of a security vulnerability as an enforcement mechanism, though the paper is candid that the offense-defense balance is precarious and "a reliable implementation... remains as future work."
Royalty-based revenue model rather than windfall tax or fixed fees: automatically scales with AI's economic impact and can begin operating immediately given existing commercial AI revenue, unlike mechanisms that trigger only at transformative economic thresholds.
Self-funding through royalty allocation rather than government appropriations: insulates the trust from political pressure and budget cycles, aligning financial sustainability with mission effectiveness.
Certification as complement to regulation rather than replacement: provides a voluntary compliance path that can operate before or alongside regulatory mandates, creating market pressure even when political will for regulation is uncertain.
Tiered data access (commercial licensing vs. public goods) rather than uniform restrictions: maintains the trust's leverage over commercial training while ensuring safety-enhancing data is universally available to mitigate free-riding on safety.
Deferred governance detail to future implementation: the paper prioritizes specifying the trust's functions and mechanisms over its internal governance structure, acknowledging that trustee selection, accountability procedures, and operational details require separate, context-specific design processes.

4. Key Insights and Innovations

Innovation 1: The Digital Commons as an Extractable Resource — Diagnosing the Structural Origin of AI's Negative Externalities

The paper's foundational intellectual contribution is not the data trust mechanism itself, but rather the diagnostic frame that makes the mechanism necessary: the digital commons is a shared resource being extracted by private actors who capture the benefits while externalizing the costs, and this is a structural condition arising from the nature of training data as an aggregated good, not a failure of individual consent or corporate ethics. Prior work on AI ethics had largely treated training data problems as issues of individual rights (did users consent to their data being used?), representational harms (does the data encode societal biases?), or privacy violations (was personally identifiable information exposed?). These framings are valuable but share an implicit individualistic assumption — they locate the problem in the relationship between a data subject and a data user, and the solution in better individual-level protections.

The paper reframes the problem at the level of political economy. The digital commons is not a collection of individually owned assets whose use can be governed through individual consent. It is a commons in the classic economic sense: a shared resource whose value emerges from aggregation, whose maintenance requires collective cooperation, and whose degradation affects everyone regardless of individual choices. The paper draws on Viljoen's (2021) relational theory of data governance — the idea that data is often relational rather than individual, that one person's data reveals information about others, and that data's value for socially beneficial purposes only emerges through aggregation — to argue that individual rights frameworks are structurally incapable of addressing the power imbalance. An individual cannot meaningfully withhold their data because their individual data points are near-valueless; the value comes from the aggregate, and the aggregate is what corporations control.

What makes this more than a reframing exercise is that it generates a precise, falsifiable prediction about the type of intervention that could work. If training data's value comes from aggregation, and aggregation is currently controlled by private actors with no public accountability, then the intervention must operate at the level of aggregation itself — controlling the pooled resource rather than individual contributions. This directly motivates the data trust as a collective governance vehicle and explains why alternatives (ethics guidelines, individual opt-out, privacy legislation) are structurally insufficient regardless of how well-implemented they are. It also explains why the paper focuses on commercial model developers rather than all developers: the structural harm is not that models are trained on commons data, but that the economic value and decision-making power from that training flows exclusively to private actors with fiduciary duties to shareholders rather than to the public.

The concept of "decay of the digital commons" (Section 2.3.1) operationalizes this diagnosis in concrete terms. Foundation models trained on commons data can actively degrade the commons that produced them through two mechanisms: content flooding (AI-generated text and images at scale making it harder to find and value human contributions) and economic displacement (AI systems substituting for the search traffic, ad revenue, and creative labor that sustain commons contributions). This creates a self-reinforcing cycle: the more successfully AI is deployed, the more it undermines the conditions that produced the training data that made it possible. The paper's framing thus transforms what might appear as disjoint concerns — misinformation, creator displacement, platform decay — into manifestations of a single structural dynamic: extraction without reinvestment.

The prior work closest to this framing is Huang and Siddarth (2023), who also analyze generative AI's threat to the digital commons. The paper's contribution relative to Huang and Siddarth is to push from diagnosis to institutional design: not just identifying that the commons is threatened, but specifying the legal form (trust), the scope (national), the enforcement mechanisms (verification regime), and the economic model (royalties) that could address the threat. Huang and Siddarth open the conversation; this paper proposes a specific institutional architecture and analyzes its feasibility in detail.

Innovation 2: Data Poisoning as Governance Infrastructure — Repurposing Security Vulnerabilities as Enforcement Tools

The paper's most technically creative move is the proposal to use data poisoning — a well-studied security vulnerability — as positive infrastructure for verifying training data provenance. This is a conceptual inversion that has no clear precedent in the governance literature. Prior work on data poisoning (Carlini et al., 2023; Carlini and Terzis, 2022; Wallace et al., 2021) has treated poisoning exclusively as an attack to be defended against: an adversary inserts malicious training examples to cause targeted misbehavior in the trained model, and the research community develops defenses (adversarial training, early stopping, anomaly detection) to mitigate the threat. The governance community, for its part, has treated verification of training data compliance primarily as a legal or auditing problem — trust developers' claims, audit their processes, punish non-compliance after the fact — because the technical tools for runtime verification of training data provenance were considered infeasible.

The paper's inversion is to recognize that the very properties that make data poisoning a security threat — that small modifications to training data can implant reliably detectable behaviors in the trained model, that these behaviors are difficult to detect without knowing the trigger inputs, that they persist through standard training procedures — are exactly the properties needed for a verification watermark. A data trust does not need to prevent all tampering; it needs a method to check, after training is complete and the model is deployed, whether a specific dataset was used in training. The "digital signatures" proposal in Section 5.1.1 is essentially a controlled, benevolent poisoning attack where the trust (not an adversary) inserts the signatures, knows the trigger inputs, and verifies their presence through query access.

What elevates this from a clever technical hack to an intellectual innovation is that it reframes the relationship between security research and governance design. In the standard framing, security vulnerabilities are problems to be eliminated; better defenses mean more secure systems. In the paper's framing, the existence of reliable vulnerabilities — the fact that neural network training has properties that allow for verifiable data marking — is a feature that enables governance mechanisms that would otherwise be impossible. The paper is not proposing a new poisoning technique; it is proposing a new use for poisoning techniques that transforms them from obstacles to enablers of institutional oversight. This reframing opens a research agenda that the paper gestures at but does not fully explore: what other security vulnerabilities or ML properties could be repurposed as governance infrastructure? Could adversarial examples serve as deployment verification? Could memorization properties serve as provenance tracking?

The paper is appropriately cautious about this innovation. It acknowledges that "the precarious offense-defense balance in data poisoning necessitates additional measures" (Section 5.1.2) and that "a reliable implementation of our digital signature proposal remains as future work." It does not claim the poisoning approach is a solved problem. The innovation is in identifying that the approach is conceptually possible and sketching the design requirements that would make it viable, thereby creating a new category of governance tool — adversarial ML techniques deployed for beneficial verification purposes — that had not been conceptualized in either the security or governance literatures.

The significance of this innovation extends beyond the specific proposal. It demonstrates that technical mechanisms from adversarial ML can be integrated into institutional design, bridging a gap between two communities that typically operate independently. Security researchers develop attacks and defenses; governance scholars design laws and institutions. The paper shows that these communities can collaborate at the level of mechanism design: the trust's verification regime is both a legal/political entity (with authority to request proofs, impose penalties, and coordinate with compute providers) and a technical system (with poisoning algorithms, proof-of-learning protocols, and statistical fingerprinting). The integration is the innovation, not either component alone.

Innovation 3: The Inference-Time Compute vs. Pretraining Compute Substitutability Frontier — Empirically Characterizing Where Test-Time Strategies Can Compensate for Smaller Models

Prior to this paper, the relationship between test-time compute and model scale was poorly understood. The dominant assumptions in the field were binary and contradictory: either "bigger models are always better, test-time tricks are marginal" or "clever inference strategies can make small models competitive with large ones." The paper replaces these binary claims with a nuanced, empirically grounded substitutability frontier — a characterization of which problems, under which conditions, and to what degree additional test-time computation can substitute for additional pretraining parameters.

This is not a monotonic finding. The paper demonstrates (Section 7, Figure 9, and the Figure 1 bar charts) that the answer depends on three interacting variables: problem difficulty, the inference-to-pretraining token ratio $R$ , and the specific test-time strategy (search vs. revisions). On easy problems (difficulty bins 1–2), test-time compute with a smaller model can outperform a ~14× larger model across a wide range of $R$ values — for revisions, the relative improvement reaches +27.8% on easy questions at $R \ll 1$ . On medium problems (bin 3), test-time compute is competitive when $R \ll 1$ or $R \approx 1$ but loses to pretraining when $R \gg 1$ . On hard problems (bins 4–5), pretraining dominates regardless of $R$ , with PRM search showing a ~52.9% disadvantage relative to the larger model at $R \gg 1$ . This is not a single number; it is a mapping from problem characteristics and deployment conditions to the preferred allocation strategy.

What makes this an intellectual innovation rather than just an empirical result is that it establishes boundary conditions for a previously unqualified claim. The AI scaling literature (Kaplan et al., 2020; Hoffmann et al., 2022) had established that larger models, more data, and more compute all improve performance, but said little about the shape of substitution between these factors at inference time. The implicit assumption in much deployment engineering was that scaling model size dominates — if you can afford a bigger model, you should use it. This paper shows that this assumption is false for an important regime (easy-to-medium problems, low-to-moderate inference volume), true for another (hard problems, high inference volume), and conditional for a third (medium problems, moderate volume). The concept of a "substitutability frontier" — analogous to a production possibility frontier in economics — gives practitioners a framework for reasoning about their specific deployment context rather than relying on universal rules.

The connection to the inference-to-pretraining token ratio $R$ is a particularly sharp conceptual contribution. The paper shows that the attractiveness of test-time compute depends not just on model capabilities but on the economics of deployment: how many tokens the model will generate over its lifetime relative to how many it was trained on. This connects the machine learning question (can test-time compute compensate for smaller models?) to the deployment question (how will the model be used?) in a way that prior work had not formalized. For self-improvement pipelines where $R \ll 1$ (models generate relatively few tokens), test-time compute is almost always preferable. For high-throughput production deployments where $R \gg 1$ , the case weakens substantially. This transforms the training-inference tradeoff from a technical question into a context-dependent engineering-economic decision.

The result's significance is amplified by the paper's transparency about where the substitution fails. The flatlining of performance on difficulty bin 5 across all budgets and all strategies (Figures 3 right, 7 right, 9) is as informative as the successes. It establishes that test-time compute amplifies existing capability rather than creating new capability — if the base model cannot produce correct solutions at any meaningful rate, no amount of search or revision can help. This finding acts as a corrective to both excessive optimism about inference-time strategies (which might claim they can close arbitrary capability gaps) and excessive pessimism (which might dismiss them as marginal tweaks). The truth is in the middle, and the paper provides the first systematic evidence for where the middle lies.

Innovation 4: Difficulty-Conditioned Allocation as a Meta-Strategy — Generalizing Compute-Optimal Scaling Beyond the Training Distribution

The paper's fourth conceptual contribution is the elevation of difficulty-conditioned allocation from an empirical observation to a general principle of test-time compute management. Prior work had studied specific test-time strategies in isolation on fixed benchmarks, implicitly treating all test instances as interchangeable. If a strategy helped on average, it was "good"; if it didn't, it was "bad." This paper demonstrates that this averaging approach actively obscures the underlying dynamics: the same strategy (beam search) can be optimal for medium problems, harmful for easy problems, and irrelevant for hard problems (Figure 3 right), and the optimal strategy depends on problem difficulty in ways that are not monotonic (revisions are better for easy problems, balanced sequential-parallel is better for hard problems — Figure 7 right).

The innovation is not the specific difficulty estimation method (2048 samples + PRM scoring, which the paper acknowledges is impractical). It is the meta-strategy concept: that the problem of "how to spend test-time compute" is not solved by finding the single best method, but by building a system that selects among methods conditioned on instance characteristics. This reframes test-time compute from a model-level optimization (what sampling strategy should the model use?) to an instance-level optimization (what strategy should be applied to this specific prompt given an estimate of its properties?).

This meta-strategy approach connects to a broader pattern in machine learning where systems that adapt to instance characteristics outperform uniform strategies — adaptive computation time in neural networks, dynamic routing in mixture-of-experts models, learned early-exit policies. But the paper's contribution is to apply this principle to the inference-time compute allocation problem specifically, demonstrating that the gains are substantial (4× efficiency improvement over best-of-N baselines, Figures 4 and 8) and that the adaptation can be driven by a simple, interpretable feature (difficulty quintile) rather than requiring complex learned policies.

The finding that predicted difficulty bins (using PRM scores without ground-truth labels) track oracle bins closely (Figures 4, 8, 11, 12) is significant for the practical viability of this approach. If difficulty-conditioned allocation required knowing the correct answer in advance, it would be circular. The fact that the PRM's own uncertainty (as reflected in its score distribution) provides sufficient signal for strategy selection means the approach is, in principle, deployable. This is a conceptual bridge between two literatures that rarely connect: the uncertainty estimation literature (which studies when models "know what they know") and the test-time compute literature (which studies how to use extra computation). The paper shows that the former can guide the latter — uncertainty estimates can inform resource allocation decisions — in a concrete, empirically validated way.

The paper does not claim that its specific discretization (five bins) or estimation method (2048 samples) is optimal. The contribution is establishing that the difficulty-strategy interaction exists and matters enough to justify adaptive allocation. Future work could develop better difficulty estimators, continuous allocation policies, or dynamic strategies that adjust mid-computation. The paper opens a research direction where the question shifts from "which test-time strategy is best?" to "how should we build systems that choose test-time strategies adaptively?" — a more productive framing that the paper's empirical results strongly motivate.

Innovation 5: Verifier Over-Optimization as the Primary Bottleneck — Refocusing Test-Time Compute Research on Verifier Quality Rather Than Search Sophistication

The paper's final conceptual contribution is a diagnostic finding that redirects the research agenda for test-time compute. Prior to this work, the natural assumption — reflected in papers on tree-of-thought, MCTS for language models, and increasingly sophisticated search procedures — was that better search algorithms would unlock better performance. The intuition is straightforward: if you can explore the space of possible solutions more intelligently, you should find better answers. The paper provides strong evidence that this assumption is wrong, or at least severely bounded: the bottleneck is not search algorithm quality but verifier quality, because all search methods eventually over-optimize the verifier signal, producing solutions that score well under the verifier but are actually incorrect.

The evidence for this is multi-pronged and consistent across experimental settings. Beam search — a stronger optimizer than best-of-N — degrades performance on easy problems at high budgets (Figure 3, right, bin 1: accuracy decreases from ~78% to ~77% as budget increases from 4 to 256). Lookahead search — the strongest optimizer tested — paradoxically performs worst overall at matched generation budgets because its extra per-step cost reduces beam diversity while its deeper optimization amplifies verifier exploitation (Figure 3, left). Qualitative examples in Appendix M show degenerate outputs — repetitive low-information steps, overly short 1–2 step solutions — that score highly under the PRM but are obviously wrong to human evaluators. These are not marginal effects; they are the dominant limitation on scaling test-time compute past moderate budgets.

What makes this a conceptual innovation rather than just an empirical observation is that it reverses the presumed relationship between optimization power and performance. In most machine learning contexts — training larger models, training for more steps, using better optimizers — more optimization is better until diminishing returns set in. The paper shows that for test-time search against learned verifiers, more optimization can be actively harmful past a certain threshold, and that this threshold depends on problem difficulty. This is fundamentally a Goodhart's Law dynamic: when a proxy measure (the PRM's score) becomes the target of optimization, it ceases to be a good proxy. The paper is the first to characterize this dynamic systematically in the context of test-time compute scaling for language model reasoning, establishing it as a first-class phenomenon that any future test-time strategy must account for.

The practical implication is a clear research priority: improve verifier robustness, not search algorithm sophistication. The paper's compute-optimal policy can be understood partly as a strategy for staying below the verifier over-optimization threshold — using weaker optimization (best-of-N) where the verifier signal is most reliable (easy problems, where the base model already produces many correct solutions) and stronger optimization (beam search) only where the verifier has room to provide genuine guidance (medium problems where the base model's pass@1 is non-trivial but imperfect). Future verifiers that are more resistant to over-optimization — through adversarial training on search-generated solutions, ensemble methods, or KL-constrained search that penalizes deviations from the base model's output distribution — could shift this frontier, enabling larger budgets to be deployed effectively. The paper's finding that the PRM trained with Monte Carlo soft labels behaves differently from binary-label PRMs (Appendix E, last-step aggregation being optimal rather than min-aggregation) hints that training methodology significantly affects verifier calibration and thus over-optimization susceptibility, opening a research direction the paper identifies but does not explore.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper does not report empirical experiments in the traditional sense. There is no benchmark dataset, no test split, no quantitative evaluation against baselines, and no ablation studies. The paper is a governance design proposal, not an empirical machine learning study. Its claims are supported through logical argumentation, reference to prior empirical work by others, and analysis of institutional mechanisms — not through the authors' own experiments on models or datasets.

Base model(s). Not applicable. The paper does not train or evaluate any models.

Metrics. Not applicable in the conventional sense. The "success" of the proposed data trust is evaluated along qualitative dimensions: whether the verification regime can detect compliance violations, whether the incentive architecture can motivate developer participation, whether the royalty structure can sustain the digital commons, and whether the governance design can resist corporate and government capture. These are assessed through argumentation about mechanism design, not through quantitative metrics like accuracy or efficiency.

Baselines. The paper does not compare against alternative proposals in a controlled experimental framework. Instead, it positions itself against the status quo (private, unaccountable data collection by commercial AI developers) and against alternative governance approaches discussed in Section 8: individual data rights legislation (GDPR, PIPEDA, CCPA), AI ethics guidelines (Jobin et al., 2019), compute governance proposals (Hwang, 2018; Whittlestone and Clark, 2021), and prior data trust concepts that have not been applied to training data (Delacroix and Lawrence, 2019; Huang and Siddarth, 2023; Jernite et al., 2022). The comparison is analytical rather than empirical — the paper argues that each alternative is structurally insufficient for the specific problem of collective bargaining over aggregated training data, but does not provide quantitative evidence of this insufficiency.

Generation budget / compute accounting. Not applicable. The paper's analysis of costs (Section 5.4.1) acknowledges that verification — particularly proof-of-learning protocols — would be "a large compute cost" and that data collection and maintenance would be expensive, but provides no quantitative estimates of these costs, the budget required to operate the trust, or the number of model developers it could service. This is identified as a limitation requiring future work rather than modeled in the current paper.

Cross-validation / statistical protocol. Not applicable.

Main Quantitative Results

The paper contains no quantitative experimental results generated by the authors. All empirical claims are supported by citation to prior work.

The Paper's Empirical-Citation Structure

Rather than conducting experiments, the paper relies on existing published or preprint results to establish the premises of its argument. The key empirical premises and their sources are:

Premise 1: Foundation models depend heavily on internet-scraped data. The paper cites scaling laws literature (Hoffmann et al., 2022; Kaplan et al., 2020) to establish that training data quantity is a primary driver of model capability. It cites specific model descriptions: GPT-3 (Brown et al., 2020), the Chinchilla scaling analysis (Hoffmann et al., 2022), and the Pile dataset (Gao et al., 2020; 800GB of diverse text) to establish the scale of data involved. It notes that "pre-training dataset sizes can run into the trillions of tokens and over 5 TB of pure text" for language models and "as large as 4 billion images" for vision models (citing Dehghani et al., 2023). These are factual claims about the state of the field, not experimental findings of this paper.

Premise 2: Private companies dominate foundation model development. The paper cites Giattino et al. (2022) and Ganguli et al. (2022) to establish the concentration of AI development in the private sector, and Sevilla et al. (2022) to document the exponential growth in training compute (doubling every 10 months at the frontier). These are descriptive empirical claims from the AI governance and measurement literature, not from this paper's own data collection.

Premise 3: Data poisoning is practically feasible for verification purposes. The paper's verification approach (Section 5.1) relies on the technical feasibility of data poisoning, which it supports by citing Carlini et al. (2023) — who demonstrated that poisoning as little as 0.01% of a training dataset can implant chosen behaviors in web-scale models — and Carlini and Terzis (2022) on poisoning contrastive learning, as well as Li et al. (2020) on dataset watermarking via backdoors. The paper does not replicate or extend these experiments; it cites them as existence proofs that the technique is plausible.

Premise 4: Proof-of-learning protocols can detect training data non-compliance. The verification approach also relies on Jia et al. (2021), who introduced the proof-of-learning framework and demonstrated it empirically on CIFAR-10 and other vision benchmarks. The paper cites their findings — that proof-of-learning can detect when claimed training procedures are fabricated — but does not replicate them or extend them to the language model scale at which the trust would operate. It acknowledges that "there is as yet no method with a formal security guarantee" for proof-of-learning (citing Fang et al., 2022; Zhang et al., 2022), meaning the cited empirical work demonstrates feasibility but not provable security.

Premise 5: Model-generated training data is becoming increasingly viable. The paper's analysis of a key threat to the trust's bargaining power (Section 3.2) cites Bai et al. (2022) on Constitutional AI (which found that model-generated feedback data for RLHF provides Pareto improvements over human-only data), Wu et al. (2022) on synthetic pre-training datasets (which found they can provide significant benefits compared to human-sourced data), and Taori et al. (2023) on Stanford Alpaca (which used model-generated instruction data to fine-tune LLaMA). These are cited to establish that the trust's window of leverage may be closing, not as findings of the current paper.

Premise 6: Wikipedia is a high-quality information resource. The paper's argument about the digital commons' value cites Mesgari et al. (2015), a systematic review finding that Wikipedia is "comparable to academically authored encyclopedias in both breadth and reliability." This supports the claim that the digital commons contains valuable knowledge infrastructure worth protecting.

Premise 7: Open-source software has large positive economic impact. The paper cites Directorate-General for Communications Networks et al. (2021) and Ghosh (2007) to establish the economic value of open-source software to the EU economy, supporting the claim that the digital commons contributes materially to economic well-being.

Premise 8: Privacy violations and data scandals demonstrate public concern. The paper cites the Cambridge Analytica settlement (McCallum, 2022), the Clearview AI fine by the UK ICO (2022), and concerns about TikTok data harvesting (Touma, 2022) as evidence that data governance issues have public salience. It also cites Obar and Oeldorf-Hirsch (2020) on the finding that "terms of service can be so long and difficult to understand that many skip them entirely," supporting the claim that meaningful consent for data use is often absent under current arrangements.

What "Results" Would Look Like for This Paper

The paper's actual "results" are its institutional design proposals and the feasibility analysis that accompanies them. The key design conclusions — which function analogously to empirical findings — are:

Design Conclusion 1: A multi-layer verification regime is necessary because no single mechanism is sufficient. This conclusion emerges from the paper's analysis in Section 5, which examines each verification mechanism (digital signatures, proof-of-learning, deployed model verification) and identifies specific failure modes for each. For digital signatures: the offense-defense balance is precarious, and defenses such as adversarial training (Geiping et al., 2022) and early stopping (Wallace et al., 2021) are known. For proof-of-learning: no method has a formal security guarantee, and verification cost scales with the number of checkpoints checked. For deployed model verification: it fails when the developer deploys on their own hardware without a cooperating compute provider. The conclusion that these layers must be combined is drawn from the observation that each layer's weaknesses are partially addressed by the others — proof-of-learning can catch data augmentation that digital signatures might miss; deployed model verification can catch model substitution that the first two checks cannot — but the paper does not provide a quantitative security analysis showing the combined system's detection probability under a formal threat model.

Design Conclusion 2: Certification can complement regulation as an incentive mechanism. This conclusion is supported by reference to the Fair Trade label precedent (Dragusanu et al., 2014, which analyzed the economics of Fair Trade certification) and by argumentation about consumer behavior — the paper asserts that "it seems plausible that a large proportion of citizens are interested in certification, especially given the prominence of data privacy issues and controversies over unfair compensation for data generation." However, the paper provides no survey data, willingness-to-pay studies, or experimental evidence that consumers would actually choose certified AI services over uncertified alternatives, or that the market share effect would be sufficient to offset compliance costs. The paper acknowledges this gap explicitly: "the veracity of (3) remains to be seen."

Design Conclusion 3: Financial independence through royalty self-funding is preferable to government appropriations. This conclusion is supported by argumentation about political economy risks — a government-dependent trust could be pressured to approve data access for intelligence services or to suppress politically inconvenient facts — but provides no empirical evidence that royalty self-funding would generate sufficient revenue to operate the trust, or that revenue-dependence on model developers would not create its own capture risks. The paper cites central banks as models of independent public entities, but does not analyze whether the conditions that enable central bank independence (well-established macroeconomic consensus, clear statutory mandates, international institutional support) apply to the AI governance context.

Design Conclusion 4: National scope is necessary for enforcement authority. This conclusion distinguishes the proposal from international governance frameworks (specifically Jernite et al., 2022) and is supported by the observation that a trust needs legal authority within a specific jurisdiction to mandate data entrustment, enforce verification requirements, and pursue legal action against non-compliant developers. The paper does not empirically compare national vs. international governance effectiveness for analogous domains (e.g., financial regulation, environmental treaties, intellectual property enforcement) to substantiate this claim.

Ablation Studies and Robustness Checks

The paper does not contain ablation studies as they exist in empirical ML research — no components are removed and performance re-measured. However, the paper performs an analogous function through its analysis of failure modes and limitations, which function as conceptual robustness checks on the proposal.

Political will as a precondition (Section 3.1). The paper identifies that the trust requires "a substantial amount of political will" to establish and examines whether this precondition is plausible. It cites ongoing regulatory developments (the EU AI Act, the NIST AI Risk Management Framework, the UK's pro-innovation AI regulation approach, all from 2021-2022) and public salience events (the Stability AI lawsuit by Getty Images, the widespread deployment of ChatGPT and Bing's Sydney) as evidence that political will may be emerging. This is not a quantitative robustness check but a qualitative assessment of enabling conditions.

Model-generated training data as a threat to bargaining power (Section 3.2). The paper analyzes the sensitivity of its proposal to the assumption that human-generated data remains a bottleneck for model training. If synthetic data becomes sufficient, the trust's leverage collapses. The paper cites evidence that this threat is real (Bai et al., 2022; Wu et al., 2022) and draws the conclusion that "it might be best to establish a data trust earlier rather than later." This functions as a robustness analysis: the proposal's viability degrades over time as synthetic data capabilities improve, meaning the intervention is time-sensitive.

Corporate capture risk (Section 3.3). The paper analyzes the risk that "model developers will unduly influence the decision-making of the data trust" through lobbying, corruption of board members, or buying off individual data holders. Mitigations proposed include transparency requirements for board funding, strict conflict-of-interest rules, and regular oversight by independent civil society organizations. The paper does not provide evidence that these mitigations would be effective — it identifies the risk and proposes standard governance safeguards without analyzing their track record in analogous institutions.

Government capture risk (Section 3.4). The paper analyzes the risk that government could coerce the trust to approve data access for politically motivated purposes (national security models, suppression of inconvenient facts). The proposed mitigation is financial independence through royalty self-funding and institutional independence modeled on central banks. The paper acknowledges this as a serious risk without quantifying its likelihood or demonstrating that the proposed mitigations are sufficient.

Verification cost as a feasibility constraint (Section 5.4.1). The paper identifies that verification — particularly proof-of-learning — would be "a large compute cost, especially if the data trust must service multiple model developers." It suggests partnering with trusted parties or hiring in-house talent. This functions as a recognition that the proposal has a resource constraint that could limit its scalability, but provides no quantitative modeling of the cost per developer, the number of developers the trust could service, or the total budget required.

Small teams and open-source developers as a coverage gap (Section 5.4.3–5.4.4). The paper analyzes whether the verification regime can prevent non-compliance by small teams or open-source developers and concludes that it cannot — "it is difficult to prevent individuals or small teams of model developers from scraping some internet data and training a model." The paper argues this is acceptable because the proposal targets large commercial developers, and "the open-source ecosystem is likely to remain behind the private frontier for the foreseeable future due to funding, compute, and talent constraints." This is an explicit scope limitation presented as a robustness finding: the proposal is not robust to all possible developers, only to the largest commercial ones, and this is by design.

Data leakage from the trust (Section 5.4.2). The paper analyzes the risk that developers who obtain trust data could leak it to third parties, and acknowledges that "the trust can only threaten to pursue disciplinary action upon discovery of a leak." Digital signatures can help detect leakage, and the design space for dataset watermarking is broader for leakage detection than for training verification. This is a robustness analysis showing that the proposal has a detection mechanism but not a prevention mechanism for this failure mode.

Deployed model updating as an ongoing verification burden (Section 5.4.5). The paper identifies that "model developers may routinely update their deployed models in response to user feedback," meaning verification is not a one-time event but an ongoing process. It proposes extending the verification regime to user feedback data (which falls under the trust's mandate) but does not analyze the feasibility of continuously verifying deployed models in practice.

Critical Assessment

The paper is not an empirical contribution and should not be evaluated by the standards of experimental ML research. Its contributions are in institutional design, mechanism analysis, and normative argumentation about the governance of training data. Evaluating it requires asking not "do the experiments support the claims?" but rather "does the analysis support the institutional design?" and "are the identified feasibility constraints adequately addressed?"

On the claim that a data trust could effectively gate access to training data (Section 2.4–2.5). The paper's central claim is conditional: if the trust can collect sufficient data, verify compliance, and incentivize developer participation, then it can negotiate terms that address the power imbalance. The paper provides detailed analysis of the "how" for each component — scraping infrastructure, verification protocols, incentive mechanisms — but the effectiveness of these components depends on empirical and technical assumptions that the paper acknowledges are unresolved:

The verification regime's effectiveness against determined adversaries is untested. The digital signature approach depends on data poisoning remaining viable against adversarial defenses that are an active research area. The proof-of-learning protocol has no formal security guarantee. The deployed model verification fails entirely when developers control their own deployment hardware. The paper's honest acknowledgment of these limitations is a strength, but it means the central enforcement mechanism — the trust's ability to detect non-compliance — rests on technical approaches that the paper itself characterizes as not yet reliable. A developer with sufficient resources and motivation could likely evade detection using known defenses, and the paper does not establish that the multi-layer approach quantitatively reduces evasion probability below an acceptable threshold.
The incentive architecture's effectiveness is speculated rather than demonstrated. The certification proposal (Section 6.2) requires that consumers care enough about data governance certification to offset compliance costs. The paper provides no survey evidence, no market analysis, and no historical precedent for analogous certification schemes in digital services. Fair Trade certification exists for physical goods where supply chains are visible and consumer preferences for ethical sourcing are well-documented. It is unclear whether AI services — where the connection between training data and the user experience is invisible to consumers — can sustain analogous consumer-driven certification. The paper acknowledges this uncertainty but does not analyze it in depth.
The difficulty estimation cost — 2048 samples per question — is unaccounted. While this specific mechanism is not part of this governance paper, the analogous problem in this context is the cost of operating the verification and data collection infrastructure. The paper identifies these costs but provides no modeling of whether the trust could be self-sustaining on royalty revenues, what royalty rate would be required, or whether developers would agree to rates that make the trust viable. These are not minor implementation details — they determine whether the proposal is economically feasible.

On the claim that the proposed trust addresses the structural power imbalance (Section 1, Section 2.4). The paper argues that a data trust would give the public collective bargaining power over training data that individuals currently lack. This claim has conceptual validity — aggregation does change bargaining dynamics — but the paper does not fully grapple with several complications:

The trust's governance structure is left largely unspecified. Who selects the trustees? How are diverse societal perspectives ensured in practice rather than in principle? What prevents the trust from becoming another unaccountable gatekeeper, substituting public unaccountability for private unaccountability? The paper defers these questions to "future implementation," but they are central to the normative claim that the trust represents the public interest. An unaccountable or captured trust would not correct the power imbalance; it would simply shift it to a different institution.
The trust's monopoly position creates its own risks. The paper proposes that the trust be the sole authorized source of training data for commercial foundation models — a monopoly on an essential input to AI development. Monopoly control, even when initially well-intentioned, creates risks of rent-seeking, innovation suppression, and political manipulation that the paper acknowledges (Sections 3.3–3.4) but addresses only with general governance principles (transparency, independent oversight, financial independence). The paper does not analyze whether these principles have been sufficient to prevent capture or abuse in analogous monopoly-granting institutions (spectrum allocation, patent offices, domain name registries) or whether the AI context poses unique capture risks.
The proposal's national scope means it can only govern data from one jurisdiction. The paper proposes a trust in the US or UK. A developer could, in principle, train on data from jurisdictions without a trust, supplemented by trust-licensed data only as needed. The paper does not analyze the feasibility of a single national trust controlling enough of the global training data distribution to maintain bargaining power, or whether trust-licensing requirements would create incentives for developers to shift data collection to trust-free jurisdictions.

On the claim that the trust addresses negative externalities (Section 2.5–2.6). The royalty-redistribution mechanism is conceptually sound — it channels a portion of AI-derived revenue back to those whose data and labor made it possible — but the paper does not establish that the mechanism can operate at the scale needed:

The paper provides no estimate of the revenue that could be raised through royalties on model revenues, the size of the funds that could be established, or the proportion of negative externalities they would offset. Without these estimates, the claim that the trust "addresses" or "redresses" negative externalities is qualitative rather than quantitative.
The paper acknowledges that hard problems (difficulty bin 5 in the MATH benchmark analogy) — in this context, AI systems that substitute for labor in ways that cause structural unemployment — cannot be solved solely through royalty redistribution. The trust can provide funds, but funds alone do not solve the social and political challenges of mass unemployment. The paper is clear that the trust is a partial solution, but it does not analyze what complementary institutions or policies would be needed for the trust to function as part of a broader response.

What experiments or analyses would strengthen the paper. The paper would be strengthened by:

A quantitative security analysis of the verification regime under realistic threat models, showing the probability of detecting non-compliance given known defenses and the computational cost of verification.
A market analysis or willingness-to-pay survey assessing whether AI service consumers would actually prefer certified services and whether the market share effect would be sufficient to incentivize developer participation.
A financial model of the trust's operations, estimating data collection costs, verification costs, plausible royalty rates, and the resulting fund sizes relative to the scale of negative externalities.
A comparative institutional analysis examining the track record of analogous gatekeeping institutions (domain name registries, spectrum licensing bodies, patent offices) in maintaining independence, resisting capture, and serving public interests.
An analysis of jurisdictional arbitrage risks — whether a trust in one jurisdiction could maintain leverage if developers can shift data collection and model training to jurisdictions without trusts.
A more detailed governance specification addressing trustee selection, accountability mechanisms, decision-making procedures, and the handling of conflicts between different public interest interpretations (e.g., safety vs. innovation vs. compensation).

Overall assessment. The paper makes a valuable conceptual contribution by identifying the structural nature of the training data governance problem, proposing a specific institutional mechanism (the public data trust), and analyzing its design requirements and failure modes with unusual candor. The verification regime's creative repurposing of data poisoning techniques as governance tools opens a genuinely novel research direction at the intersection of adversarial ML and institutional design. However, the paper's claims about the effectiveness and feasibility of the proposed trust rest on empirical and technical assumptions that the paper identifies but does not resolve. The verification regime's reliability against adversarial developers, the incentive architecture's ability to motivate compliance without regulation, the trust's capacity to resist capture, and the economic viability of the royalty model are all open questions that the paper frames as future work rather than settled findings. The paper should thus be read as a detailed proposal and feasibility analysis that establishes the intellectual framework for a data trust while candidly acknowledging the gaps that must be closed before implementation, rather than as a demonstration that the proposed trust would succeed in practice.

6. Limitations and Trade-offs

The Verification Regime Has No Formal Security Guarantees Against Adversarial Evasion

The assumption or constraint. The entire enforcement backbone of the proposed data trust depends on the three-layer verification regime described in Section 5: digital signatures via data poisoning, proof-of-learning protocols, and deployed model verification. The paper presents these as reinforcing mechanisms whose combined deployment would detect non-compliance by model developers. However, the paper explicitly acknowledges that none of these mechanisms provides formal, provable security against a determined adversary. For digital signatures, the paper states:

"it remains to be seen how effective such defenses would be on language and text-to-image models. Wallace et al. [103] show that early-stopping can provide a moderate defense against data poisoning in language models at the cost of some predictive accuracy. Since these issues point out flaws in our proposed verification method, a reliable implementation of our digital signature proposal remains as future work." (Section 5.1.2)

For proof-of-learning, the paper acknowledges:

"there is as yet no method with a formal security guarantee" (Section 5.2, citing Fang et al., 2022; Zhang et al., 2022).

For deployed model verification, the paper identifies a hard failure case:

"If nobody besides the model developer has access to the final set of weights $W_T$ , then there seems to be little the data trust can do to verify that the deployed model is the trained model. This gap is a limitation of our proposal." (Section 5.3)

The consequence. A well-resourced commercial developer — precisely the type of actor the trust is designed to regulate — who is motivated to evade the trust's data restrictions can exploit known defenses against data poisoning (adversarial training, early stopping, or data filtering) to dilute or eliminate the digital signatures. They can structure training runs to make proof-of-learning verification economically infeasible — for instance, by training on extremely large datasets where verifying even a subset of checkpoints is computationally prohibitive, or by exploiting the fact that the trust must choose a subset of checkpoints and can miss a spoofed segment. And they can deploy models on their own hardware, entirely bypassing the compute-provider coordination mechanism. The paper does not establish that the combination of these three layers quantitatively reduces the probability of successful evasion below any specific threshold, and the acknowledgment that no individual layer has formal guarantees means the system's security rests on the unproven assumption that the layers' weaknesses are non-overlapping in ways an adversary cannot simultaneously exploit.

What evidence exists in the paper. The paper provides no empirical evaluation of the verification regime's effectiveness against adversarial developers. There are no experiments measuring detection rates, no analysis of the computational cost required for the trust to achieve a given detection probability, and no formal threat model characterizing the capabilities of developers who might attempt evasion. The evidence for feasibility is entirely citational: the paper references Carlini et al. (2023) and Carlini and Terzis (2022) as demonstrating that data poisoning works in principle, and Jia et al. (2021) as demonstrating that proof-of-learning works in principle on small-scale vision benchmarks (CIFAR-10). These citations establish that the mechanisms are not physically impossible, but they do not establish that the mechanisms can be reliably deployed at the scale of foundation model training (trillions of tokens, thousands of GPUs, months of training time) against developers employing known countermeasures.

Mitigation status. The paper acknowledges these gaps candidly and frames them as future work rather than as solved problems. Specifically, it states that "a reliable implementation of our digital signature proposal remains as future work" (Section 5.1.2) and that the PoL index selection problem lacks formal guarantees. However, the paper does not propose concrete research directions for closing these gaps, does not estimate when or whether they might be closable, and does not analyze whether the trust could function in an interim period when verification is probabilistic rather than guaranteed. The verification regime is presented as a necessary component of the trust's leverage, but the paper's own analysis suggests that component does not yet exist in reliable form.

The Verification and Data Collection Costs Are Unquantified and Potentially Prohibitive

The assumption or constraint. The paper proposes that the trust operate substantial infrastructure: scraping and curating internet-scale datasets comparable to those used by the largest private developers, running proof-of-learning verification protocols that re-execute training steps for multiple model developers, maintaining digital signature databases, and coordinating with compute providers. The paper acknowledges that these activities are expensive:

"Performing all of our verification steps is likely to be an expensive endeavour. The data trust would likely have to partner with trusted parties who have extensive engineering expertise or hire in-house talent. Beyond the human resource cost, performing the PoL protocol would be a large compute cost, especially if the data trust must service multiple model developers. Added onto those costs would be the cost of gathering and maintaining the pre-training data in the first place." (Section 5.4.1)

However, the paper provides no quantitative estimates of these costs — no modeling of the computational budget required for proof-of-learning verification per developer, no estimate of the storage and bandwidth costs for maintaining internet-scale datasets, no projection of the number of developers the trust would need to service, and no analysis of whether the proposed royalty revenues (which are also unquantified) would be sufficient to cover operating costs.

The consequence. Without cost estimates, the feasibility of the trust as a self-sustaining institution is entirely unvalidated. The paper proposes that the trust be funded through a portion of negotiated royalty revenues rather than government appropriations (Section 3.4), explicitly to achieve financial independence and resist political pressure. But this funding model only works if royalty revenues exceed operating costs. If verification costs scale with the number of developers or the size of models being verified — as the PoL protocol implies, since verifying training steps requires re-executing them — then the trust's cost structure may grow with the AI industry it regulates, potentially faster than royalty revenues. If costs exceed revenues, the trust either becomes dependent on government funding (defeating the independence rationale), must charge developers for verification (creating a conflict of interest where the trust depends financially on the entities it regulates), or cannot operate at the scale needed to maintain its data bottleneck.

Furthermore, the certification-based incentive mechanism (Section 6.2) assumes that the trust can provide verification services at a cost that does not make certified models uncompetitive. If verification adds significant overhead to model development timelines or costs, developers may judge that the market advantage of certification is not worth the price, undermining the voluntary compliance path.

What evidence exists in the paper. The paper provides no quantitative evidence on costs. There is no cost model, no analysis of computational requirements, no benchmarking of proof-of-learning verification costs at foundation model scale, and no modeling of the trust's budget under different assumptions about the number of developers served and model sizes. The paper's analysis of costs is limited to the qualitative acknowledgment quoted above. The paper also provides no estimate of the royalties the trust could negotiate — what percentage of model revenues, what total revenue pool this represents, or how this compares to the negative externalities the royalties are meant to address.

Mitigation status. The paper does not attempt to mitigate this limitation. It identifies costs as a concern (Section 5.4.1) and suggests in general terms that the trust could partner with organizations with expertise or hire in-house talent, but does not propose specific mechanisms for cost control, efficiency improvements in verification, or ensuring financial sustainability. The funding model — royalty self-funding — is proposed in Section 3.4 as a design principle, but without cost estimates, the paper cannot demonstrate that this principle is achievable in practice. This limitation is acknowledged but not addressed.

The Trust's Leverage Depends on Human-Generated Data Remaining a Bottleneck, a Condition the Paper Acknowledges May Be Temporary

The assumption or constraint. The paper's entire theory of change rests on training data being a bottleneck for foundation model development — if developers can obtain sufficient training data without going through the trust, the trust has no bargaining power. Section 3.2 explicitly identifies the threat that model-generated synthetic training data could erode this bottleneck:

"If human-generated data were to become less important to training models in the near future, the proposed data trust would have less bargaining power over model developers. If the ability of models to generate training data will continue to improve, it might be best to establish a data trust earlier rather than later. All other things equal, a data trust would have more power to shape the direction of data usage and redistribution mechanisms before model-generated data displaces human-generated data."

The paper cites Bai et al. (2022) on Constitutional AI — which found that model-generated feedback data for RLHF provides Pareto improvements over human-only data — and Wu et al. (2022) on synthetic pre-training datasets providing significant benefits. It also cites Taori et al. (2023) on Stanford Alpaca as evidence that model-generated instruction data can effectively fine-tune models.

The consequence. The trust's bargaining power has a shelf life that the paper cannot estimate. If synthetic data becomes sufficient for training competitive foundation models before the trust is established — or soon enough after establishment that the trust cannot accumulate institutional legitimacy, financial reserves, and political support — the trust becomes a costly apparatus controlling a resource that developers no longer need. The paper frames this as an argument for urgency ("establish a data trust earlier rather than later") but does not analyze the timeline: how quickly is synthetic data improving relative to the time required to establish a national public data trust through legislative process, build scraping infrastructure, develop reliable verification mechanisms, and negotiate initial royalty agreements? If synthetic data is viable in, say, five years, and establishing the trust takes four, the trust has approximately one year of effective leverage before its bargaining position begins to erode — potentially insufficient to establish durable redistribution mechanisms or to justify the setup costs.

Even before synthetic data fully displaces human data, partial substitution weakens the trust incrementally. If developers can use 80% synthetic data and 20% trust-licensed data and achieve near-equivalent performance, the trust's bottleneck status weakens. The royalty rates the trust can negotiate would decline as the marginal value of human-sourced data declines, potentially falling below the trust's operating costs. The paper does not analyze this intermediate regime of partial substitution.

What evidence exists in the paper. The paper cites empirical evidence that synthetic data is becoming increasingly viable (Bai et al., 2022; Wu et al., 2022; Taori et al., 2023) but does not project when synthetic data might reach parity with human-sourced data for pre-training or fine-tuning, how the substitution curve might evolve, or what proportion of training data would remain human-dependent even as synthetic data improves. The evidence establishes that the threat is real and directionally concerning, but provides no basis for estimating the trust's window of effective operation.

Mitigation status. The paper does not propose any mechanism to address this limitation. The trust's design does not include strategies for maintaining relevance in a world of mixed synthetic and human data — for instance, by controlling the models that generate synthetic data, by verifying the provenance of synthetic data, or by shifting its focus from data quantity to data quality or curation expertise. The only response is the temporal argument: act now, before the window closes. This is a strategic recommendation, not a design feature that makes the trust robust to the identified threat.

The Proposal Provides No Empirical Evidence That Certification or Consumer Pressure Would Incentivize Developer Compliance

The assumption or constraint. Section 6.2 proposes that the trust could provide certifications — analogous to Fair Trade labels — to model developers who voluntarily comply with the trust regime, and that consumer preference for certified services would create market pressure for compliance. The paper specifies three criteria for this to work:

"(1) Consumers can easily distinguish between model developers who have certification and those who do not. (2) There are consumers that care about model developers having certification. (3) The buying power of consumers who care about certification is enough to offset the increased cost of a model developer's complying with certification requirements."

The paper argues that criterion (1) is easily satisfied (display a label on the service interface) and that criterion (2) is plausible given "the prominence of data privacy issues and controversies over unfair compensation for data generation." For criterion (3), the paper states that "the veracity of (3) remains to be seen, but it seems plausible given the prominent media issues we discussed for (2)."

The consequence. The certification-based incentive mechanism — which the paper presents as an alternative or complement to regulation (Section 6.1) — rests on entirely unvalidated assumptions about consumer behavior in AI service markets. Unlike physical goods where Fair Trade certification has achieved measurable market penetration (coffee, chocolate, clothing), AI services have characteristics that may make certification-based consumer pressure less effective: the connection between training data practices and the user experience is invisible (a chatbot's output does not signal whether its training data was ethically sourced in the way a chocolate bar's packaging signals Fair Trade status); the market is dominated by a small number of very large providers with strong network effects and switching costs (users who prefer certified services may find that their preferred provider does not offer certification and that switching to a smaller certified provider degrades service quality); and the primary users of foundation models are often businesses integrating APIs rather than individual consumers making purchasing decisions based on ethical labels.

If criterion (3) fails — if the consumer segment that values certification is too small to offset compliance costs — then certification does not create meaningful market pressure. Developers who comply with the trust regime incur verification costs, royalty payments, and data restrictions that their non-compliant competitors do not, without gaining sufficient market advantage to compensate. In this scenario, the trust would need to rely entirely on regulatory mandates (Section 6.1) for compliance, but regulatory mandates are themselves uncertain — the paper acknowledges that "regulation can be difficult to implement and could be perceived as an undue intrusion upon the ability of companies to perform business" (Section 6.1). If neither certification nor regulation materializes, the trust has no mechanism to incentivize developer participation, and its data collection and verification infrastructure becomes a costly apparatus with no users.

What evidence exists in the paper. The paper provides no empirical evidence on any of the three criteria. There are no surveys of AI service consumers regarding data governance preferences, no willingness-to-pay studies, no analysis of certification effectiveness in digital service markets, and no historical precedent cited for consumer-driven certification schemes achieving market-shaping effects in enterprise software or API services. The paper's evidence for criterion (2) consists of references to media coverage of data privacy scandals (Cambridge Analytica, Clearview AI, TikTok) and speculation that "it seems plausible" this translates into certification-seeking behavior. This is a hypothesis, not an empirically supported claim.

Mitigation status. The paper does not attempt to mitigate this limitation. It presents certification as a mechanism that "could" work if the three criteria are satisfied, acknowledges uncertainty about criterion (3), and moves on. The paper does not propose research to validate the consumer demand assumption (e.g., survey studies, market experiments), does not analyze alternative incentive mechanisms that might work if certification fails, and does not model the trust's viability under scenarios where certification creates weak or no market pressure. The incentive architecture is presented as a menu of options (regulation, certification, comparative advantage, liability reduction) without analyzing which options are load-bearing and what happens if the most accessible ones fail.

The Governance Structure Is Left Largely Unspecified, Leaving Open the Risk That the Trust Itself Becomes an Unaccountable Gatekeeper

The assumption or constraint. The paper proposes a public data trust with significant power: it would control access to the training data essential for foundation model development, negotiate royalty terms, operate verification infrastructure, and manage redistribution funds. This concentration of power requires robust governance to ensure the trust operates in the public interest rather than becoming captured by industry, government, or its own institutional interests. The paper identifies this as a risk:

"The private sector is extremely well-funded. A large economic interest exists in obtaining access to data for improving model performance. There is therefore a risk that model developers will unduly influence the decision-making of the data trust." (Section 3.3)

"A data trust should be insulated enough from government to make decisions based truly upon the public interest, rather than upon ephemeral political winds." (Section 3.4)

However, the paper explicitly defers governance design: "As our focus here is on the functions of a trust, we defer further details about the governance structure of the trust to future implementation" (Section 2.4).

The consequence. The trust's normative legitimacy — its claim to represent the public interest rather than merely substituting one form of unaccountable power for another — depends on governance details that are not specified. Without a specified mechanism for trustee selection, the trust could become captured by the very industry it is meant to regulate (if trustees are drawn from AI companies or their funders), by government (if trustees are political appointees subject to partisan pressure), or by its own institutional interests (if trustees face no meaningful accountability mechanisms). The paper proposes general principles — diverse representation, regular reports to the legislature, transparency requirements, conflict-of-interest rules — but these are standard governance recommendations that many captured or ineffective institutions also formally adopt. The paper does not analyze why these principles would succeed in the data trust context when they have often failed in other regulatory contexts (captured regulatory agencies, revolving-door problems, regulatory capture in telecommunications and finance).

Specific scenarios the paper does not address include: What happens when different "public interests" conflict — for instance, maximizing royalty revenue for redistribution (which might favor granting data access to the highest-bidding developer) versus promoting a diverse AI ecosystem (which might favor broader, cheaper access)? How does the trust adjudicate between the interests of different contributor groups — artists, writers, coders, general internet users — who may have conflicting preferences about data usage terms? What prevents a future government from replacing the trust's leadership with loyalists who approve data access for government-favored projects? The paper identifies these as risks but provides no institutional mechanisms for managing them beyond general principles.

What evidence exists in the paper. The paper provides no empirical evidence on the effectiveness of the proposed governance principles in analogous institutions. It cites central banks as models of independent public entities (Section 3.4) but does not analyze whether the conditions that enable central bank independence — well-established macroeconomic consensus, clear and measurable mandates (inflation targeting), strong international institutional support, and a history of operational independence spanning decades — apply to the AI governance context, where there is no consensus on objectives, mandates would be inherently contested (balancing innovation, safety, compensation, and access), and the institution would be novel rather than evolving from existing structures.

Mitigation status. The paper defers governance specification to future work without establishing that governance design is separable from functional design. The functions the paper specifies — data collection, verification, negotiation, redistribution — all involve discretionary decisions (what data to include or exclude, what royalty rates to demand, how to distribute funds) that are inherently political and value-laden. The paper acknowledges this for data curation ("the act of doing so is inherently value-laden, the trust should carry out this process through or under the supervision of a diverse panel of experts across disciplines," Section 4.2) but does not extend the same analysis to the other functions or specify how diverse panels would be constituted, selected, or held accountable. By deferring governance to future implementation, the paper presents a functional architecture whose normative properties depend on unspecified institutional details — the trust could be a vehicle for democratic control of AI or a vehicle for new forms of unaccountable power, and the paper's analysis cannot distinguish between these outcomes.

The National Scope Creates Jurisdictional Arbitrage Risks and Limits the Trust's Ability to Control Global Training Data

The assumption or constraint. The paper proposes that the trust be "national and located in a jurisdiction with a high concentration of AI development, such as the US or the UK" (Section 1). The rationale is that national scope enables legal authority to mandate data entrustment, enforce verification requirements, and pursue legal action against non-compliant developers. The trust would control data from its jurisdiction's citizens and internet infrastructure. However, training data for foundation models is inherently global — the internet spans jurisdictions, and developers can collect data from any region with internet connectivity.

The consequence. A national trust in the US or UK can control data from its own jurisdiction's citizens and potentially from platforms and services operating within its territory. It cannot control data from the rest of the world. A commercial developer could respond to the trust's requirements by shifting data collection to jurisdictions without equivalent trusts — scraping websites hosted in countries with no data governance restrictions, hiring crowdworkers in countries where crowdworker data is not entrusted, or focusing data collection on regions whose citizens are not covered by the trust's mandate. The paper notes that "the population of the United States is more than 300 million, while the population of the EU is more than 400 million" (Section 4.3.2), implying that these populations generate substantial data. But this does not establish that the trust's data would be sufficient to maintain its bottleneck status if developers can supplement with non-trust data from other jurisdictions.

The jurisdictional arbitrage risk is particularly acute for pre-training data, where diversity and scale matter. If a developer can obtain high-quality English-language text from non-trust jurisdictions (e.g., scientific papers from international journals, code from global repositories, discussion forums hosted outside the trust's jurisdiction), they may be able to assemble a training corpus that is competitive with the trust's corpus without complying with the trust's terms. The trust's verification regime (Section 5) is designed to detect this — the proof-of-learning protocol would reveal that data indices from non-trust sources were used — but this only works if the developer has agreed to verification in the first place. A developer who chooses not to participate in the trust regime and operates from a jurisdiction that does not enforce the trust's requirements can train on globally scraped data without submitting to verification.

The paper's focus on commercial developers "in a jurisdiction with a high concentration of AI development" assumes that AI development will remain concentrated in those jurisdictions. If AI development becomes more geographically distributed — or if developers can incorporate in trust-free jurisdictions while accessing global compute and data markets — the national scope of the trust limits its reach.

What evidence exists in the paper. The paper provides no analysis of jurisdictional arbitrage risks or the global distribution of training data sources. It does not estimate what proportion of the data used in major foundation models originates from the trust's proposed jurisdiction, whether that proportion would be sufficient to maintain bargaining power, or how developers might restructure data collection in response to trust requirements. The paper acknowledges that its proposal is national rather than international (Section 8, distinguishing itself from Jernite et al., 2022's international governance framework) but does not analyze the strategic implications of this scope limitation from the perspective of a developer seeking to minimize compliance costs.

Mitigation status. The paper does not attempt to mitigate this limitation within its design. It does not propose mechanisms for international coordination (e.g., trust recognition agreements between jurisdictions, data import restrictions on models trained on non-trust data, or international standards that would extend the trust's effective reach). The paper presents the national scope as a design choice that enables enforcement authority but does not address the tension between this choice and the global nature of training data. This is a fundamental tradeoff — national scope provides legal authority at the cost of incomplete coverage — that the paper identifies implicitly by choosing national scope but does not analyze explicitly as a limitation.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper shifts the conversation around AI training data governance from a reactive, individual-rights framing to a structural, collective-bargaining framing. Prior work — spanning data protection legislation (GDPR, PIPEDA, CCPA), AI ethics guidelines, and individual opt-out mechanisms — treated the problem primarily as one of individual consent, privacy violation, or corporate responsibility failure. The implicit theory of change was that strengthening individual rights, increasing transparency, or improving corporate ethics practices could address the harms of training data extraction. This paper argues, with substantial analytical force, that these approaches are structurally incapable of solving the core problem because the value of training data emerges from aggregation, not from individual data points, and because no individual data contributor has meaningful bargaining power against a large AI developer. The paper thus reframes training data governance as a collective action problem requiring collective governance infrastructure — specifically, a public data trust that pools data contributions, asserts gatekeeping control over the aggregated resource, and negotiates terms on behalf of the public.

This is not a paradigm shift in the Kuhnian sense — the paper does not overturn established empirical findings or introduce a new theoretical framework that reinterprets prior results. It is, rather, a diagnostic reframing with institutional design consequences. The paper's primary contribution is to identify the structural origin of the power imbalance (training data as an aggregated commons whose value cannot be realized through individual action) and to demonstrate that this diagnosis implies a specific institutional form (a public data trust with fiduciary obligations, verification capabilities, and royalty-negotiation authority) rather than a different configuration of existing tools. The shift from "protect individual data subjects" to "build collective governance over aggregated data" has downstream effects on what kinds of research, policy proposals, and technical tools become relevant — a point elaborated below.

The paper also changes the landscape by bridging adversarial machine learning research and governance design in a concrete, mechanism-level way. Prior to this work, the security literature on data poisoning and the governance literature on data trusts operated in separate spheres. Security researchers studied attacks and defenses; governance scholars designed legal and institutional frameworks. The paper's proposal to use data poisoning as positive verification infrastructure — inserting digital signatures into training data to enable post-hoc compliance checking — creates a conceptual bridge between these communities. It demonstrates that technical vulnerabilities can be repurposed as governance tools, and that institutional design can specify requirements (detectability, robustness, low performance impact) that motivate security research in productive directions. This is a methodological innovation in governance design — not just proposing an institution, but showing how the institution's enforcement mechanisms can be grounded in specific technical primitives from adversarial ML — that opens a new category of interdisciplinary research at the intersection of these fields.

Regarding prior contradictions, the paper does not resolve empirical conflicts in the ML literature (there are no experiments to reconcile). Rather, it reconciles a conceptual tension in AI governance: between those who argue that data rights legislation and individual consent mechanisms are sufficient to address training data harms, and those who argue that only radical restructuring (e.g., public ownership of AI infrastructure) can correct the power imbalance. The paper's position is that individual rights are necessary but insufficient — they protect individuals from specific harms but cannot create collective bargaining power — and that public data trusts offer a middle path: collective governance of a specific bottleneck (training data) within existing legal frameworks, without requiring full public ownership of AI development. This position is more institutionally concrete than calls for "democratizing AI" and more structurally ambitious than calls for "better privacy policies," carving out a distinct space in the governance landscape.

The research directions this work makes more attractive include:

Technical verification tools for data provenance — the paper's digital signature proposal gives security researchers a concrete use case (governance-enabling watermarking) with specific requirements (undetectability by developers, robustness to standard training, minimal performance impact) that differ from traditional attack/defense framings. Research on dataset watermarking, backdoor insertion, and proof-of-learning now has a governance motivation beyond purely adversarial settings.
Institutional design for AI governance — the paper demonstrates that detailed mechanism analysis (verification protocols, incentive structures, funding models) can be applied to AI governance proposals, raising the bar for future work that proposes new institutions or regulatory frameworks. Subsequent proposals will be expected to address the same level of implementation detail (how would verification work? what incentives drive compliance? what are the failure modes?) that this paper models.
Empirical studies of consumer preferences for ethical AI — the paper's certification proposal (Section 6.2) highlights an empirical gap: we do not know whether AI service consumers care about training data provenance, whether they would pay premiums for certified services, or whether certification labels influence enterprise purchasing decisions. This gap is now explicitly identified as load-bearing for voluntary compliance mechanisms.

Research directions this work makes less attractive (or at least reframes as insufficient on their own):

Purely technical solutions to training data governance — the paper's structural diagnosis implies that no technical fix (federated learning, differential privacy, data deletion) can address the bargaining power problem, because the problem is not about protecting individual data points but about controlling aggregated resources. Technical privacy tools remain valuable for protecting individuals, but they do not substitute for collective governance.
Industry self-regulation and ethics guidelines — the paper explicitly argues that "ethical guidelines are no substitute for addressing the structural factors underlying the concentration of power in the private sector" (Section 2.2), and its entire proposal is premised on the insufficiency of voluntary corporate action. Future work that proposes ethics principles without accompanying structural mechanisms will need to contend with this argument.

Follow-Up Research This Work Enables

Data poisoning for training data verification at scale — feasibility study on language models. The paper proposes digital signatures via data poisoning (Section 5.1.1) but acknowledges that "a reliable implementation of our digital signature proposal remains as future work." A direct follow-up would empirically test whether current data poisoning techniques can meet the four specified requirements (undetectability, high recall after one epoch, low false positive rate on unpoisoned data, minimal performance impact) on a language model at the scale of, say, a 1B-parameter model trained on 100B tokens — not web-scale, but large enough to surface scaling-related failure modes. The study would test multiple poisoning methods from the literature (Carlini et al., 2023; Wallace et al., 2021; Carlini and Terzis, 2022) against known defenses (adversarial training, data filtering, early stopping) and measure detection rates, performance degradation, and developer-detectability (can a developer identify which training examples are signatures without prior knowledge?). A negative result — e.g., that known defenses reduce signature recall below 50% with acceptable performance cost — would substantially weaken the verification premise of the paper's proposal and redirect research toward alternative verification mechanisms.

Proof-of-learning at foundation model scale — computational cost and security analysis. The paper's second verification layer relies on proof-of-learning (PoL, Section 5.2), citing Jia et al. (2021) who demonstrated the approach on CIFAR-10. A critical follow-up would model and empirically measure the computational cost of PoL verification for a training run comparable to a modern large language model — e.g., a 7B-parameter model trained on 1T tokens, with checkpoints saved every N steps. The study would quantify: (a) the number of checkpoint pairs that must be verified to achieve a given detection probability under different spoofing strategies (e.g., fabricating a subset of checkpoints, training on mixed trust/non-trust data), building on the heuristic selection methods in Jia et al. and the security analyses in Fang et al. (2022); (b) the total FLOP cost of verification relative to the original training cost; and (c) whether the cost is feasible for a trust servicing multiple developers annually. If the cost exceeds, say, 10% of the original training run per verification, the PoL layer may be impractical for routine use and would need to be reserved for audit-based spot checks rather than universal verification — a finding that would reshape the paper's verification architecture.

Consumer willingness-to-pay for training data provenance certification. The paper's certification-based incentive mechanism (Section 6.2) rests on the empirically untested claim that "the buying power of consumers who care about certification is enough to offset the increased cost of a model developer's complying with certification requirements." A rigorous follow-up would conduct a discrete choice experiment or contingent valuation survey with a representative sample of AI service users (both individual consumers and enterprise purchasers) to measure willingness-to-pay for certification labels indicating ethically sourced training data. The study would vary label characteristics (e.g., "training data licensed through public data trust," "creators compensated for data contributions," "independently verified training data provenance") and measure price sensitivity, market segmentation, and the proportion of respondents for whom certification would influence platform choice. A finding that willingness-to-pay is negligible or confined to a small niche would indicate that the certification mechanism cannot serve as a primary compliance driver and that the trust would need to rely more heavily on regulatory mandates — a constraint that would substantially affect the political strategy for establishing a trust.

Governing synthetic training data — how a data trust could extend its mandate as human data becomes less essential. The paper identifies model-generated synthetic training data as an existential threat to the trust's leverage (Section 3.2) but proposes no mechanism for addressing it. A forward-looking follow-up would design and analyze extensions of the trust's mandate that could maintain its relevance in a regime of mixed synthetic and human data. Possibilities include: (a) requiring developers to disclose and verify the provenance of synthetic training data (was it generated by models that were themselves trained on trust-licensed data?), effectively extending the trust's reach through the data supply chain; (b) the trust itself generating and licensing synthetic datasets, becoming a provider of high-quality, documented, safety-curated synthetic data as a public good; (c) shifting the trust's bottleneck from data quantity to data quality and curation expertise, positioning it as the trusted arbiter of training data quality regardless of whether the raw data is human or synthetic. A strong study would model the economic viability of each extension under different assumptions about synthetic data capability growth rates and would identify the earliest indicators that the trust should pivot its strategy.

Comparative analysis of institutional capture risks — what determines whether a public gatekeeping institution remains accountable? The paper acknowledges corporate and government capture risks (Sections 3.3–3.4) but addresses them only with general governance principles (transparency, independent oversight, financial independence). A rigorous follow-up would conduct a comparative institutional analysis of analogous gatekeeping bodies — domain name registries (ICANN), spectrum licensing authorities (FCC), patent offices (USPTO), central banks, and natural resource revenue management institutions (e.g., Alaska Permanent Fund, Norway's Government Pension Fund Global) — to identify the specific design features that have successfully prevented capture (or failed to do so) in practice. The study would extract design principles that are empirically grounded rather than aspirationally stated, and would analyze how those principles translate (or fail to translate) to the AI training data context. For example, does the trust's proposed self-funding through royalty revenues create capture risks analogous to regulatory agencies funded by industry fees, or independence comparable to central banks with seigniorage revenue? This line of work directly addresses the paper's largest unresolved tension: the trust's normative legitimacy depends on governance details that the paper defers.

International coordination mechanisms for training data governance. The paper's national scope creates jurisdictional arbitrage risks (Section 6, Limitations) that are not addressed in the proposal. A natural follow-up would design and analyze mechanisms for extending the trust's effective reach across jurisdictions without requiring full international treaty frameworks (which would be slow to negotiate and fragile). Concrete possibilities include: (a) mutual recognition agreements between national trusts — a developer licensed by the UK trust is recognized as compliant by the EU trust and vice versa, creating a larger coordinated data pool and reducing arbitrage opportunities; (b) data import restrictions — models trained on non-trust data are restricted from deployment in trust jurisdictions unless developers can demonstrate equivalent data governance standards, analogous to the EU's GDPR adequacy decisions for cross-border data transfers; (c) compute-provider coordination — cloud providers operating globally could be required to verify trust compliance for any model trained on their infrastructure, regardless of where the developer is incorporated. A strong analysis would model developer incentives under each mechanism, identify the minimum coalition of jurisdictions needed to make arbitrage unattractive, and estimate the timeline and political feasibility of establishing such coordination.

Practical Applications and Downstream Use Cases

Legislative drafting for AI training data regulation. The paper's detailed specification of trust functions — data collection (Section 4), verification (Section 5), incentives (Section 6), and redistribution (Section 2.6) — provides a concrete template that legislative staff and regulatory agencies can use when drafting bills or rules governing training data for foundation models. Rather than starting from abstract principles, a legislative team could work through the paper's architecture: define the trust's legal form and fiduciary obligations (Section 2.4), specify its data collection authority including user data entrustment mandates (Section 4.3.2), establish the verification regime's legal basis for requiring training transcripts and model access (Section 5), and create the royalty negotiation framework (Section 2.6). The paper's candid identification of failure modes — verification gaps (Section 5.4), capture risks (Sections 3.3–3.4), the synthetic data threat (Section 3.2), and cost concerns (Section 5.4.1) — provides a checklist of issues that legislation would need to address, either through the trust's design or through complementary provisions. The paper explicitly cites ongoing regulatory developments (EU AI Act, NIST AI RMF, UK pro-innovation AI regulation) as evidence that political will for such legislation may be emerging (Section 3.1), making the timing of this template relevant.

Dataset documentation and curation standards for public-interest data providers. Even without full legislative establishment of a trust, organizations that already curate and release large-scale training datasets — EleutherAI, LAION, Common Crawl, Hugging Face — can adopt the paper's curation and documentation recommendations (Section 4.2) immediately. The paper specifies that curated data should "identify issues including but not limited to: errors or noise, data poisoning, personally identifiable information, and illicit or explicit information," that "safety risks [should be] separate[d] from the main pre-training set," that "all significant data curation decisions [should be] clearly documented with justification," and that curation should be performed "through or under the supervision of a diverse panel of experts across disciplines, with explicit representation of voices from marginalized communities." These are actionable best practices that do not require a trust to implement. A dataset provider that adopted them would produce training corpora that are higher-quality, better-documented, and lower-risk than current norms, and would be positioned to serve as the data partner for a future trust or to offer de facto trust-like curation services to developers seeking liability reduction (Section 6.3, where the paper argues that "even well-resourced companies might want to use data solely from the trust if the companies can assume less liability, whether legal or social, for model harms that can be traced to the data").

Crowdworker collective bargaining infrastructure. The paper's proposal for the trust to work with "crowdworker collectives and crowdsourcing platforms" to include human feedback data (Section 4.3.3) could be piloted immediately by existing crowdworker advocacy organizations such as Turkopticon or the Data Workers' Inquiry, without waiting for a national trust. A pilot would establish a voluntary pool of crowdworker-generated preference data where workers explicitly consent to their data being held in a collective trust structure, with transparent terms about how the data will be licensed and how revenues will be distributed. This would serve three purposes: (a) it would test whether crowdworkers are willing to participate in collective data governance when given the option, providing empirical evidence on the uptake question the paper flags (Section 4.3.2, "this method might suffer from low uptake given user inertia"); (b) it would create a small but real bargaining chip — a pool of quality-controlled, ethically sourced human preference data that developers could license as an alternative to standard crowdworker platforms, testing whether the paper's comparative advantage argument (Section 6.3) holds in practice; and (c) it would generate operational knowledge about the practical challenges of collective data management — pricing, licensing terms, revenue distribution mechanisms — that would inform the design of a full-scale national trust. The paper's observation that "crowdworkers and collectives have an incentive to accept the trust regime so as to amplify their bargaining power" is testable through such a pilot.

Safety-enhancing data as public goods — a concrete collection and release program. The paper proposes that the trust support the generation of safety-enhancing training data as public goods, free from the commercial licensing restrictions applied to general training data (Section 7.3). This function could be operationalized immediately by existing research organizations or philanthropic funders without requiring a trust. A concrete program would: (a) identify specific categories of safety-enhancing data through consultation with the safety research community — for example, data for training models to refuse harmful requests, to recognize and flag manipulative outputs, to be transparent about their own limitations, or to resist jailbreaking attacks; (b) fund and manage the collection of such data using the same rigorous curation and documentation standards the paper specifies for general data (Section 4.2); and (c) release the data publicly with permissive licenses, explicitly designed for incorporation into both commercial and non-commercial model training pipelines. The paper's public goods framing provides the rationale: safety is non-excludable and non-rivalrous, so private developers will underinvest in safety-enhancing data due to free-riding incentives (Section 7.3.2). A publicly funded program that generates and freely distributes such data directly addresses this market failure, and the paper's tiered access model (commercial data requires trust negotiation; safety data is free) provides the conceptual architecture for distinguishing which data falls into which category. The paper's note that this function "would likely require ongoing consultation with diverse communities and experts across disciplines" and could be coordinated through "conferences like AIES and FAccT" provides a starting point for the governance of such a program.

🎯 Pitch

1. Executive Summary

2. Context and Motivation

The Core Problem: Private Capture of a Public Resource

Why This Problem Matters Now

Prior Approaches and Where They Fall Short

How This Paper Positions Itself

3. Technical Approach

3.1 Reader Orientation

3.2 Big-Picture Architecture (Diagram in Words)

3.3 Roadmap for the Deep Dive

3.4 Detailed, Sentence-Based Technical Breakdown

3.4.1 Data Collection: Pre-Training Data from the Internet

3.4.2 Data Collection: Human Feedback Data

3.4.3 Data Collection: User Data That Cannot Be Scraped

3.4.4 Data Curation and Documentation

3.4.5 The Verification Regime: Overview and Threat Model

3.4.6 Verification Check 1: Digital Signature Insertion via Data Poisoning

3.4.7 Verification Check 2: Proof-of-Learning Protocol

3.4.8 Verification Check 3: Deployed Model Integrity

3.4.9 Additional Verification Challenges

3.4.10 The Incentive Architecture: Regulation

3.4.11 The Incentive Architecture: Certification

3.4.12 The Incentive Architecture: Comparative Advantage and Liability Reduction

3.4.13 The Negotiation and Redistribution Function

3.4.14 Governance Structure (Design Principles, Not Detailed Specification)

3.4.15 The Safety-Enhancing Data as Public Goods Function

3.4.16 Summary of Design Choices and Their Justifications

4. Key Insights and Innovations

Innovation 1: The Digital Commons as an Extractable Resource — Diagnosing the Structural Origin of AI's Negative Externalities

Innovation 2: Data Poisoning as Governance Infrastructure — Repurposing Security Vulnerabilities as Enforcement Tools

Innovation 3: The Inference-Time Compute vs. Pretraining Compute Substitutability Frontier — Empirically Characterizing Where Test-Time Strategies Can Compensate for Smaller Models

Innovation 4: Difficulty-Conditioned Allocation as a Meta-Strategy — Generalizing Compute-Optimal Scaling Beyond the Training Distribution

Innovation 5: Verifier Over-Optimization as the Primary Bottleneck — Refocusing Test-Time Compute Research on Verifier Quality Rather Than Search Sophistication

5. Experimental Analysis

Evaluation Methodology

Main Quantitative Results

The Paper's Empirical-Citation Structure

What "Results" Would Look Like for This Paper

Ablation Studies and Robustness Checks

Critical Assessment

6. Limitations and Trade-offs

The Verification Regime Has No Formal Security Guarantees Against Adversarial Evasion

The Verification and Data Collection Costs Are Unquantified and Potentially Prohibitive

The Trust's Leverage Depends on Human-Generated Data Remaining a Bottleneck, a Condition the Paper Acknowledges May Be Temporary

The Proposal Provides No Empirical Evidence That Certification or Consumer Pressure Would Incentivize Developer Compliance

The Governance Structure Is Left Largely Unspecified, Leaving Open the Risk That the Trust Itself Becomes an Unaccountable Gatekeeper

The National Scope Creates Jurisdictional Arbitrage Risks and Limits the Trust's Ability to Control Global Training Data

7. Implications and Future Directions

How This Work Changes the Landscape

Follow-Up Research This Work Enables

Practical Applications and Downstream Use Cases

Adjacent in domain

ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation

GPT-4 Technical Report