MOSEL: 950,000 Hours of Speech Data for Open-Source Speech Foundation Model Training on EU Languages

ArXiv: 2410.01036

🎯 Pitch

No existing speech foundation model truly qualifies as “open source” because training data licenses restrict commercial use or modification—this paper fixes that by releasing 950k hours of license‑vetted speech across all 24 EU languages, including 441k hours with new permissive transcripts, and demonstrates that even a near‑zero‑resource language like Maltese can be bootstrapped to ~24% word error rate using only these OS‑compliant resources.

---

1. Executive Summary

This paper introduces MOSEL (Massive Open-source compliant Speech data for the European Languages), a curated inventory of 950,000 hours of speech training data across the 24 official EU languages, all verified to carry licenses compliant with the Open Source AI Definition — requiring freedoms to use, study, modify, and share without restrictions on derivative works or commercial use. The work surveys existing automatic speech recognition datasets and unlabeled corpora, identifying 18 OS-compliant sources totaling 505.7k labeled hours and 444.5k unlabeled hours, and augments the latter by releasing automatically generated transcripts (pseudo-labels produced via Whisper large v3) for 441k hours under the permissive CC-BY license. A proof-of-concept on Maltese — one of the lowest-resourced languages — demonstrates that training on the collected labeled data plus pseudo-labeled data yields a word error rate of ~39% (roughly halving Whisper's ~80% baseline), with simple filtering further reducing it to ~24%, establishing that the inventory constitutes a viable bedrock for training an open-source speech foundation model even in extremely low-resource regimes where the base model used for pseudo-labeling itself performs poorly.

2. Context and Motivation

The Core Problem: No Existing Speech Foundation Model Is Actually Open Source

The fundamental gap this paper addresses is deceptively simple: despite widespread claims of openness, no existing speech foundation model satisfies the formal definition of open-source AI. The paper identifies this as a concrete, definitional failure — not merely a philosophical complaint. The Open Source Initiative's definition requires four specific freedoms (use, study, modify, and share — all for any purpose, including commercial use) and mandates that three artifacts be available under OS-compliant terms: the model weights, the training and inference code, and the training data itself. Existing SFMs fail on different legs of this tripod.

The paper names specific offenders to ground the diagnosis. SeamlessM4T's model is released under a license that is simply "not OS compliant" (Section 1). Whisper releases model weights and inference code under the permissive Apache 2.0 license, but the training code and training data are not public — that is, you can run the model but you cannot reproduce or extend it. OWSM comes closest to meeting the requirements but was trained on datasets including MuST-C and SPGISpeech, which carry licenses restricting derivative works or commercial use, thereby contaminating the entire pipeline with non-OS-compliant provenance. The paper's claim is binary and provocative: to the best of the authors' knowledge, no current SFM can legitimately call itself an Open Source Speech Foundation Model (OSSFM).

This is not an academic distinction. The paper ties the open-source definition directly to regulatory and societal pressures. The EU AI Act (European Parliament, 2023) and other governance frameworks create legal obligations around transparency, auditability, and fairness. Models that are merely "weights-available" but trained on opaque or restrictively licensed data frustrate these goals: a researcher cannot audit the training data for bias, a company cannot legally build commercial derivatives on NC-licensed foundations, and a regulator cannot verify claims about data provenance. The gap the paper identifies is therefore simultaneously a compliance problem, a reproducibility problem, and a barrier to both commercial innovation and scientific scrutiny.

Why This Matters: The Strategic Importance of EU Language Coverage

The paper's focus on the 24 official EU languages is not arbitrary — it reflects a deliberate intersection of policy, linguistic diversity, and market failure. The EU's language policy mandates equal status for all 24 official languages, but the economics of commercial speech technology development naturally concentrate investment on high-resource languages (English, German, French) while leaving languages like Maltese, Irish, and Estonian severely underserved. This creates what the paper implies is a market failure: the private sector will not, on its own, build open-source speech infrastructure for languages with small speaker populations because the return on investment is insufficient. Yet precisely these languages need infrastructure most — they lack the commercial ecosystems that would produce proprietary alternatives.

The paper positions an OSSFM as the mechanism to bridge this gap. By providing a fully open, legally unencumbered foundation model, downstream developers could build speech applications (transcription, voice assistants, accessibility tools, meeting summarization — note the paper's acknowledgment of the Meetween project in Section 6) for any EU language without navigating a thicket of incompatible or restrictive licenses. This matters for government services (which may be legally required to support all official languages), for minority language communities (who cannot rely on commercial providers to serve their needs), and for European technological sovereignty (reducing dependence on non-European companies whose training data practices are opaque).

Where Prior Approaches Fall Short: The Licensing Labyrinth

The paper's survey work reveals that the primary obstacle to building an OSSFM is not the absence of speech data — there is plenty of data — but rather the near-universal presence of licensing terms that violate open-source principles. This is a structural problem in the speech research community, and the paper's contribution is partly to document its pervasiveness.

The Share-Alike (SA) trap. A substantial collection of datasets uses CC-BY-SA or similar share-alike licenses (Appendix C, Table 8). These include large resources like People's Speech (30,000 hours of English), ParlaSpeech-HR (1,816 hours of Croatian), and ARTHUR 1.0 (884 hours of Slovenian). The SA restriction requires that derivative works be distributed under the same license terms — which, for an AI model, means the model's license must be CC-BY-SA or a compatible license. This conflicts with the open-source requirement that the model can be used, modified, and shared "for any purpose" without downstream licensing constraints. A model trained on SA-licensed data is legally entangled with that data's license in perpetuity, which the Open Source AI definition explicitly rejects.

The Non-Commercial (NC) and No-Derivatives (ND) restrictions. An even larger set of widely used datasets carries NC or ND terms (Appendix C, Table 9). Major resources — MuST-C (504 hours), Europarl-ST (201 hours), mTEDx (679 hours), SPGISpeech (5,000 hours), BEA-Base (71 hours of Hungarian), MASRI (8 hours of Maltese), GOS (120 hours of Slovenian) — all carry restrictions that prevent commercial use or derivative works. For the lowest-resourced languages, this is particularly damaging: MASRI, one of the few available Maltese corpora, is NC-licensed and therefore cannot be used in a commercially distributable OSSFM. The paper notes that Irish has only 17 hours of labeled OS-compliant data collected in total — a direct consequence of the best available Irish resources (like CMU Wilderness and UK and Ireland English Dialect) carrying NC, ND, or SA terms.

The YouTube license problem. Several large-scale datasets derive from YouTube videos (GigaSpeech at 33,000 hours, VoxLingua107 at 1,352 hours, How-2 at 2,000 hours, MediaSpeech at 20 hours). These datasets inherit YouTube's standard license terms, which are not OS-compliant. The paper specifically notes this for GigaSpeech, which is distributed under Apache 2.0 but "is categorized as non-OS compliant since it contains YouTube videos under restrictive CC licenses." This is an important clarification: the dataset license and the data license are different things, and the latter governs what downstream models can legally be built. A model trained on GigaSpeech, even if the dataset's own wrapper license is permissive, is trained on data that carries restrictive terms.

The Bible corpus problem. CMU Wilderness and MaSS — two large multilingual resources (236 hours and 126 hours respectively) — derive from Bible recordings by the Faith Comes By Hearing organization, which imposes NC and ND terms of use. This locks a substantial amount of multilingual speech data (covering EU languages including Finnish, French, Polish, Portuguese, Spanish, and Swedish) behind restrictions that make it unusable for an OSSFM.

The practical consequence of this licensing landscape is that most of the data the speech community has been using to train models cannot legally be used in an open-source system. The paper's inventory makes this visible: Tables 8 and 9 collectively list resources far larger than the 18 OS-compliant datasets in Table 1. The gap between what researchers could use and what they can use under open-source terms is enormous, and it disproportionately affects lower-resourced languages where every available dataset matters.

The Pseudo-Labeling Bottleneck: Why Unlabeled Data Needs Transcription

Even after identifying 505.7k hours of OS-compliant labeled data, the paper's language coverage analysis (Table 2) reveals a severe imbalance. English alone accounts for 437,239 of those labeled hours (86.5%). Only 6 other languages — German, Spanish, French, Italian, Dutch, and Portuguese — exceed 3,000 labeled hours. At the bottom, 8 languages have fewer than 100 labeled hours: Irish (17), Maltese (19), Danish (20), Slovenian (32), Greek (35), Lithuanian (36), Swedish (58), and Estonian (60). Training a foundation model on such small supervised datasets would be impossible — the Maltese experiment in Section 4 explicitly confirms this, noting that the purely supervised model "failed to converge due to its limited size (16 hours)."

The solution is to leverage the 444.5k hours of OS-compliant unlabeled data, which is more evenly distributed across languages (Table 2, right column) and provides at least 8,000 hours for every language except Irish. However, unlabeled speech requires transcription before it can be used for supervised ASR training — a process known as pseudo-labeling or weak supervision. This is computationally expensive: the paper reports that transcribing 441k hours with Whisper large v3 required approximately 25,500 GPU hours on NVIDIA A100s, translating to over $100,000 on commercial cloud services and an estimated 35,625 kg CO2 equivalent emissions.

The paper frames its decision to release these pseudo-labels as motivated not just by convenience but by Green AI principles: avoiding duplicated effort across different institutions reduces the overall environmental impact and cost. If every team aiming to build an EU-OSSFM had to independently transcribe 441k hours of speech, the aggregate waste would be enormous. By releasing the transcripts under CC-BY, the paper provides a public good that eliminates this redundant computation for all downstream users.

How This Paper Positions Itself

The paper positions itself as the first step in a pipeline toward an EU-OSSFM, not as the final model itself. The title is explicit: "950,000 Hours of Speech Data for Open-Source Speech Foundation Model Training on EU Languages." The contribution is the data inventory and the pseudo-labels, not the trained model. This is a deliberate scoping decision that distinguishes the work from efforts like OWSM (which built a model but failed on licensing) and Whisper (which released a model but not the training pipeline). By focusing exclusively on data collection and licensing verification, the paper addresses the bottleneck that has prevented all prior efforts from achieving genuine open-source status.

The paper also positions itself within the broader open-source AI movement. The opening paragraphs of Section 1 reference the LLM open-source discourse — citing OLMo (Groeneveld et al., 2024), BLOOM (Workshop et al., 2023), and critiques of open-washing (Eiras et al., 2024; Liesenfeld and Dingemanse, 2024) — and explicitly analogizes the speech domain to the language model domain. The implication is that speech has lagged behind text in open-source adherence, and that the same standards being advocated for LLMs should apply to SFMs. The White et al. (2024) Model Openness Framework is cited as the standard against which existing SFMs fail.

Finally, the paper frames the EU language scope pragmatically: the 24-language constraint makes the problem tractable (compared to the 99-151 languages targeted by OWSM and Whisper) while addressing a coherent policy context where the need is acute and the regulatory framework (EU AI Act) provides strong incentives for compliance. Section 7 explicitly notes that extending to "hundreds of languages" toward a "universal OSSFM" is a natural next step, but the current work deliberately starts with a bounded, achievable scope.

3. Technical Approach

3.1 Reader Orientation

This paper is a data curation and release effort, not a modeling paper — it builds the legally and practically necessary training data inventory for a future open-source speech foundation model covering all 24 official EU languages, and its primary technical contribution is the systematic survey of speech datasets against open-source licensing criteria plus the large-scale pseudo-labeling (automatic transcription) of unlabeled data. The problem it solves is that no existing speech foundation model is genuinely open-source because its training data carries restrictive licenses (NC, ND, SA, or proprietary terms), so the solution has the shape of a legally verified inventory — identifying which datasets can be used, quantifying how much data exists per language, and filling the transcription gap for unlabeled material — rather than training a model.

3.2 Big-Picture Architecture (Diagram in Words)

The system (or more precisely, the data pipeline) has four major components:

1. License Compliance Filter — a systematic review of all known speech datasets for EU languages that checks each dataset's license terms against the Open Source Initiative's definition, accepting only those that permit use, study, modification, and sharing for any purpose (including commercial use) without downstream licensing restrictions. This is the gatekeeper that determines which data enters the inventory.

2. Dataset Inventory (MOSEL) — the resulting catalog of 18 OS-compliant speech corpora, totaling 505.7k hours of labeled data (speech with reference transcripts) and 444.5k hours of unlabeled data (speech only, no transcripts), organized by language and documented with license, hour counts, and labeling status in Table 1 and Table 2.

3. Pseudo-Labeling Engine — an ASR inference pipeline using Whisper large v3 (Apache 2.0 licensed) that transcribes the 444.5k hours of unlabeled speech (from VoxPopuli and LibriLight specifically, excluding YouTube-Commons due to quality issues), producing automatic transcripts released under the permissive CC-BY 4.0 license.

4. Validation Training Run — a proof-of-concept ASR model trained on the combined labeled and pseudo-labeled data for Maltese (the worst-case language with Whisper WER >80%) to demonstrate that the collected data actually works for model training, even when the pseudo-labeling model itself performs poorly on that language.

Information flows as follows: existing speech datasets enter the license filter → compliant datasets are cataloged with their language coverage and labeling status → unlabeled datasets are fed through the pseudo-labeling engine using Whisper large v3 → the resulting pseudo-labels are released under CC-BY alongside the inventory → a downstream trainer combines labeled and pseudo-labeled data to train an ASR model, with the Maltese experiment serving as a minimum-viability demonstration.

3.3 Roadmap for the Deep Dive

• First, the license compliance criteria — exactly which licenses are accepted and which are rejected, with the legal reasoning — because this is the paper's foundational contribution and determines which data enters the inventory at all.
• Second, the dataset survey process and inventory construction, including how the paper handles the distinction between dataset wrapper licenses and underlying data licenses, because this is where most prior efforts have made mistakes that this paper explicitly avoids.
• Third, the pseudo-labeling pipeline — Whisper configuration, inference infrastructure, language handling, segment splitting, cost accounting, and the release strategy — because this transforms 444.5k hours of unusable audio into usable training data and is the paper's largest computational investment.
• Fourth, the proof-of-concept ASR training on Maltese — model architecture, training configuration, data filtering techniques (LID-based language filtering, textual hallucination detection), and the progression from supervised-only to supervised-plus-pseudo-labeled to filtered-pseudo-labeled — because this validates that the inventory is not merely a legal exercise but produces practically useful training data even in the most adverse conditions.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a data curation and infrastructure paper whose core idea is that building a genuinely open-source speech foundation model requires first solving the legal provenance problem — identifying training data that carries no downstream restrictions — and then solving the practical coverage problem — generating transcripts for the large fraction of that data that is unlabeled, especially for low-resource languages where labeled data is scarce.

---

The License Compliance Criteria: What "Open-Source Compliant" Means in Practice

The paper's central technical contribution (as distinct from its empirical contribution) is the rigorous application of a specific, externally defined standard — the Open Source Initiative's Open Source AI Definition — to the speech dataset landscape. This is not a standard the authors invented; they are applying an existing definition and documenting which datasets survive it.

The Open Source AI Definition requires four specific freedoms (Section 1):

1. Use the system for any purpose without asking permission.
2. Study how the system works and inspect its components.
3. Modify the system for any purpose, including to change its output.
4. Share the system, with or without modifications, for any purpose.

Critically, the definition further requires that the model weights, training and inference code, and training data all be available under licenses that respect these four freedoms. The training data requirement is the one that prior SFMs have systematically failed, and it is the one this paper addresses.

The paper operationalizes these criteria as a concrete checklist applied to each dataset's license:

Accepted licenses: The paper accepts data released under the following terms because they impose no restrictions beyond attribution (i.e., acknowledging the source), which does not limit any of the four essential freedoms:

• CC-0 (Creative Commons Zero) — the most permissive, equivalent to public domain dedication; no conditions at all.
• CC-BY 4.0 (Creative Commons Attribution 4.0) — requires attribution but imposes no restrictions on use, modification, sharing, or commercial exploitation.
• CC-BY 3.0 — the earlier version of the same; accepted for the same reasons.
• Public Domain — works not subject to copyright (either because copyright has expired, was forfeited, or never applied); the paper treats this as functionally equivalent to CC-0, as confirmed in the Fishman (2006) citation.
• Apache 2.0 — a permissive software license that is OSI-approved; the paper uses this for the Whisper model that generates pseudo-labels and for the code release, and notes that Apache 2.0-licensed generated content can be re-released under any license of the user's choosing (which is why Whisper outputs can be released under CC-BY).

Rejected licenses: The paper explicitly rejects several license classes, each for a specific, legally grounded reason:

• CC-BY-NC (Non-Commercial): The NC clause prohibits commercial use, directly violating the "for any purpose" requirement in all four freedoms. A model trained on NC-licensed data cannot be commercially deployed, distributed, or monetized, which means the downstream model inherits a use restriction. This eliminates major datasets including AMI (CC-BY-NC 4.0), AudioCite.net (CC-BY-NC), BEA-Base (NC), FT Speech (NC), SPGISpeech (NC, 5,000 hours), MASRI (NC, 8 hours of Maltese), and SWARA (CC-BY-NC 4.0, 21 hours of Romanian).

• CC-BY-ND (No-Derivatives): The ND clause prohibits creating derivative works. Since a trained model is legally a derivative work of its training data (the model encodes statistical patterns extracted from the data), training on ND-licensed data would violate the license. This eliminates datasets including CMU Wilderness (NC, ND terms inherited from Faith Comes By Hearing), MaSS (NC, ND, 126 hours), and the various TED/TEDx corpora that carry CC-BY-NC-ND terms (TED-LIUM v3, TEDx Spanish, mTEDx).

• CC-BY-SA (Share-Alike): The SA clause requires that derivative works be distributed under the same license terms or a compatible license. For an AI model, this would mean the model must be released under CC-BY-SA (or a BY-SA Compatible License, as defined at creativecommons.org/licenses/by-sa/4.0/legalcode#s3b), which is not compliant with open-source terms because it imposes a downstream licensing restriction — users cannot modify and share the model for any purpose because they are constrained by the SA requirement. This eliminates datasets including ARTHUR 1.0 (CC-BY-SA 4.0, 884 hours of Slovenian), Vystadial (CC-BY-SA 3.0, 63 hours), ParlaSpeech-HR (CC-BY-SA, 1,816 hours of Croatian), and People's Speech (CC-BY-SA 4.0, 30,000 hours of English). The paper's footnote 6 explicitly quotes the CC-BY-SA legal code to justify this rejection: the model's license "must be a Creative Commons license with the same License Elements [...] or a BY-SA Compatible License," which conflicts with open-source terms.

• YouTube-derived data (YouTube Standard License): Several datasets are compiled from YouTube videos and therefore inherit YouTube's standard license terms, which are not OS-compliant. This affects GigaSpeech (33,000 hours, Apache 2.0 wrapper but YouTube-derived), How-2 (2,000 hours), VoxLingua107 (1,352 hours covering 23 EU languages), MediaSpeech (20 hours of French), and mTEDx (whose videos come from YouTube). The paper carefully distinguishes between the dataset's own wrapper license (e.g., GigaSpeech is distributed under Apache 2.0) and the license of the underlying data.

• Project Gutenberg License (M-AILABS): M-AILABS (867 hours) derives from Project Gutenberg audiobooks, whose license terms are not OS-compliant.

• Custom restrictive terms: Some datasets carry custom terms rather than standard Creative Commons licenses. Faith Comes By Hearing imposes NC and ND terms on Bible recordings used in CMU Wilderness and MaSS. The GOS corpus (120 hours of Slovenian) uses CC-BY-SA-NC 2.5, a custom combination that is rejected on both SA and NC grounds. PDTSC1.0 (122 hours of Czech) uses CC-BY-NC-SA 4.0, rejected on both NC and SA grounds.

The collection vs. underlying data distinction: The paper makes an important technical-legal clarification about how datasets containing mixed-license content are treated. It notes (Section 2) that "CC licenses allow licensed material to be included in collections [...], however this does not change the license applicable to the original material." This means that if a dataset's transcripts are CC-BY but the corresponding speech recordings carry non-OS terms, the dataset as a whole is non-OS-compliant because the speech — the actual training data — is restrictively licensed. The paper applies this principle: "in cases where the transcripts are OS compliant (e.g., CC-BY where only attribution is required) but the corresponding speech (or part of it) is not, we document the dataset under the most restrictive license." This is a conservative interpretation that ensures downstream users do not inadvertently train on data with hidden restrictions. GigaSpeech is the most prominent example: released under Apache 2.0, but categorized as non-OS-compliant because its YouTube-sourced audio carries restrictive terms.

Why this framework matters operationally: The license filter is not a pedantic exercise — it is the mechanism that determines the entire inventory. The paper's Tables 8 and 9 collectively list datasets totaling many times the size of the accepted 18 datasets, meaning that most of the speech data the research community has been using cannot legally be used in an open-source model. The primary contribution of the paper is making this distinction explicit and actionable, so that downstream model builders have a clear, legally verified set of permitted data rather than having to perform their own (potentially erroneous) license review.

---

Dataset Survey Process and Inventory Construction

The inventory construction involves systematically identifying, reviewing, and cataloging speech datasets for all 24 EU languages. The process is not fully automated — it requires manual license verification because dataset documentation is often inconsistent, and the relationship between wrapper licenses and underlying data licenses requires case-by-case legal judgment.

Survey scope: The survey covers two categories of data:

1. ASR datasets — corpora that include both speech recordings and reference (human-generated) transcripts. These are the primary training resource for supervised ASR.
2. Unlabeled speech corpora — collections of speech recordings without transcripts. These require pseudo-labeling to be useful for supervised training but can also be used for self-supervised pre-training (e.g., wav2vec-style models), though the paper focuses on the pseudo-labeling pathway.

The survey is explicitly limited to datasets that are freely accessible (excluding paid/commercial datasets) and whose license permits the necessary freedoms. The paper does not claim exhaustiveness — it acknowledges that the inventory "will be continuously updated" (Section 1) — but aims to capture all known major resources.

The resulting inventory (Table 1) consists of 18 datasets:

• Labeled datasets with transcripts: CommonVoice (CC-0, 6,732 hours, 23 EU languages), CoVoST2 (CC-0, 687 hours, 10 EU languages), CSS10 (Public Domain, 99 hours, 7 EU languages), EMU (CC-BY 3.0, 56 hours, Polish only), EU Parliament (CC-BY 4.0, 32 hours, Polish only), FLEURS (CC-BY 4.0, 215 hours, 23 EU languages), Large Corpus of Czech Parliament Plenary Hearings (CC-BY 4.0, 444 hours, Czech only), LibriTTS (CC-BY 4.0, 585 hours, English only), LibriSpeech (CC-BY 4.0, 360 hours, English only), LibriVoxDeEn (Public Domain, 547 hours, German only), MC Speech (CC-0, 22 hours, Polish only), MLS (CC-BY 4.0, 50,687 hours, 8 EU languages), SIWIS (CC-BY 4.0, 11 hours, French only), Speech Commands (CC-BY 4.0, 18 hours, English only), VCTK (CC-BY 4.0, 44 hours, English only), and notably YouTube-Commons (CC-BY 4.0, 443,396 hours, 19 EU languages) and VoxPopuli (CC-0, 383,500 hours, 23 EU languages).

• Unlabeled datasets without transcripts: LibriLight (Public Domain, 57,706 hours, English only) and VoxPopuli (the same dataset listed above, which is unlabeled — Table 1 reports it as providing 383,500 hours total, containing both labeled hours and unlabeled hours; the paper's Table 2 separates these into 1,791 labeled hours and the remainder as unlabeled).

• Note on VoxPopuli's dual role: Table 1 lists VoxPopuli as providing 383,500 (in the Hours column) and 1,791 (in the Label column, indicating the subset with reference transcripts). Table 2 then breaks this down per language, showing total labeled hours (1,791 across 16 languages, from the VoxPopuli ASR subset) and unlabeled hours (the remainder of the 383,500, distributed across 23 languages). This means VoxPopuli is the single largest contributor to both the labeled and unlabeled portions of the inventory.

• YouTube-Commons caveat: The paper flags significant quality issues with YouTube-Commons (443,396 hours, CC-BY 4.0): manual inspection revealed that it "includes videos without speech (e.g., with only music)," "the language identification (LID) tag and the transcripts are often inaccurate," and "sentence-level segmentation of the speech is not provided (it contains unsegmented transcripts for the entire YouTube videos)." The paper notes that "further checks and processing work would be needed to effectively exploit the dataset for OSSFM training." This dataset is counted in the total inventory but is excluded from the pseudo-labeling effort (the paper pseudo-labels only VoxPopuli and LibriLight) and is not used in the Maltese proof-of-concept. Its inclusion in the inventory total is therefore somewhat aspirational — the data exists and is OS-compliant, but its quality is insufficient for immediate use without substantial cleaning.

Language coverage quantification (Table 2) is the paper's primary analytical output. It aggregates across all 18 accepted datasets to produce per-language totals for labeled hours, unlabeled hours, and combined hours. The labeled distribution is heavily skewed — English dominates with 437,239 hours (86.5%), followed by French (26,984), Spanish (17,471), German (9,236), Portuguese (5,492), Italian (3,756), and Dutch (3,395). All other languages have fewer than 600 labeled hours, with Irish at only 17 hours (all labeled, from CommonVoice — the paper found no unlabeled OS-compliant Irish data at all).

The unlabeled distribution (from VoxPopuli and LibriLight) is substantially more balanced: English has 84,704 unlabeled hours, but every other language except Irish has at least 8,000 hours, with most languages clustering in the 10,000–23,000 hour range. This more even distribution is what makes the unlabeled data crucial for low-resource languages — without it, languages like Maltese (19 labeled hours, 9,100 unlabeled hours), Estonian (60 labeled, 10,604 unlabeled), and Lithuanian (36 labeled, 14,400 unlabeled) would have essentially no usable training data.

Design choice: conservative license interpretation: The paper errs on the side of excluding data when the license situation is ambiguous. For datasets that combine content under multiple licenses, it applies the most restrictive license to the entire dataset. This is a deliberate choice to ensure that downstream model builders face no legal risk — a model trained only on MOSEL-inventoried data should be distributable under any OS-compliant license without restriction. The cost of this conservatism is excluding substantial data (as evidenced by the long lists in Tables 8 and 9), but the paper implicitly argues that ambiguous inclusion would defeat the purpose of building a verifiably open-source model.

Design choice: EU language scope: The 24-language constraint serves multiple purposes. Practically, it makes the survey tractable (compared to surveying all ~7,000 world languages). Strategically, it aligns with EU policy priorities (the EU's official language equality mandate, the EU AI Act's transparency requirements) and with the paper's funding sources (the Meetween and FAIR projects, both EU-funded). Scientifically, it provides a well-defined coverage target that exposes precisely where data scarcity is most severe (Irish, Maltese) and where it is manageable (the major European languages).

---

The Pseudo-Labeling Pipeline

The pseudo-labeling pipeline transforms 444.5k hours of unlabeled speech (all from VoxPopuli and LibriLight, excluding YouTube-Commons as noted) into transcribed training data. The paper treats this as both a practical contribution (saving downstream users the computational cost) and a principled one (releasing the transcripts under CC-BY ensures the entire pipeline remains OS-compliant).

Model choice: Whisper large v3. The paper uses OpenAI's Whisper large v3 model, which is released under the Apache 2.0 license. Apache 2.0 is an OSI-approved open-source license, and critically, it permits the user to release the model's outputs under any license of their choosing. This is the legal mechanism that allows the paper to take a model trained on (presumably) non-OS-compliant data and use it to generate OS-compliant data: the Apache 2.0 license on the model covers the act of generation, and the generated content is not itself subject to the model's training data licenses. The paper is explicit about this: "Whisper large v3, which is released under the OS Apache 2.0 License that allows the generated content to be released under any license."

Inference configuration: The paper specifies the following Whisper invocation:

• Language ID forcing: "feeding Whisper with the corresponding language ID." This means the model is told explicitly which language to transcribe (using VoxPopuli's or LibriLight's metadata) rather than relying on Whisper's built-in language detection. This is important because automatic language identification can fail, particularly for closely related languages or low-resource languages, and a mis-identified language would produce garbled transcripts. For VoxPopuli, each segment carries a language tag in the dataset metadata, which is used directly.
• Timestamp suppression: The <|notimestamp|> token is provided, which instructs Whisper not to generate word-level timestamps. This likely reduces computational overhead and prevents the model from spending capacity on a task (timestamp prediction) that is irrelevant for training data generation.
• Beam size: Set to 5. Beam search with width 5 means the model maintains 5 candidate hypothesis sequences at each decoding step, selecting the one with the highest overall likelihood at the end. This is a modest beam width that balances transcription quality against computational cost.
• HuggingFace integration: Whisper large v3 is run via HuggingFace transformers version 4.38.2, specifically openai/whisper-large-v3.

Segment splitting for LibriLight: LibriLight contains audio segments longer than Whisper's 30-second maximum input duration (Whisper's architecture uses a fixed positional encoding that limits input length). The paper handles this by splitting long segments "into chunks of up to 30s each." The specific splitting strategy (overlap, alignment preservation, etc.) is not described in detail, which is a minor gap — the exact chunking method can affect transcription quality at boundaries.

Computational infrastructure: Inference is executed on NVIDIA A100 64GB GPUs. The paper achieves a throughput of "∼1.5-2k samples per GPU hour" by fitting 16 samples per batch and enabling FlashAttention (Dao et al., 2022), which reduces the memory footprint and speeds up the attention computation in the Transformer architecture. FlashAttention is particularly important for long sequences (Whisper processes audio at high temporal resolution, producing long token sequences) and for high-throughput batched inference, since attention computation is the primary bottleneck.

Cost analysis: The paper provides unusually detailed cost accounting, reflecting its Green AI motivation (the Schwartz et al., 2019 citation in Section 3):

• Total GPU hours: ∼25,500 GPU hours. This is the aggregate across all languages and all unlabeled data.
• Cloud cost equivalent: "On popular cloud services such as AWS, this would translate to >100k USD." The footnote clarifies that this is based on AWS P4 instance pricing (8 A100 GPUs at >32 USD per hour as of June 10, 2024).
• Carbon emissions: 35,625 kg CO2 equivalent, estimated using the Machine Learning Impact calculator from Lacoste et al. (2019). This is a substantial but not extraordinary environmental cost — equivalent to roughly 8 passenger vehicles driven for a year, using typical conversion factors — and the paper frames it as a cost that should be paid once (by the authors) rather than repeatedly (by every downstream team that would otherwise need to do the same transcription).

Data release format and licensing: The pseudo-labeled data is released under CC-BY 4.0 on HuggingFace at hf.co/datasets/FBK-MT/mosel. CC-BY requires only attribution, which is fully OS-compliant. The release includes the automatic transcripts paired with their corresponding audio segments, preserving the original dataset's segment boundaries (for VoxPopuli) or the 30-second chunks (for LibriLight). The paper also releases the inference code under Apache 2.0 on GitHub to ensure reproducibility.

What is NOT pseudo-labeled: The paper explicitly excludes YouTube-Commons from pseudo-labeling "due to the issues described in §2" — namely, the presence of non-speech content, inaccurate language tags, and lack of sentence-level segmentation. This is a pragmatic decision: generating transcripts for data where the language is uncertain and much of the audio is music or noise would produce mostly garbage pseudo-labels that could harm downstream training. The paper also does not pseudo-label any data outside VoxPopuli and LibriLight (e.g., CommonVoice has labeled data, and other datasets are either labeled or not OS-compliant).

Per-language pseudo-labeled hours (Table 4): The release covers 23 languages (all EU languages except Irish, which has no unlabeled data). The distribution closely follows the unlabeled data distribution from Table 2, with English having the most (81,806 hours) and the smallest languages having around 8,000–10,000 hours (Croatian 8,100, Maltese 9,100). The totals in Table 4 sum to 441,206 hours, which is slightly less than the 444,467 unlabeled hours from Table 2 — the discrepancy likely reflects slight differences in how hours are counted after segment splitting, or segments that failed transcription (produced no output). The paper does not explicitly account for this 3,261-hour gap, which is less than 1% of the total and may be attributable to segments that were too short, contained only silence, or caused Whisper inference failures.

Whisper's baseline performance on EU languages (Table 7, Appendix D): To contextualize the quality of the pseudo-labels, the paper reports Whisper large v3's word error rate (WER) on two standard test sets across all EU languages where data is available. The results reveal enormous variation:

• Maltese: 80.8 WER on CommonVoice test, 73.8 on FLEURS — essentially unusable for many practical applications, but as Section 4 shows, still useful as training data after filtering.
• Estonian: 29.9 on CommonVoice, 18.1 on FLEURS — substantially worse than high-resource languages but far better than Maltese.
• Lithuanian: 27.6 on CommonVoice, 23.7 on FLEURS.
• Latvian: 16.7 on CommonVoice, 19.4 on FLEURS.
• Slovenian: 16.8 on CommonVoice, 18.3 on FLEURS.
• Slovak: 23.4 on CommonVoice, 9.2 on FLEURS.
• High-resource languages: English (9.3/4.1), Dutch (4.3/5.2), German (5.7/4.9), Italian (5.5/3.0), Spanish (4.7/2.8) — all below 10 WER.

Why this variation matters: The pseudo-label quality directly determines the quality of models trained on the pseudo-labeled data. For high-resource languages where Whisper achieves <10 WER, the pseudo-labels are essentially near-human quality, and training on them should work well. For the low-resource languages with >20 WER, the pseudo-labels are noisy, and training on them may propagate errors or require filtering (as demonstrated in the Maltese experiment). The paper's contribution of pseudo-labels is therefore most valuable for exactly the languages where Whisper performs worst — because those languages have the least labeled data and need the pseudo-labels most, even if those labels are imperfect. The Maltese experiment (Section 4) directly validates that even 80% WER pseudo-labels can be useful when combined with filtering.

---

The Maltese Proof-of-Concept: Model Training and Data Filtering

Section 4 and Appendix B describe a small-scale ASR training experiment on Maltese designed to validate that the MOSEL data (labeled plus pseudo-labeled) can actually train a working model. Maltese is chosen deliberately as the hardest case: it has only 19 hours of labeled data, Whisper's WER on Maltese is >80%, and the language is Semitic (structurally different from Indo-European languages, meaning transfer from other languages is less effective).

Model architecture: The paper trains a sequence-to-sequence model with a hybrid CTC-attention loss:

• Encoder: 12-layer Conformer (Gulati et al., 2020), a variant of the Transformer architecture that interleaves self-attention with convolutional modules within each layer, which has been shown to be particularly effective for speech because the convolutions capture local spectral-temporal patterns that pure attention may miss. The convolutional modules use a kernel size of 31 features ("31-feature kernel").
• Decoder: 6-layer standard Transformer decoder (Vaswani et al., 2017).
• Input preprocessing: Two 1D convolutional layers with stride 2 and kernel size 5 precede the Conformer encoder, which subsamples the input by a factor of 4 (two strides of 2), reducing the sequence length before the computationally expensive Conformer layers. The raw input is 80-dimensional Mel-filterbank features extracted every 10 ms with a 25 ms window.
• Embedding dimension: 512. Feed-forward dimension: 2048 (the standard 4× expansion ratio in Transformer architectures).
• Target vocabulary: 8,000 subword units built with SentencePiece (Kudo, 2018), applied to the Maltese text.
• Total parameters: 116M — a relatively small model by modern standards, reflecting the limited training data and the proof-of-concept nature of the experiment.

Loss function: The model is trained with a combination of two losses, which is standard practice in modern end-to-end ASR:

$\mathcal{L} = \mathcal{L}_{\text{CE}} + 0.5 \cdot \mathcal{L}_{\text{CTC}}$

where $\mathcal{L}_{\text{CE}}$ is the label-smoothed cross-entropy loss on the decoder output and $\mathcal{L}_{\text{CTC}}$ is the Connectionist Temporal Classification loss computed on the output of the 8th encoder layer.

What it computes: The cross-entropy term measures how well the decoder's autoregressive predictions match the reference transcript, with label smoothing (factor 0.1) preventing overconfidence by replacing the one-hot target distribution with a slightly smoothed distribution (0.9 on the correct token, 0.1 distributed across all others). The CTC loss is applied to the encoder's intermediate representations (layer 8, not the final layer), which provides an auxiliary training signal that helps the encoder learn to align the acoustic input with the output sequence without requiring the decoder to converge first — this is known to speed up training and improve final performance, especially in low-resource settings.

Why this form: The 0.5 weight on the CTC loss balances the two objectives. If the CTC weight were too high, the model might optimize for the simpler CTC task (which assumes conditional independence between output tokens) at the expense of the more powerful but harder-to-train autoregressive decoder. If the CTC weight were too low, the auxiliary benefit would be negligible. The choice of layer 8 (out of 12) for the CTC head is a common design: earlier layers might not have extracted sufficient linguistic information, while later layers might be too specialized for the decoder's needs.

Optimization: Adam optimizer with $\beta_1 = 0.9$, $\beta_2 = 0.98$ (standard for Transformer training), Noam learning rate scheduler (Vaswani et al., 2017) with peak learning rate 2e-3 and 25,000 warmup steps. The Noam scheduler linearly increases the learning rate from 0 to the peak over the warmup steps, then decays it proportionally to $1/\sqrt{\text{step}}$. Dropout is set to 0.1, weight decay to 0.001, and SpecAugment (Park et al., 2019) — a data augmentation technique that masks random time steps and frequency bands in the input spectrogram — is applied during training to improve robustness.

Encoder initialization: To bootstrap training given the tiny amount of Maltese data, the Conformer encoder weights are initialized from a similar ASR model trained on 4,000 hours of labeled English data (CommonVoice, LibriSpeech, CoVoST, VoxPopuli). This is a form of transfer learning: the encoder has already learned to extract phonetically meaningful features from English speech, and those features partially transfer to Maltese because the human vocal tract is universal, even though Maltese phonology differs from English. The decoder is trained from scratch on Maltese text.

Training scale: Mini-batches of 40,000 tokens, update frequency 2 (accumulating gradients over 2 batches before updating parameters), 4 NVIDIA A100 GPUs (64GB RAM), 150,000 updates (equivalent to approximately 300,000 effective batches), with final model weights obtained by averaging the last 7 checkpoints (a common regularization technique that reduces variance from the stochastic optimization trajectory).

Supervised-only failure: The paper reports that training on only the 16 hours of labeled Maltese data "failed to converge." This is expected: with only 16 hours of labeled speech, the model has insufficient signal to learn the mapping from acoustics to text, especially for a language with complex phonological properties. This failure motivates the use of pseudo-labeled data.

Pseudo-labeled data integration: The model is trained on a combination of the labeled Maltese data (16 hours) and the full 9,100 hours of pseudo-labeled Maltese data. Despite the pseudo-labels having ~80% WER (meaning most words in the automatic transcripts are incorrect), the model achieves 39.4 WER on CommonVoice and 38.9 on FLEURS — roughly halving Whisper's error rate. This demonstrates a counterintuitive but important phenomenon: noisy pseudo-labels can still provide useful training signal, especially when the alternative is no data at all. The model learns to extract signal from the noise, effectively averaging over many examples where the pseudo-labels are wrong in different ways. This is analogous to the well-known finding in semi-supervised learning that imperfect labels from a teacher model can improve a student model if the teacher is at least partially informative and the student has sufficient capacity to model the underlying distribution.

Filtering strategies: The paper applies two simple filters to the pseudo-labeled data to further improve training quality:

Language identification (LID) filtering: Since the Whisper transcription forces the language tag based on VoxPopuli's metadata, any segment where the actual spoken language differs from the metadata will receive garbled transcripts (the model will try to interpret, say, English speech as Maltese). The paper runs language identification using the same Whisper large v3 model (without forcing the language tag, letting the model predict the language naturally) and filters out any segment where the predicted language is not Maltese. Table 5 shows the LID results on the Maltese section of VoxPopuli:

This means approximately 23% of what VoxPopuli labels as Maltese is actually in other languages, primarily English. Removing these segments eliminates a source of systematic noise (forcing Maltese transcription on non-Maltese audio). The paper notes that the LID output for all transcribed data will be released alongside the transcripts, enabling downstream users to apply their own filtering thresholds.

Textual hallucination filtering: The paper identifies a specific failure mode in Whisper's outputs — "nonsensical hallucinations" where the model generates text that "fails to convey any relevant or comprehensible information" (Ji et al., 2023). The paper describes several patterns:

• Repetitive short words: Sequences like "Hey, hey, hey, here, hey. No, no, no, no, no, no, no, no." (from Table 6, line 2). These typically occur when the audio contains background noise or music rather than speech, and Whisper produces low-information filler rather than remaining silent.
• Long noisy strings: Patterns like "T-J-N-D-F-Z-3-2-8-W-M-L-G-0-Z-P-[...]" — sequences of letters, numbers, and special characters that represent the model's uncertain "guesses" when the audio is unintelligible.
• Repetitive religious or formulaic text: "Amen.Amen.Amen.Amen.Amen.Amen.[...]" — likely triggered by religious audio content that the model recognizes as formulaic but cannot transcribe accurately.
• Single common words: For some languages, entire audio segments are transcribed as a single, very frequent word — "Dˇekuji" (thank you) for Czech, "Aˇci¯u" (thank you) for Lithuanian. While sometimes correct, these are flagged as suspicious because they occur disproportionately in low-confidence regions.

The paper flags segments containing these patterns, with the option of filtering them out during training.

Impact of filtering (Table 3):

Filtering reduces WER from ~39 to ~24 — an additional 40% relative improvement. This demonstrates that while raw pseudo-labeled data is useful, targeted cleaning (removing language-mismatched segments and hallucinated transcripts) substantially improves training outcomes. The model trained on filtered data approaches usable accuracy (24% WER), which is remarkable given it started from only 16 hours of labeled data and used pseudo-labels from a model that itself achieved 80% WER.

Implications for the broader inventory: The Maltese experiment serves as a lower-bound validation: if the pseudo-labels work for Maltese (the worst case — highest Whisper WER, fewest labeled hours), they should work even better for languages where Whisper achieves lower WER and where more labeled data is available for supervised fine-tuning. The dramatic improvement from filtering also suggests that the paper's release of LID outputs and hallucination flags enables downstream users to apply similar cleaning to all languages, not just Maltese.

Reproducibility commitments: The paper commits to releasing the inference code (under Apache 2.0, with the repository at https://github.com/hlt-mt/FBK-fairseq/), the LID outputs for all transcribed data, the hallucination detection metadata, and the pseudo-labels themselves (under CC-BY 4.0). Evaluation uses the Whisper Normalizer for text normalization and JiWER for WER computation, ensuring consistency with standard ASR evaluation practices.

---

Design Choices and Their Justifications Across the Pipeline

Why Whisper large v3 rather than a custom or open-source ASR model: The paper uses Whisper large v3 because it is (a) released under Apache 2.0, satisfying the OS-compliant tooling requirement, (b) state-of-the-art or near-SOTA across all tested languages, providing the highest possible pseudo-label quality, and (c) widely available and well-documented, reducing the barrier to reproducibility. The alternative — training a custom ASR model for pseudo-labeling — would itself require OS-compliant training data, creating a chicken-and-egg problem. Whisper solves this by providing a pre-trained model under a permissive license.

Why CC-BY 4.0 for the pseudo-label release: CC-BY requires only attribution, imposes no restrictions on use, modification, sharing, or commercial exploitation, and is OSI-compatible. The alternatives (CC-0 or Public Domain) would be even more permissive but might raise concerns in some jurisdictions about the provenance of machine-generated content; CC-BY provides a minimal attribution requirement that is legally conventional and practically easy to satisfy (cite the MOSEL dataset).

Why 16 samples per batch and FlashAttention: The throughput optimization is motivated by the scale of the task (441k hours, ~25,500 GPU hours even with optimization). Without FlashAttention, the memory footprint of the attention computation would be higher, forcing smaller batch sizes and reducing throughput. FlashAttention reduces the memory complexity from $O(n^2)$ to $O(n)$ in the sequence length $n$ for the attention computation, which is significant because Whisper processes long audio sequences at high temporal resolution.

Why Maltese for the proof-of-concept: Maltese is chosen because it is simultaneously (a) one of the lowest-resourced EU languages in terms of labeled data (19 hours), (b) the language where Whisper performs worst (>80% WER), and (c) structurally different from most other EU languages (a Semitic language with significant Romance vocabulary, written in Latin script). If the approach works for Maltese under these adverse conditions, it is likely to work for all other EU languages.

Why LID filtering and hallucination detection rather than more sophisticated data cleaning: The paper explicitly notes that "more advanced filtering techniques can provide further benefits" but that "data cleaning and normalization are common steps in training pipelines, going beyond the scope of this work." The simple filters are provided as a baseline that demonstrates the principle — that even crude cleaning helps — while leaving more sophisticated approaches (e.g., confidence-based filtering, consensus-based filtering using multiple ASR models, prosodic consistency checks) to downstream users.

Why the Conformer architecture: The Conformer is chosen as a well-established, strong architecture for ASR that combines the global context modeling of Transformers (via self-attention) with the local spectral-temporal pattern recognition of CNNs (via depthwise convolutions within each layer). For Maltese, with limited training data, the inductive bias from the convolutional modules may be particularly helpful — it provides a structural prior that nearby time-frequency bins are related, reducing the amount of data needed to learn this from scratch.

4. Key Insights and Innovations

Innovation 1: Redefining "Open Source" for Speech as a Licensing-Verification Problem, Not an Access Problem

The paper's most fundamental intellectual move is reframing the question "Is this model open source?" from a matter of weight availability to a matter of training data licensing provenance. Prior to this work, the speech community's discourse around openness had been largely performative: OWSM claimed to be "open" because it released weights and training code, Whisper claimed openness because its weights and inference code carried an Apache 2.0 license, and the field generally operated under the assumption that weight-release-plus-code-release constituted sufficient openness. The paper argues — convincingly, with formal reference to the Open Source Initiative's definition — that this is insufficient and indeed misleading, because the training data carries restrictive licenses that propagate downstream.

This is not merely a classification exercise. The paper's diagnostic move is to show that every existing SFM fails the OS test for a different reason, and that the failure modes are systematic rather than accidental. SeamlessM4T fails on model licensing. Whisper fails on training data and code transparency. OWSM fails on data licensing (MuST-C's CC-BY-NC-ND, SPGISpeech's NC terms). These are not edge cases or minor oversights — they are structural consequences of the speech community's reliance on datasets that were collected and released under terms designed for academic research, not for open-source AI development. The NC clause, in particular, is pervasive in speech corpora (it appears in major resources like AMI, BEA-Base, FT Speech, SPGISpeech, MASRI, AudioCite.net, and many others listed in Table 9), and it was largely invisible as a problem until the paper made it visible by applying the OSI standard rigorously.

The significance of this reframing extends beyond the paper's own inventory. It establishes a methodology for licensing verification that any downstream model builder can apply: (1) identify the four required freedoms, (2) check the training data license against each freedom, (3) treat the underlying data license as the binding constraint regardless of wrapper licenses, (4) reject SA terms for imposing downstream license propagation. This methodology is transportable to other domains (text, vision, code) and establishes a standard that was missing from prior open-source AI discourse in speech. The paper's Tables 8 and 9 — which collectively list datasets far larger than the accepted 18 — are not just appendices; they are the evidentiary backbone of the diagnosis, showing that the licensing landscape is the primary barrier to open-source speech AI, not data scarcity.

This innovation is at least partly fundamental rather than incremental, because it shifts the object of study from "how do we get more data?" to "how do we verify that the data we have is legally unencumbered?" — and in doing so, it reveals that the dominant approach to speech dataset construction over the past two decades has inadvertently created a legal minefield for open-source model development.

Innovation 2: The Demonstration That ~80% WER Pseudo-Labels Can Still Yield Substantial Training Gains — With a Bounded Failure Mode

The paper's Maltese experiment (Section 4, Table 3) establishes an empirical finding that is genuinely counterintuitive and has implications far beyond this paper: pseudo-labels generated by a model with ~80% word error rate — meaning most words in the automatic transcripts are incorrect — can still approximately halve the error rate of a model trained on the combination of those pseudo-labels plus a tiny amount of labeled data. Starting from Whisper's 80.8 WER on CommonVoice Maltese, the model trained on labeled Maltese data (16 hours) plus pseudo-labeled data (9,100 hours) achieves 39.4 WER — a roughly 2× improvement over the teacher model. With simple filtering (LID-based language filtering plus hallucination removal), performance further improves to 23.8 WER — nearly a 3.4× improvement.

This finding challenges the intuition, common in the weak supervision and semi-supervised learning literature, that noisy labels are useful only when they are "mostly correct." The prior dominant assumption — embodied in work like that of Jia et al. (2019) on weakly supervised speech translation and Zhang et al. (2022) on semi-supervised ASR — was that pseudo-label quality must be reasonably high (e.g., <30% WER) for the training signal to outweigh the noise. The paper's Maltese result shows that even in a regime where the teacher model is essentially unusable as a standalone system (80% WER means less than one word in five is correctly transcribed), the student model can still extract useful signal because errors in the pseudo-labels are not uniformly distributed — they are correlated with acoustic difficulty, and the model can learn to attend to regions where the pseudo-label is more reliable, especially when a small amount of clean labeled data provides anchor points.

The paper does not develop this theoretical explanation (it is framed as a practical validation, not a theoretical contribution), but the empirical result itself represents a diagnostic finding with practical consequences: it means that the inventory's 444.5k hours of unlabeled data is genuinely useful for training, even for languages where the best available ASR system performs poorly. This directly addresses the concern that pseudo-labeling with a weak teacher would be a waste of computation for low-resource languages — the Maltese experiment shows the opposite.

However, the paper also establishes a clear boundary condition: the supervised-only model "failed to converge due to its limited size (16 hours)." This is the companion negative result that defines the regime: pseudo-labeling helps when the alternative is no usable model at all, but it does not substitute for at least some clean supervision. The 16-hour threshold for Maltese is not universal — it would vary by language complexity, model architecture, and labeled data quality — but the principle generalizes: pseudo-labels from a weak teacher work when they expand a small-but-nonzero labeled dataset into a large training corpus.

This innovation is incremental in its mechanism (pseudo-labeling for ASR is well-established; see Kahn et al., 2020, Zhang et al., 2022) but fundamentally informative in its empirical finding about the tolerable noise level. It provides a lower-bound on pseudo-label quality that is far below what prior work assumed was necessary, and it demonstrates this in a real-world extreme-low-resource setting (Maltese, 16 labeled hours, 80% WER teacher) rather than in a controlled or simulated regime.

Innovation 3: The Identification of License-Based Data Scarcity as the Primary Bottleneck for Open-Source Speech AI — Not Raw Audio Availability

The paper's survey reveals a structural pattern that was not previously documented in a systematic way: the amount of speech data that is both acoustically available and legally unencumbered for open-source model training is far smaller than the total amount of speech data that exists, and this gap is most severe for precisely the languages that need open-source infrastructure most. This is not a data scarcity problem in the traditional sense — there are thousands of hours of speech data for languages like Croatian (ParlaSpeech-HR has 1,816 hours), Slovenian (ARTHUR has 884 hours, GOS has 120 hours), Hungarian (BEA-Base has 71 hours), and Maltese (MASRI has 8 hours) — but all of these carry NC, ND, or SA restrictions that render them unusable for an OSSFM.

The paper quantifies this gap implicitly through the contrast between Table 1 (18 accepted datasets) and Tables 8 and 9 (31 rejected datasets). The rejected datasets collectively represent many times the total hour count of the accepted ones. For English alone, the rejected datasets include People's Speech (30,000 hours, SA-restricted), SPGISpeech (5,000 hours, NC-restricted), GigaSpeech (33,000 hours, YouTube-licensed), and several large TED-derived corpora (NC-ND-restricted). For the lowest-resourced languages, the losses are existential: Irish has 17 hours of accepted data because the only other available Irish resource is UK and Ireland English Dialect (CC-BY-SA), and CMU Wilderness (NC, ND) covers some Irish but is rejected. Maltese's MASRI corpus (8 hours) is rejected for NC terms.

The conceptual contribution here is to reframe the open-source speech problem from a data collection challenge to a licensing reform challenge. Building an OSSFM does not (primarily) require recording more speech — it requires convincing dataset creators, funding agencies, and the research community to release existing speech data under OS-compliant licenses. This insight has policy implications: organizations like the EU that fund speech data collection should mandate OS-compliant licensing as a condition of funding, because data released under NC or ND terms is effectively locked out of the open-source ecosystem. The paper does not make this argument explicitly, but it is the natural implication of the licensing analysis.

This innovation is fundamentally a diagnostic contribution rather than a methodological one. It provides the evidence base for a claim that many in the open-source AI community have made anecdotally — that restrictive licensing is the primary barrier — but it does so with systematic, per-dataset, per-language quantification that was previously unavailable. The MOSEL inventory itself is the evidence, and the innovation is in making the licensing constraint visible and measurable.

Innovation 4: The Deliberate Separation of Data Curation from Model Training as an Open-Source Strategy

The paper's most subtle but potentially impactful conceptual move is its decision to decouple data preparation from model building and to treat the data inventory as a standalone, citable, updatable artifact. This departs from the dominant approach in speech foundation model development, where data collection, cleaning, and model training are treated as internal, often undocumented steps in a pipeline whose primary output is the model itself. Whisper's training data is not public. OWSM's data is described but not released as a curated, verified inventory. SeamlessM4T's data documentation is limited.

By contrast, MOSEL is designed as a living resource — "this inventory of OS-compliant data, which will be continuously updated, is called MOSEL" (Section 1) — that exists independently of any particular model trained on it. The pseudo-labels are released under CC-BY not as an appendix to a model but as a standalone dataset. The GitHub repository is structured as a catalog, not as a training script with an embedded data list. This separation has several consequences:

• It enables cumulative improvement: As new OS-compliant datasets are released (or existing ones re-licensed), they can be added to the inventory without requiring a new model training run. Downstream users always have access to the most current list of permitted data.
• It enables legal composability: A model builder can train on any subset of the MOSEL inventory and be confident that the resulting model is OS-compliant, because every entry has been pre-vetted. This eliminates the need for each downstream team to perform its own license review — a task that, as Tables 8 and 9 show, is non-trivial and error-prone.
• It shifts the bottleneck from licensing expertise to model engineering: By doing the legal work once and releasing it as a public good, the paper allows model builders to focus on architecture, training, and evaluation rather than on interpreting CC license clauses.
• It creates a standard: If MOSEL becomes the reference inventory for OS-compliant speech data in the EU, it establishes a benchmark against which new datasets and models can be evaluated. A claim of "trained on OS-compliant data" can be verified by checking against MOSEL.

This innovation is incremental in form (data catalogs exist; the paper is essentially a very well-documented, licensing-focused catalog) but potentially fundamental in its strategic implications for how the open-source AI community should organize data curation work. It suggests that dedicated, community-maintained, legally verified data inventories are a necessary infrastructure layer for open-source AI — analogous to package registries in software engineering — and that the current practice of each model team independently assembling and vetting training data is both inefficient and error-prone.

The evidence for this innovation's value is partly the paper's own existence: by producing MOSEL, the authors have already identified a gap (Irish: 17 hours, no unlabeled data) that motivates targeted data collection efforts, and they have already produced a public good (the pseudo-labels) that would otherwise have been duplicated by every team attempting to build an EU-OSSFM, at a cost the paper quantifies at ~25,500 GPU hours and >$100,000 in cloud compute per duplication.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The primary dataset used for validation experiments is the Maltese subset of MOSEL, comprising 16 hours of labeled data (from the M-AILABS Maltese corpus; the paper does not explicitly name the labeled Maltese sources used beyond what is in the MOSEL inventory, but CommonVoice, FLEURS, and VoxPopuli contribute labeled Maltese hours in Table 2) and 9,100 hours of unlabeled data from VoxPopuli, with transcripts generated by Whisper large v3 as described in Section 3. For evaluation, the paper uses the CommonVoice Maltese test set and the FLEURS Maltese test set, both standard ASR benchmarks. The choice of Maltese is deliberate: it is "one of the lowest-resourced languages" and "the one for which Whisper achieves the worst results" (Section 4), making it the hardest possible test case for the pseudo-labeling strategy.

• Base model(s). The paper evaluates Whisper large v3 (Radford et al., 2023) as the pre-trained ASR system used for pseudo-label generation and as a baseline for comparison. Whisper large v3 is a sequence-to-sequence Transformer model released under the Apache 2.0 license, trained on 5 million hours of (largely proprietary) weakly supervised data covering 99 languages. For the proof-of-concept training experiment, the paper trains a custom 116M-parameter Conformer-Transformer sequence-to-sequence model (described in detail in Appendix B and Section 3.4 of the prior analysis) with a 12-layer Conformer encoder and 6-layer Transformer decoder, using an 8,000-token SentencePiece vocabulary and 80-dimensional Mel-filterbank input features. The Conformer encoder is initialized from a model trained on 4,000 hours of labeled English data to enable transfer learning, given the extremely limited Maltese labeled data.

• Metrics. The primary metric is Word Error Rate (WER), the standard evaluation measure for ASR, computed as:

  $\text{WER} = \frac{S + D + I}{N} \times 100\%$

  where S is the number of word substitutions, D is deletions, I is insertions, and N is the total number of words in the reference transcript. WER is reported as a percentage (lower is better). Computation uses the Whisper Normalizer for text normalization (handling punctuation, casing, and formatting differences) and JiWER for the metric calculation, ensuring consistency with standard ASR evaluation practices. The paper reports WER on two test sets (CommonVoice and FLEURS) for Maltese to assess robustness across evaluation domains.

• Baselines. The paper uses the following baselines for comparison in the Maltese experiment:

  • Whisper large v3 (zero-shot): The pre-trained Whisper model evaluated directly on the Maltese test sets without any fine-tuning or adaptation. This establishes the performance of the model used for pseudo-label generation and represents the state of a generic multilingual ASR system on a low-resource language. Whisper achieves 80.8 WER on CommonVoice Maltese and 73.8 WER on FLEURS Maltese.
  • Supervised-only training: Training the 116M-parameter custom ASR model exclusively on the 16 hours of labeled Maltese data available in MOSEL. The paper reports that this model "failed to converge due to its limited size (16 hours)" (Section 4), establishing a lower bound that demonstrates the necessity of pseudo-labeled data. No specific WER is reported because the model did not produce meaningful outputs.
  • Labeled + pseudo-labeled training: Training on the combination of 16 hours of labeled data plus the full 9,100 hours of pseudo-labeled data, without any filtering. This establishes the baseline improvement from adding raw pseudo-labels.
  • Labeled + filtered pseudo-labeled training: Training on labeled data plus pseudo-labeled data after applying two filtering strategies (LID-based language filtering and textual hallucination removal). This establishes the upper bound of what simple data cleaning can achieve.

  The paper does not compare against other open-source or Maltese-specific ASR models (e.g., a Maltese model from the MASRI project or a fine-tuned XLSR-53), which is a notable omission but consistent with the proof-of-concept framing — the goal is to validate that MOSEL data works, not to achieve state-of-the-art Maltese ASR.

• Generation budget / compute accounting. The paper's primary computational cost is the pseudo-labeling process, not the proof-of-concept training. As reported in Section 3, the transcription of 441k hours required approximately 25,500 GPU hours on NVIDIA A100 64GB GPUs, with an estimated cloud cost of over $100,000 USD and carbon emissions of 35,625 kg CO2 equivalent. For the Maltese proof-of-concept training, the paper uses 4 NVIDIA A100 GPUs (64GB RAM) for 150,000 updates with mini-batches of 40,000 tokens, which is a relatively modest training run (likely a few hundred GPU hours). The paper does not report the exact training time for the Maltese experiment, focusing instead on the pseudo-labeling cost as the primary shared infrastructure investment.

• Cross-validation / statistical protocol. The paper applies two filtering strategies (LID-based and hallucination-based) to the pseudo-labeled data and evaluates which combination yields the best WER. The filtering is applied as a deterministic pre-processing step, not as a hyperparameter sweep — the paper does not tune filtering thresholds or report cross-validation for filter selection. Results are reported as single-point WER estimates on the CommonVoice and FLEURS test sets without confidence intervals or statistical significance tests. The paper commits to releasing the LID outputs and hallucination flags for all 441k hours of pseudo-labeled data, enabling downstream users to apply their own filtering strategies and evaluate robustness (though the paper does not itself perform such robustness analyses). The 150k-update training run uses checkpoint averaging over the last 7 checkpoints (a common variance reduction technique), but the paper does not report variance across random seeds or data orderings.

Main Quantitative Results

Whisper Baseline Performance Across EU Languages

Before presenting the Maltese proof-of-concept, the paper establishes baseline ASR performance for Whisper large v3 across all 24 EU languages on two standard test sets (Table 7, Appendix D). This is not the main experimental contribution but provides essential context for interpreting the pseudo-label quality and the significance of the Maltese results.

Headline finding: Whisper performance varies enormously across EU languages, from near-human levels on high-resource languages (<5 WER) to essentially unusable on Maltese (>80 WER). The specific WER values are:

The paper notes that "Maltese stands out as the worst language by a wide margin" and that "only Estonian, Latvian, Lithuanian, and Slovenian exceed 15 WER" on either test set. The FLEURS scores are generally lower (better) than CommonVoice scores, likely because FLEURS consists of read speech in controlled conditions while CommonVoice contains more diverse, spontaneous, and noisy recordings. The absence of CommonVoice results for Croatian and the complete absence of results for Irish reflects gaps in test set availability rather than model performance on those languages.

This table is critical for two reasons: (1) it establishes that the pseudo-labels for high-resource languages (WER <10) are near-human quality and should work well for downstream training, and (2) it establishes Maltese as the most adverse test case, with Whisper's WER so high that the utility of its pseudo-labels for training is not obvious — making the positive Maltese results in Section 4 genuinely surprising.

Maltese ASR Training: Supervised-Only Failure

The paper reports a negative result that defines the experimental regime: training the 116M-parameter Conformer-Transformer model on only the 16 hours of labeled Maltese data "failed to converge due to its limited size." No WER is reported, and the paper does not provide learning curves or diagnostic metrics (training loss, validation loss, CTC loss) to characterize the failure mode. This is a qualitative rather than quantitative result, but it is essential for establishing the counterfactual: without pseudo-labeled data, no usable Maltese ASR model can be trained from the MOSEL labeled data alone.

This failure is not surprising from a data efficiency perspective. Contemporary end-to-end ASR models typically require hundreds to thousands of hours of labeled data for acceptable performance on a new language, depending on the language's phonetic complexity and the model architecture. Maltese, as a Semitic language with a complex phonological inventory (including pharyngeal and emphatic consonants not found in most European languages) and non-concatenative morphology, likely requires more data than an average European language, not less. The 16-hour threshold is therefore likely higher (i.e., more data would be needed) than for a typical Indo-European language with simpler phonology.

The paper does not explore whether a smaller model, a different architecture (e.g., purely CTC-based rather than sequence-to-sequence), or a different training strategy (e.g., self-supervised pre-training on the unlabeled VoxPopuli data followed by fine-tuning on the labeled data) could converge on 16 hours. This is a limitation — the supervised-only failure is treated as a binary condition rather than a point on a data scaling curve — but consistent with the proof-of-concept framing.

Maltese ASR Training: Labeled + Pseudo-Labeled Data (No Filtering)

Headline finding: Training on the combination of 16 hours of labeled data plus the full 9,100 hours of pseudo-labeled data yields WER of 39.4 on CommonVoice and 38.9 on FLEURS, roughly halving Whisper's error rate (80.8 → 39.4 on CommonVoice, a 51.2% relative reduction; 73.8 → 38.9 on FLEURS, a 47.3% relative reduction). These results are reported in Table 3:

This is the paper's central empirical result. The finding is significant for three reasons:

First, it demonstrates that pseudo-labels from a model with very high error rate (~80% WER) can still provide useful training signal. The student model substantially outperforms its teacher, which is counterintuitive given that most words in the pseudo-labels are incorrect. The paper does not analyze why this works (e.g., whether the model learns from partial correctness, from consistent errors, or from some other mechanism), but the empirical result itself is striking.

Second, it establishes that MOSEL's unlabeled data is genuinely useful even for the worst-case language. Maltese is the language where Whisper performs worst and where labeled data is scarcest. If pseudo-labeling works here, it should work for all other EU languages (Estonian, Lithuanian, Latvian, etc.), where Whisper's WER ranges from 16–30% (substantially better than 80%) and where more labeled data is available for supervision. This is a strong lower-bound validation of the MOSEL approach.

Third, it demonstrates a practical training recipe that any downstream user can replicate: combine the few available labeled hours with the MOSEL pseudo-labels, train a standard Conformer-Transformer architecture, and achieve usable ASR performance even when starting from essentially nothing. The recipe is not novel — pseudo-labeling for ASR is well-established — but its application to Maltese with 80% WER pseudo-labels is a more extreme test than prior work attempted.

Maltese ASR Training: Impact of Data Filtering

Headline finding: Applying simple filtering strategies to the pseudo-labeled data substantially improves performance, reducing WER from ~39 to ~24 — an additional ~40% relative improvement (39.4 → 23.8 on CommonVoice, 38.9 → 24.5 on FLEURS). These results appear in the third row of Table 3.

The paper applies two filtering strategies, with the rationale and implementation described in Section 3.4 of the prior analysis:

1. Language identification (LID) filtering: The paper runs language identification on all VoxPopuli Maltese segments using Whisper large v3 and removes any segment where the predicted language is not Maltese. Table 5 shows that approximately 23% of the "Maltese" data in VoxPopuli is actually in other languages:

   • Maltese: 77.1%
   • English: 9.9%
   • Italian: 3.5%
   • French: 2.2%
   • Arabic: 1.9%
   • Other: 5.4%

   Removing these 22.9% of segments eliminates a systematic source of noise — forcing Whisper to generate Maltese transcripts for non-Maltese speech produces entirely incorrect pseudo-labels that cannot provide useful training signal.

2. Textual hallucination filtering: The paper identifies and removes segments where Whisper's output contains patterns indicative of "nonsensical hallucinations" — repetitive short words, long noisy character strings, repetitive formulaic text, or suspicious single-word transcripts. The paper describes examples in Table 6 but does not quantify what fraction of segments are filtered by this criterion or what specific detection heuristics (e.g., character repetition thresholds, word frequency criteria) are used.

The paper does not report separate ablation results for each filtering strategy (LID-only vs. hallucination-only vs. both), which would have revealed the relative contribution of each. It also does not report how many total hours remain after filtering (likely ~7,000 hours if 23% are removed by LID, plus additional removals from hallucination filtering). The filtering is presented as a package rather than a systematic ablation.

The improvement from filtering is particularly notable because the filtered model achieves 23.8 WER — a level that might be usable for some practical applications (e.g., gisting, keyword spotting), especially given the starting point of 80.8 WER from the best available pre-trained model. This represents a 3.4× reduction in error rate from the Whisper baseline (80.8 → 23.8), which is a dramatic improvement for a language with only 16 hours of labeled data.

The Convergence Story: From Unusable to Marginally Usable

Taken together, the Maltese results trace a clear trajectory: Whisper (80.8 WER, unusable) → Labeled + pseudo-labeled (39.4 WER, weak but functional) → Labeled + filtered pseudo-labeled (23.8 WER, approaching usability). The paper does not claim state-of-the-art Maltese ASR or that 23.8 WER is sufficient for production deployment; it claims that the MOSEL data "can effectively be used for training ASR models" and that the results "support the conclusion that the collected and transcribed data represent a promising bedrock for developing an EU-OSSFM" (Section 4). The evidence for this claim is the monotonic improvement at each step of the pipeline.

Ablation Studies and Robustness Checks

The paper conducts minimal formal ablation studies, as is appropriate for a data curation paper whose primary contribution is the inventory and pseudo-labels, not model architecture optimization. The following elements constitute the nearest equivalents to ablation analyses:

Filtering impact (LID + hallucination removal vs. no filtering): The comparison between the "label. + pseudo-lab." and "label. + filtered pseudo-lab." rows in Table 3 constitutes the paper's primary intervention analysis. The improvement (~39 → ~24 WER, a ~40% relative reduction) demonstrates that data quality interventions matter substantially, but the paper does not decompose this into the contributions of LID filtering vs. hallucination filtering vs. their interaction. This is a genuine gap — it is possible that one of the two filters accounts for most of the improvement, and understanding which one would guide downstream users in prioritizing cleaning efforts for other languages (especially languages where Whisper WER is lower, making hallucination patterns potentially different).

Whisper language-specific performance variation (Table 7): While not an ablation in the traditional sense, the per-language Whisper WER results in Table 7 serve as a natural experiment documenting how pseudo-label quality varies across languages. The variation is extreme: from 2.8 WER (Spanish, FLEURS) to 80.8 WER (Maltese, CommonVoice). This can be interpreted as an implicit ablation over pseudo-label quality — the same Whisper model, the same inference procedure, but very different output quality depending on the language. The paper leverages this variation by choosing the worst case (Maltese) for its validation experiment, establishing that if pseudo-labels work in the worst case (80% WER), they should work in all better cases (2.8–30% WER). However, the paper does not validate this extrapolation by training models on intermediate-difficulty languages (e.g., Estonian at 29.9 WER, Lithuanian at 27.6 WER) to confirm that performance does indeed improve monotonically with pseudo-label quality.

Conformer encoder initialization from English: The Maltese training experiment initializes the Conformer encoder from a model trained on 4,000 hours of English data. This is a form of cross-lingual transfer, and the paper does not ablate it (i.e., training the same architecture from scratch on the same Maltese data). This is a significant omission: without this ablation, it is unclear how much of the model's ability to learn from noisy Maltese pseudo-labels comes from the English-initialized encoder's phonetic feature extraction capabilities versus the pseudo-labels themselves. If the English initialization provides most of the benefit, then the pseudo-label utility might be weaker than the paper implies for language pairs where English transfer is less effective (e.g., languages with very different phonological inventories that are not well-modeled by English-trained features). The paper's choice to use English initialization is reasonable (it reflects standard practice in low-resource ASR), but the absence of a from-scratch baseline means the pseudo-labels' contribution is confounded with the transfer learning benefit.

Supervised-only convergence failure: The paper's report that the supervised-only model "failed to converge" is treated as a binary outcome rather than a data scaling curve. This is a very coarse ablation — it would be more informative to see how WER varies as labeled data is increased from 0 to 16 hours (perhaps by subsampling the available labeled data) to understand whether the failure to converge is a threshold effect (e.g., 8 hours is insufficient but 12 hours works) or a smooth function of data quantity. This would help downstream users estimate how much labeled data they need for a given target WER in their own language. The binary "failed to converge" versus "39.4 WER" comparison leaves a large information gap.

No architecture or hyperparameter search: The paper does not explore alternative model architectures (e.g., purely CTC-based, wav2vec 2.0-style self-supervised pre-training followed by fine-tuning, or smaller/larger model variants), alternative pseudo-labeling models (e.g., other multilingual ASR systems that might perform better on Maltese), or alternative filtering strategies (e.g., confidence-based filtering, consensus-based filtering using multiple models, or more sophisticated hallucination detectors). This is entirely consistent with the paper's scope — it is a data paper, not a modeling paper — but it means the reported WER numbers should be interpreted as existence proofs (a model can be trained) rather than upper bounds (this is the best model that can be trained on MOSEL data). A more extensive hyperparameter search or architecture exploration might yield better WER for Maltese.

No test-time data augmentation or ensembling: The Maltese model is evaluated as a single checkpoint (with last-7 averaging) without test-time augmentation (e.g., SpecAugment at inference, multiple decoding passes) or ensembling. This is standard for ASR but means the reported WER is a conservative estimate that could potentially be improved with standard engineering techniques.

Critical Assessment

Does the Maltese Experiment Demonstrate That MOSEL Data Is Suitable for Training an EU-OSSFM?

The paper's central empirical claim is that the results "support the conclusion that the collected and transcribed data represent a promising bedrock for developing an EU-OSSFM" (Section 4). The Maltese experiment provides credible but limited evidence for this claim. Here is what was demonstrated and what was not:

What was demonstrated: Training a Conformer-Transformer model on 16 hours of labeled Maltese data plus filtered Whisper-generated pseudo-labels reduces WER from 80.8 (Whisper zero-shot) to 23.8 (Table 3). This is a substantial improvement and demonstrates that the MOSEL data is not merely legally clean but practically useful — the pseudo-labels provide genuine training signal even when the teacher model's WER is extremely high. The monotonic improvement at each step of the pipeline (Whisper → +pseudo-labeled → +filtered) is consistent and credible.

What was NOT demonstrated:

1. Single-language validation. The experiment covers only Maltese. The paper argues that Maltese is the worst case (highest Whisper WER) and that results should therefore generalize to all other EU languages, but this is an extrapolation, not a demonstrated fact. Different languages have different phonological properties, different amounts of labeled data, different Whisper WER levels, and different types of pseudo-label errors. Without at least one additional validation language (e.g., Estonian at 29.9 WER, or Lithuanian at 27.6 WER), the claim that the data works for all EU languages rests entirely on the logical argument that "if it works in the worst case, it should work in better cases." This is a reasonable inference but not an experimentally validated one.

2. No model scaling evidence. The paper trains one model of one size (116M parameters). The goal is a foundation model, which would likely be much larger (hundreds of millions to billions of parameters). Foundation models have different data requirements and training dynamics than small proof-of-concept models — they may need more data, may be more sensitive to label noise, or may benefit differently from pseudo-labeled vs. labeled data. Training a 116M-parameter model does not directly demonstrate that a 1B-parameter model would also train successfully on the same data, especially given that larger models have higher capacity and might overfit to noise in low-quality pseudo-labels.

3. No comparison to alternative data sources. The paper does not compare the MOSEL-trained model to models trained on any non-OS-compliant data (e.g., what WER could be achieved if MASRI's 8 hours of labeled Maltese were included, or if the model were trained on all available Maltese data regardless of license). This is understandable — the point is to validate OS-compliant training — but it means the reader cannot assess the "cost of compliance" in terms of ASR performance. If a model trained on all available Maltese data (including NC-restricted MASRI) achieves 10 WER, the 23.8 WER from OS-compliant-only data represents a substantial performance penalty that might affect the practical viability of the approach.

4. No self-supervised pre-training baseline. The paper's training recipe uses labeled data plus pseudo-labels in a supervised framework. An alternative approach — self-supervised pre-training (e.g., wav2vec 2.0 or HuBERT) on the 9,100 hours of unlabeled VoxPopuli data, followed by fine-tuning on the 16 hours of labeled data — might achieve better performance without requiring pseudo-labels at all. This approach would avoid the noise from Whisper's 80% WER transcripts entirely. The paper does not explore this, and it is not obvious that supervised training on noisy pseudo-labels would outperform self-supervised pre-training followed by clean supervised fine-tuning. The absence of this baseline is a significant gap for a paper whose stated goal is to enable "open-source speech foundation model training" — self-supervised pre-training is a standard foundation model training paradigm that is directly applicable to the unlabeled data in MOSEL.

5. No data scaling analysis. The paper uses all available pseudo-labeled data (9,100 hours) and all available labeled data (16 hours). Without ablating the quantity of pseudo-labeled data (e.g., training with 1k, 2k, 5k, 9k hours), it is impossible to know how much pseudo-labeled data is actually needed, whether the improvement is saturating, or whether a small amount of pseudo-labeled data provides most of the benefit. This matters for downstream users who may want to train models on subsets of the data (e.g., for faster experimentation).

6. No evaluation of the pseudo-labels themselves. The paper does not report the WER of the pseudo-labels against any reference (the VoxPopuli data is unlabeled, so no ground truth exists). It would be possible to evaluate on a small held-out labeled subset (e.g., the 16 hours of labeled data) to quantify pseudo-label accuracy directly, which would help explain why pseudo-labels with ~80% WER still help training (e.g., perhaps the pseudo-label WER is lower on easy segments and higher on hard ones, and the model learns from the easy ones).

Does the Experiment Support the Claim of "950,000 Hours of Speech Data for Open-Source Speech Foundation Model Training"?

The paper's title and abstract claim 950,000 hours of speech data for OSSFM training. The Maltese experiment validates that one specific subset of this data — the 9,100 hours of Maltese VoxPopuli data with Whisper-generated pseudo-labels — works for training. The other 940,900 hours (including 443,396 hours of YouTube-Commons, which the paper flags as having quality issues and does not pseudo-label) are not experimentally validated at all. The 950k hour figure is the total of the inventory, not the total of validated training data.

Specifically:

• 505.7k hours of labeled data: The paper does not run any controlled experiment showing that training on MOSEL labeled data alone (e.g., the 437k hours of English) produces a working ASR model. This is presumably obvious — labeled data is by definition suitable for supervised training — but the paper does not demonstrate it.
• 444.5k hours of unlabeled data (441k with pseudo-labels): The paper validates the Maltese subset (9,100 hours, 2% of the pseudo-labeled total). The other ~432k hours (for the remaining 22 languages) are released with pseudo-labels but are not validated through training experiments.
• 443k hours of YouTube-Commons: Explicitly not pseudo-labeled and flagged for quality issues. Its inclusion in the 950k total is aspirational — the data exists and is OS-compliant, but the paper provides evidence that it is not usable without "further checks and processing work."

The 950k hour claim is therefore a data inventory claim, not a usable training data claim. The Maltese experiment validates the approach (pseudo-labeling + filtering works) rather than the full inventory. This is a reasonable scoping decision for a data paper — the goal is to provide the data, not to train models on all of it — but the distinction between "data exists in the inventory" and "data is validated for training" should be clear to readers. The paper's claim that the Maltese results "support the conclusion that the collected and transcribed data represent a promising bedrock" is appropriately hedged ("promising"), but the experiment provides evidence only for the Maltese subset, and the extrapolation to the full inventory rests on unverified assumptions about pseudo-label quality scaling and language similarity.

Missing Experiments and Baselines That Would Strengthen the Claims

The following experiments, while not required for a data curation paper, would substantially strengthen the validation and are noted as absences:

1. At least one additional language for validation. Training on Estonian (29.9 WER Whisper) or Lithuanian (27.6 WER) would provide evidence about the intermediate regime where pseudo-labels are noisy but not catastrophically so. Training on a high-resource language with very low Whisper WER (e.g., Dutch at 4.3 WER) would establish an upper bound on pseudo-label utility.

2. Comparison of supervised training on pseudo-labels vs. self-supervised pre-training on raw audio. This is the most important missing baseline, as it directly addresses the question: is it better to transcribe unlabeled data with a weak model and train supervised, or to use the audio directly for self-supervised learning? The answer likely depends on language, Whisper WER, and available labeled data, and understanding this tradeoff would directly inform how downstream users should use the MOSEL data.

3. Pseudo-label quality measurement. Even on a small held-out labeled set (the 16 hours of Maltese could serve this purpose), measuring the pseudo-labels' WER would provide direct insight into why an 80% WER teacher can train a 39% WER student.

4. Data scaling curves. Training with varying amounts of pseudo-labeled data (e.g., 10%, 25%, 50%, 100% of the available Maltese data) would reveal whether performance is saturating or could benefit from even more data — directly relevant to languages with more unlabeled hours in MOSEL.

5. Filtering ablation. Separate evaluation of LID-only filtering and hallucination-only filtering would reveal which cleaning strategy is most impactful, guiding downstream users.

6. Confidence intervals or multi-seed results. Training with multiple random seeds and reporting mean ± standard deviation would provide information about training stability, which is particularly important for low-resource settings where convergence can be erratic.

These absences are consistent with the paper's framing as a data curation and release effort — the proof-of-concept is explicitly a "proof of concept" rather than a comprehensive empirical study — but they limit the strength of the experimental claims that can be made beyond the Maltese case.

6. Limitations and Trade-offs

Limitation 1: The YouTube-Commons Dataset Is Included in the Headline 950k Hour Figure but Not Validated or Pseudo-Labeled

The assumption or constraint. The paper's title and core numerical claim — "950,000 hours of speech data" — include 443,396 hours from the YouTube-Commons dataset (Table 1), which alone constitutes 46.6% of the total inventory. However, the paper explicitly acknowledges (Section 2) that manual inspection revealed three serious quality issues: (i) "it includes videos without speech (e.g., with only music)," (ii) "the language identification (LID) tag and the transcripts are often inaccurate," and (iii) "sentence-level segmentation of the speech is not provided (it contains unsegmented transcripts for the entire YouTube videos)." The paper further notes that "further checks and processing work would be needed to effectively exploit the dataset for OSSFM training." YouTube-Commons is excluded from the pseudo-labeling effort (Section 3: "YouTube-Commons was excluded due to the issues described in §2") and is not used in the Maltese proof-of-concept, meaning no experiment in the paper validates that any model can be successfully trained on its content.

The consequence. The 950k hour figure is a data inventory claim, not a usable training data claim, and the distinction matters enormously for downstream users evaluating whether to invest in building an OSSFM from MOSEL data. The effectively usable data is closer to 507k hours (the 505.7k labeled hours plus the 441k pseudo-labeled hours minus YouTube-Commons, accounting for the near-complete overlap between VoxPopuli and LibriLight pseudo-labeled hours and their unlabeled counts). This is still a substantial resource, but it is roughly half the headline number. Moreover, the quality issues described — non-speech content, mislabeled languages, unsegmented transcripts — are not minor. Training on data where the language tag is frequently wrong means that a model expecting, say, Bulgarian speech would encounter audio in other languages, producing garbled training signal (as demonstrated for VoxPopuli Maltese in Table 5, where 23% of "Maltese" segments were actually in other languages). Training on unsegmented hours-long YouTube videos with transcripts covering the entire video is fundamentally incompatible with standard ASR training pipelines, which expect sentence- or utterance-level audio-text pairs. Making YouTube-Commons usable would require substantial preprocessing: LID-based language filtering, speech activity detection (to remove music-only segments), and segmentation into utterance-length chunks — tasks that the paper does not perform and that would require additional engineering effort and computational cost not accounted for in the paper's cost analysis.

What evidence exists in the paper. The paper describes the YouTube-Commons issues explicitly in Section 2 and excludes it from pseudo-labeling in Section 3. Table 1 reports the 443,396 hours as part of the labeled data inventory. Table 2 includes YouTube-Commons hours in the per-language labeled totals (e.g., English labeled hours jump from ~437k to the total including YouTube-Commons; the paper does not provide a clean breakdown of YouTube-Commons hours by language separately, making it difficult for readers to assess language-specific data quality without consulting the YouTube-Commons dataset directly). The Maltese experiment (Section 4) does not use YouTube-Commons data at all.

Mitigation status. The paper does not mitigate this limitation — it explicitly flags the issues and defers cleanup to downstream users. Section 7 (Limitations) does mention that "the quality of the supervision of the surveyed dataset cannot always be taken for granted, advocating for dedicated inspections before using it to train an OSSFM. This is particularly true for the metadata and transcripts of YouTube videos under OS-compliant licenses as those collected in YouTube-Commons." This is an acknowledgment but not a solution. A future version of MOSEL could include cleaned, filtered, and segmented versions of YouTube-Commons data, or the paper could have reported how many hours remain after basic cleaning (LID filtering, speech activity detection) to give a realistic lower bound, but neither is provided.

---

Limitation 2: Only One Language (Maltese) Is Experimentally Validated for Training Viability

The assumption or constraint. The paper's central empirical claim — that the collected data "can effectively be used for training ASR models" and "represents a promising bedrock for developing an EU-OSSFM" (Section 4) — rests entirely on a single proof-of-concept experiment in one language (Maltese). The paper argues (Section 4) that Maltese is the worst case because it is "one of the lowest-resourced languages" and "the one for which Whisper achieves the worst results," and that if the approach works for Maltese, it should work for all other EU languages. This is a logical extrapolation, not an experimentally verified fact. The other 22 EU languages (Irish is excluded because it has no unlabeled data and only 17 labeled hours) are not validated through any training experiment. The paper does not train even a single additional language model — not a high-resource language where Whisper achieves <5 WER and pseudo-labels are near-perfect (e.g., Spanish: 4.7 WER), not an intermediate-difficulty language where Whisper achieves 15-30 WER (e.g., Estonian: 29.9 WER, Lithuanian: 27.6 WER), and not a structurally different language family (e.g., Finnish, a Uralic language with complex morphology, where Whisper achieves 24.6 WER).

The consequence. The extrapolation from Maltese to all EU languages rests on several unverified assumptions. First, it assumes that the relationship between pseudo-label WER and downstream training utility is roughly monotonic — that if 80% WER pseudo-labels help, then 30% WER pseudo-labels help more, and 5% WER pseudo-labels help even more. This is plausible but not guaranteed; it is possible that pseudo-labels of intermediate quality introduce different types of errors (e.g., confident but systematically wrong transcriptions that the student model learns to replicate) that are more harmful than the random noise in 80% WER pseudo-labels. Second, it assumes that all languages benefit similarly from English-initialized Conformer encoders. Maltese, as a Semitic language with significant Romance vocabulary, may benefit from English initialization in ways that a Uralic language (Finnish, Estonian, Hungarian) or a Baltic language (Latvian, Lithuanian) does not, because phonetic feature extractors trained on English may transfer better to languages with similar phonological inventories. Third, it assumes that the availability of labeled data — which varies from 19 hours (Maltese) to 437,239 hours (English) — does not qualitatively change the training dynamics. In high-resource regimes, pseudo-labeled data may contribute little marginal improvement over abundant labeled data, making the pseudo-labeling investment less cost-effective. In intermediate regimes (e.g., 100-1,000 labeled hours), the optimal ratio of labeled to pseudo-labeled data may differ from the Maltese case.

What evidence exists in the paper. Table 3 reports the Maltese results (Whisper 80.8 → 39.4 → 23.8 WER). Table 7 reports Whisper WER across all EU languages, establishing the variation in pseudo-label quality but without training experiments to connect this to downstream model performance. The paper acknowledges (Section 7, Limitations) that "an obvious next step for this work is its extension to many other spoken languages" and that the current scope is EU languages only, but does not acknowledge the single-language validation gap within the EU scope. The paper's title and abstract claim applicability to "EU languages" (plural), which is consistent with the inventory scope but not with the experimental validation scope, creating a potential mismatch between claimed and demonstrated coverage.

Mitigation status. The paper does not mitigate this limitation. A multi-language validation experiment — even training models for 3-4 languages spanning the Whisper WER range (e.g., Spanish at ~5 WER, Estonian at ~30 WER, Lithuanian at ~28 WER) — would require substantial additional computation (perhaps 1,000-2,000 additional GPU hours across languages) but would transform the evidence from a single-point existence proof to a credible demonstration of generalizability. The paper could also have analyzed the relationship between Whisper WER (Table 7) and expected downstream model WER using scaling laws from prior work (e.g., Zhang et al., 2022) to provide theoretical grounding for the extrapolation, but this analysis is not attempted.

---

Limitation 3: The Cost of Producing OS-Compliant Pseudo-Labels Is Not Amortized Over Users and May Not Be the Most Efficient Path to Working Models

The assumption or constraint. The paper's pseudo-labeling strategy — transcribing 441k hours with Whisper large v3 — consumed ~25,500 GPU hours, costing >$100,000 on cloud infrastructure and producing 35,625 kg CO2 equivalent (Section 3). The paper frames this as a Green AI contribution: by doing the transcription once and releasing the results, "avoiding duplicated efforts across different institutions can significantly reduce the overall environmental impact and costs" (Section 3). This framing assumes that (a) the primary pathway to building an OSSFM is supervised training on pseudo-labeled data, and (b) the pseudo-labels produced are the ones that downstream users would want — that Whisper large v3 is the optimal teacher model, that the specific inference configuration (beam size 5, <|notimestamp|>, forced language tags) produces the most useful transcripts, and that supervised training on these transcripts is more effective than alternative approaches that use the same raw audio data in different ways.

The consequence. The paper's investment in Whisper-based pseudo-labeling may not represent the most efficient allocation of shared infrastructure resources, and the released pseudo-labels may lock downstream users into a specific training paradigm that is suboptimal for some languages or applications. The most significant alternative — self-supervised pre-training on the raw unlabeled audio (using wav2vec 2.0, HuBERT, or similar methods) followed by fine-tuning on the available labeled data — would use exactly the same unlabeled audio but without requiring any transcription. This approach has been demonstrated to work well for low-resource languages (Baevski et al., 2020) and is a standard paradigm for speech foundation model development. The paper does not compare its pseudo-labeling + supervised training approach to self-supervised pre-training on the same data, so the reader cannot assess whether the 25,500 GPU hours spent on transcription produced better downstream models than spending those same GPU hours on self-supervised pre-training would have. For Maltese specifically — where Whisper's WER is 80% and pseudo-labels are extremely noisy — it is entirely plausible that self-supervised pre-training on the 9,100 hours of raw VoxPopuli audio, followed by fine-tuning on the 16 hours of labeled data, would outperform the 23.8 WER achieved with filtered pseudo-labels. The paper provides no evidence either way.

Additionally, the paper's choice of Whisper large v3 — while Apache 2.0 licensed and therefore OS-compliant for the generation tool — is itself trained on 5 million hours of proprietary, non-OS-compliant data (Radford et al., 2023). The paper's legal rationale (Section 3) is that "the Apache 2.0 License allows the generated content to be released under any license," which is correct. However, this creates a structural dependency: building an OSSFM requires first using a model that was trained on non-open data, which some open-source purists might consider philosophically inconsistent even if legally permitted. The paper does not engage with this tension.

What evidence exists in the paper. Section 3 reports the transcription costs. Table 3 reports the Maltese downstream performance. The paper does not report any results from self-supervised pre-training on the same data, nor does it discuss this as an alternative paradigm. The Maltese experiment's Conformer encoder uses English-initialized weights rather than self-supervised pre-training on Maltese audio — a transfer learning approach that is conceptually distinct from both supervised pseudo-labeling and self-supervised pre-training, and whose contribution to the final WER is not isolated (no from-scratch or self-supervised baseline is provided).

Mitigation status. The paper does not mitigate this limitation. The pseudo-labels are released as a take-it-or-leave-it resource. Downstream users can of course ignore the pseudo-labels and perform self-supervised pre-training on the same raw audio instead (VoxPopuli and LibriLight are publicly available), but the paper's core contribution — the 25,500 GPU hours of transcription — would then be wasted for those users. A more robust approach would have been to provide both resources (pseudo-labels AND self-supervised pre-trained checkpoints) and to compare their downstream performance, but this would have required a different scope and substantially more computation. The paper's Green AI framing is partially undermined by the absence of evidence that the chosen computation (Whisper inference) was the most impactful use of those 25,500 GPU hours compared to alternatives.

---

Limitation 4: The Maltese Proof-of-Concept Model Architecture and Scale Do Not Match the Foundation Model Ambition

The assumption or constraint. The paper is titled and framed around "Speech Foundation Model Training" — the explicit goal is to enable training of an EU-OSSFM. The Maltese proof-of-concept (Section 4, Appendix B) trains a 116M-parameter Conformer-Transformer model with a 12-layer Conformer encoder and 6-layer Transformer decoder, using an 8,000-token SentencePiece vocabulary and 80-dimensional Mel-filterbank inputs. This is a standard ASR architecture at a moderate scale — by comparison, Whisper large v3 has approximately 1.55 billion parameters, and contemporary speech foundation models (e.g., USM, SeamlessM4T) range from hundreds of millions to billions of parameters. The paper does not train any model at foundation-model scale, nor does it provide evidence that the MOSEL data (or the pseudo-labels specifically) supports training of larger models.

The consequence. The gap between the proof-of-concept model scale (116M parameters) and the ambition (EU-OSSFM, likely 500M–2B+ parameters) introduces uncertainty about whether the MOSEL data is sufficient in both quantity and quality for foundation-model-scale training. Larger models have higher capacity and may exhibit different training dynamics when trained on noisy pseudo-labels. Specifically: (a) Overfitting risk: a 1B-parameter model trained on, say, 9,100 hours of noisy Maltese pseudo-labels plus 16 hours of clean labeled data may memorize the pseudo-label noise rather than learning generalizable acoustic-linguistic mappings, because its capacity far exceeds the information content of the (mostly incorrect) labels. Regularization techniques, data augmentation, and careful early stopping might mitigate this, but the paper provides no evidence that these techniques work at scale on this specific data. (b) Data quantity adequacy: the MOSEL inventory provides 9,100 hours of pseudo-labeled Maltese — a substantial amount for a 116M-parameter model but potentially insufficient to saturate a 1B+ parameter model. Foundation models are typically trained on tens to hundreds of thousands of hours per language; Whisper v3 used 5 million hours across 99 languages. Whether the MOSEL data volumes (e.g., 8,100 hours for Croatian, 13,600 for Danish, 9,100 for Maltese) are sufficient for foundation-model-scale training per language is unknown. (c) Multilingual vs. monolingual training: the proof-of-concept trains a monolingual Maltese model, but a foundation model covering 24 EU languages would likely be multilingual (trained jointly on all languages). Multilingual training introduces additional complexities — language interference, capacity allocation across languages, data imbalance handling — that are not addressed by the monolingual Maltese experiment.

What evidence exists in the paper. The Maltese experiment (Table 3) uses a 116M-parameter model. The paper does not report any experiments with larger models, with multilingual training, or with alternative architectures. Appendix B provides full architectural and training details for the 116M model. The paper does not discuss the gap between proof-of-concept scale and foundation model scale, nor does it acknowledge that the validation experiment's architecture may not be representative of the target use case.

Mitigation status. The paper does not mitigate this limitation. Section 7 (Limitations) does not mention the scale gap. The paper's framing as "the first step toward filling this gap" (Section 1) and "a promising bedrock for developing an EU-OSSFM" (Section 4) is appropriately hedged for a data curation paper, but the gap between a 116M-parameter monolingual proof-of-concept and a multilingual foundation model is large enough that the experimental validation provides only weak evidence about the suitability of the data for the stated goal. A more informative validation — even if still small-scale — would have trained a multilingual model covering, say, 5 languages (representing high, medium, and low resource levels) to demonstrate that MOSEL data supports the multilingual training paradigm that a foundation model would use.

---

Limitation 5: The License Compliance Framework Depends on Legal Interpretations That May Vary Across Jurisdictions

The assumption or constraint. The paper's entire inventory is gated by a specific interpretation of open-source licensing: which Creative Commons variants are acceptable (CC-0, CC-BY), which are rejected (CC-BY-SA, CC-BY-NC, CC-BY-ND), and how the relationship between dataset wrapper licenses and underlying data licenses is resolved (the conservative principle that the most restrictive license governs). The paper relies on the Open Source Initiative's Open Source AI Definition (footnotes 1-3 in Section 1) and on Creative Commons FAQ guidance (footnotes 5-7 in Section 2). However, several of the key legal judgments the paper makes are in areas where the law is unsettled or jurisdiction-dependent: (a) whether a trained AI model constitutes a "derivative work" of its training data under copyright law — this varies by jurisdiction (U.S. fair use doctrine vs. EU database rights vs. other national frameworks) and is the subject of active litigation; (b) whether the CC-BY-SA "share-alike" provision legally applies to AI model weights (the paper argues yes based on the CC license text requiring derivatives to be distributed under the same license terms, but this interpretation has not been tested in court); and (c) whether the "collection" exception in CC licenses (allowing CC-BY material to be aggregated with restrictively licensed material in a collection without the collection's license superseding the individual item licenses) applies to datasets containing mixed-license audio and transcripts.

The consequence. A downstream user who relies on the paper's license determinations to train and release an OSSFM could face legal risk if a court or regulatory body interprets the relevant licenses differently. The most acute risk involves the CC-BY-SA rejection. If a court were to rule that AI model weights are not derivative works of SA-licensed training data (or that SA licensing does not apply to models trained on SA-licensed data), then the paper's exclusion of large SA-licensed datasets — People's Speech (30,000 hours, English), ParlaSpeech-HR (1,816 hours, Croatian), ARTHUR 1.0 (884 hours, Slovenian), Vystadial (63 hours, English and Czech) — would have been unnecessarily conservative, and the resulting OSSFM trained only on CC-0/CC-BY data would be substantially weaker than one that could have included these resources. Conversely, if a court were to rule that the paper's accepted licenses (CC-BY) do impose implicit restrictions on model distribution that the paper did not anticipate, an OSSFM trained on MOSEL data could face legal challenges despite the paper's vetting. The paper provides no legal analysis, cites no case law, and does not discuss jurisdictional variation.

Additionally, the paper's treatment of the Whisper large v3 model as an acceptable tool for generating OS-compliant outputs — because the model itself is Apache 2.0 licensed — rests on the assumption that the license of a generative model covers the license of its outputs. This is generally accepted for Apache 2.0 (the license explicitly does not claim ownership over outputs), but it means that the OS compliance of the pseudo-labeled data depends on the OS compliance of a model (Whisper) whose own training data is proprietary and non-OS-compliant. Some interpretations of the Open Source AI Definition's "training data must be OS-compliant" requirement might argue that using a non-OS-compliant model to generate training data introduces a dependency that undermines the spirit of the definition, even if the letter of the law is satisfied. The paper does not engage with this nuance.

What evidence exists in the paper. Section 2 and its footnotes 5-9 provide the legal reasoning for license acceptance and rejection. Appendix C (Tables 8 and 9) lists the rejected datasets with their specific licenses. The paper cites Liesenfeld and Dingemanse (2024) and Eiras et al. (2024) on open-washing and the misuse of the term "open source," and White et al. (2024) on the Model Openness Framework, establishing that the paper's framework is grounded in the current discourse but not that it is legally tested.

Mitigation status. The paper partially mitigates this limitation through transparency and conservatism. The license inventory is published in detail (Tables 1, 8, 9), the reasoning is explicit (Section 2 and footnotes), and the paper errs on the side of excluding data when the license situation is ambiguous (e.g., treating GigaSpeech as non-OS-compliant because its YouTube-derived audio carries restrictive terms despite the dataset's Apache 2.0 wrapper license). The GitHub repository format ("will be continuously updated") allows for corrections if legal interpretations evolve. However, the paper does not include a legal disclaimer, does not acknowledge the jurisdictional variation in copyright and derivative work doctrine, and does not discuss the potential for future legal developments to change which datasets are considered OS-compliant. Section 7 (Limitations) does not mention legal uncertainty as a limitation. A more robust approach would have included a brief discussion of the legal assumptions and their uncertainty, perhaps with a recommendation that downstream model builders consult legal counsel before commercial deployment — which is standard practice in open-source AI release documentation.

---

Limitation 6: The Irish Language Is Left Without Any Viable Training Path

The assumption or constraint. The paper's survey identifies only 17 hours of OS-compliant speech data for Irish (all labeled, from CommonVoice per Table 1's language coverage), and zero hours of unlabeled OS-compliant data (Table 2). This is by far the worst coverage of any EU language — the next-lowest is Maltese with 9,119 total hours (19 labeled + 9,100 unlabeled). The paper acknowledges this: "Irish, for which, unfortunately, we did not find unlabeled OS-compliant data" (Section 2, end of language coverage discussion). Section 7 (Limitations) explicitly calls out "collecting and releasing new material — possibly with human-generated transcripts — under permissive licenses for the least-resourced language" as an "important future direction," noting that "this is especially critical for Irish, for which we were able to collect only 17 hours of (labeled) speech."

The consequence. The MOSEL inventory, as it stands, provides no path to training a working Irish ASR model, let alone contributing to an Irish component of an EU-OSSFM. The Maltese experiment demonstrated that 16 hours of labeled data alone is insufficient for model convergence (Section 4: "it failed to converge due to its limited size"), and Irish has a comparable amount (17 hours) with no unlabeled data to augment it. The paper's two strategies for addressing data scarcity — pseudo-labeling of unlabeled data (Section 3) and leveraging labeled data from other sources (Section 2) — both fail for Irish. There is no unlabeled Irish data to pseudo-label, and the non-OS-compliant Irish resources that exist (CMU Wilderness, which includes some Irish but carries NC and ND terms from Faith Comes By Hearing; UK and Ireland English Dialect, which includes Irish but is CC-BY-SA) are rejected by the license filter. This means that the paper's stated goal of covering "all 24 official EU languages" (Section 1) is not achieved for Irish — the inventory contains data, but the data is insufficient for training, and the paper provides no solution.

This is not merely a practical gap; it is a policy-relevant failure. The EU's language equality principle mandates equal status for all official languages, including Irish. An EU-OSSFM that covers 23 languages but omits Irish would fail to meet the policy objective that the paper itself cites as motivation (the "regulatory efforts addressing their risks and impacts" mentioned in Section 1). Moreover, Irish is an official EU language with constitutional status in Ireland and specific EU institutional support for its preservation and technological development. Its omission from a viable OSSFM training set is therefore a more significant gap than, say, a slightly smaller amount of data for Danish or Swedish (which at least have enough to train a working model).

What evidence exists in the paper. Table 2 reports Irish as having 17 labeled hours and 0 unlabeled hours. Table 1 lists CommonVoice (which includes Irish) and other datasets, but none provide substantial Irish data. The paper acknowledges the gap explicitly in Section 7 (Limitations). Table 7 in Appendix D shows that Whisper does not support Irish at all (no WER results reported for Irish on CommonVoice or FLEURS), meaning that even if unlabeled Irish data were found, the current pseudo-labeling pipeline using Whisper large v3 would not work — a different transcription model would be needed.

Mitigation status. The paper acknowledges but does not mitigate this limitation. Section 7 (Limitations) correctly identifies the Irish gap and proposes future data collection as the solution: "collecting and releasing new material — possibly with human-generated transcripts — under permissive licenses for the least-resourced language." This is a reasonable recommendation but is entirely external to the paper's contributions — MOSEL provides no Irish data beyond the existing 17 hours, and the paper does not produce any new Irish recordings, transcripts, or pseudo-labels. The paper's Green AI argument (Section 3: avoiding duplicated effort) does not apply to Irish because there is no effort to duplicate — there is no existing OS-compliant Irish data pipeline. The limitation is structural: MOSEL's methodology (survey existing data, verify licenses, pseudo-label unlabeled data) cannot address a language where no OS-compliant data of any kind (labeled or unlabeled) exists in sufficient quantity. A different approach — original data collection under OS-compliant terms — would be required, and the paper does not undertake this.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper redefines the problem of building open-source speech AI from a model engineering challenge into a data licensing and provenance verification challenge. This is not a paradigm shift in speech technology itself — the models, architectures, and training recipes discussed (Whisper, Conformer-Transformer, pseudo-labeling) are all established — but it is a fundamental reframing of what counts as success in open-source speech foundation model development. Prior to MOSEL, the speech community's implicit success criteria were: release model weights under a permissive license, document the training procedure, and call it "open." The paper demonstrates that this standard is both legally and practically insufficient, because the training data carries restrictions that propagate to downstream models, making them non-compliant with the Open Source AI Definition regardless of the weight license.

The evidence for this reframing is the paper's systematic catalog of failure modes across existing SFMs. Whisper fails on training data and code transparency (Section 1). OWSM fails on data licensing — MuST-C's CC-BY-NC-ND and SPGISpeech's NC terms contaminate the entire training pipeline. SeamlessM4T fails on model licensing. These are not edge cases or minor oversights; they are structural consequences of the speech community's historical reliance on datasets collected under terms designed for academic research (NC, ND, SA restrictions) rather than for open-source AI development. The paper's diagnostic contribution is to make this invisible constraint visible and quantifiable: Tables 8 and 9 collectively list 31 rejected datasets representing many times the total hours of the 18 accepted ones, and for several low-resource languages (Irish, Maltese, Slovenian, Croatian), the best available data is locked behind restrictions that make it legally unusable for an OSSFM.

This reframing has several concrete consequences for how the field operates:

It shifts research investment priorities. Before MOSEL, the dominant approach to improving speech foundation models was to build better architectures, scale up training, or collect more data. The paper's analysis reveals that for EU languages, the binding constraint is not data volume — there are hundreds of thousands of hours of speech available — but data licensing reform. The highest-impact intervention for building an OSSFM is therefore not a new model architecture but convincing dataset creators, funding agencies, and research institutions to re-release existing speech data under OS-compliant terms (CC-0 or CC-BY). This is a legal and policy challenge, not a technical one, and the paper provides the evidence base — the per-dataset, per-language breakdown of what is lost to restrictive licensing — that makes the case to funding bodies and dataset curators.

It establishes a methodology for licensing verification that is transportable to other domains. The paper's systematic application of the Open Source AI Definition's four freedoms to speech datasets — checking each license against use, study, modify, and share rights, treating the underlying data license as binding regardless of wrapper licenses, rejecting SA terms for imposing downstream license propagation — is not specific to speech. The same methodology applies directly to text corpora, vision datasets, code repositories, and any other domain where foundation models are built on externally sourced training data. The paper's distinction between dataset wrapper licenses and underlying data licenses (Section 2: "in cases where the transcripts are OS compliant but the corresponding speech (or part of it) is not, we document the dataset under the most restrictive license") is particularly important because it addresses a systematic error that many prior efforts made — GigaSpeech being the canonical example, distributed under Apache 2.0 but built on YouTube-licensed content.

It reveals a previously unrecognized tension in the Green AI argument for shared infrastructure. The paper frames its 25,500 GPU-hour pseudo-labeling investment as an environmental contribution — avoiding duplicated effort across downstream teams (Section 3). However, the paper also demonstrates that the quality of those pseudo-labels varies enormously by language: from near-perfect (<5 WER for Spanish, Dutch, Italian) to essentially unusable (80.8 WER for Maltese). This means downstream users face a dilemma: accept the paper's Whisper-generated pseudo-labels (which may be suboptimal for their specific language and application) and train supervised, or invest their own computation in alternative approaches (self-supervised pre-training, better pseudo-labeling with a different model, targeted data collection) that may produce better results but would "waste" the paper's shared computation. The paper does not resolve this tension — it provides the pseudo-labels as a take-it-or-leave-it resource — but its existence highlights that shared infrastructure investments in AI need to be complemented by evidence that the chosen approach is genuinely optimal for the downstream use cases, not just efficient to compute once.

It reconciles contradictory prior claims about open-source speech AI. OWSM claimed to be "open" because it released weights and code; the paper shows that this claim fails on the training data criterion. Whisper claimed to be "open" because its model and inference code carry an Apache 2.0 license; the paper shows that the training data and training code are not public, violating both the transparency and reproducibility requirements of the Open Source AI Definition. These are not disagreements about facts — they are disagreements about definitions — but by adopting and rigorously applying an externally defined standard (the OSI definition), the paper provides a common reference frame within which these claims can be evaluated. The result is a clear verdict: to the best of the authors' knowledge, no existing SFM meets the open-source standard, and the primary gap is training data licensing, not model or code availability.

It makes the case that dedicated, community-maintained, legally verified data inventories are a necessary infrastructure layer for open-source AI. The paper's deliberate separation of data curation from model training — MOSEL is a standalone, updatable catalog, not an appendix to a specific model release — represents a strategic choice that, if adopted widely, would change how the open-source AI community organizes its work. Currently, each model team independently assembles and vets training data, duplicating effort and risking inconsistent (or incorrect) license determinations. A single, maintained, community-accepted inventory eliminates this duplication and creates a standard against which claims of open-source compliance can be verified. This is analogous to package registries in software engineering (npm, PyPI), which succeeded because they solved a coordination problem — vetting code provenance and licensing once rather than per-project. Whether MOSEL achieves this status depends on community adoption and ongoing maintenance (the paper commits to continuous updates), but the architectural insight — that licensing verification is a shared infrastructure problem, not a per-project task — is broadly applicable.

Follow-Up Research This Work Enables

Multi-language validation of pseudo-label training utility across the full Whisper WER range. The paper's central empirical claim — that MOSEL pseudo-labels enable ASR training across EU languages — rests on a single Maltese experiment (80.8 WER pseudo-labels, 23.8 WER downstream). A compelling follow-up would train the same 116M-parameter Conformer-Transformer architecture on 4-5 additional languages spanning the Whisper WER spectrum from Table 7: Spanish (~5 WER, high quality), Estonian (~30 WER, intermediate), Lithuanian (~28 WER, intermediate), Bulgarian (~14 WER, moderate), and perhaps Slovenian (~17 WER, moderate). For each language, the experiment would train three variants: labeled-only (whatever labeled data is available for that language, to establish the supervised baseline), labeled + unfiltered pseudo-labeled, and labeled + filtered pseudo-labeled (using the same LID and hallucination filters the paper developed for Maltese). The key output would be a scatterplot of Whisper WER vs. downstream model WER improvement from pseudo-labels, which would reveal whether the relationship is monotonic (does lower Whisper WER predictably produce larger gains?), whether there are diminishing returns (do 5 WER pseudo-labels help less than expected because the labeled data is already sufficient?), and whether the Maltese result (80 → 24 WER) is anomalous or representative. This experiment would require perhaps 2,000-3,000 GPU hours across languages but would transform the paper's claim from a single-point existence proof to a credible demonstration of generalizability.

Direct comparison of pseudo-labeling + supervised training vs. self-supervised pre-training on the same raw audio. The paper's Maltese experiment uses 9,100 hours of VoxPopuli audio to generate Whisper pseudo-labels, then trains a Conformer-Transformer on those labels plus 16 labeled hours. An alternative paradigm — self-supervised pre-training (e.g., wav2vec 2.0 BASE, ~95M parameters, comparable to the paper's 116M-parameter model) on the exact same 9,100 hours of raw audio, followed by fine-tuning on the same 16 labeled hours — would use the same data but avoid pseudo-label noise entirely. This experiment is directly enabled by MOSEL because the unlabeled VoxPopuli data is identified, organized, and OS-compliant; a researcher can download it and run standard self-supervised pre-training toolkits without additional data wrangling. The comparison would quantify the "pseudo-label noise penalty" — how much WER is lost by training on 80%-error transcripts compared to learning directly from the audio signal — and would establish whether the paper's 25,500 GPU-hour pseudo-labeling investment was the optimal use of shared infrastructure. For Maltese specifically, where Whisper achieves 80.8 WER, self-supervised pre-training is likely to outperform pseudo-labeled training; for Spanish (4.7 WER), the pseudo-labels are near-human quality and supervised training on them might match or exceed self-supervised pre-training plus fine-tuning. A 3-language comparison (Maltese, Estonian, Spanish) would map the tradeoff across the pseudo-label quality spectrum and provide downstream users with a decision rule for when to use the paper's pseudo-labels vs. when to self-supervise from scratch.

Targeted re-licensing campaigns for the highest-impact non-OS-compliant datasets. The paper's Tables 8 and 9 identify 31 datasets that are excluded from MOSEL due to licensing restrictions, several of which are transformative for specific low-resource languages. A follow-up research effort — not technical but institutional — would identify the "most impactful re-licensing targets" and work with dataset creators to release existing data under OS-compliant terms. The paper provides the impact quantification needed to prioritize: for Irish, the UK and Ireland English Dialect dataset (CC-BY-SA, 31 hours, Table 8) and the Irish portions of CMU Wilderness (NC, ND, Table 9) are the only substantial resources, and re-licensing either to CC-BY would multiply Irish training data by 2-3× from the current 17 hours. For Croatian, ParlaSpeech-HR (CC-BY-SA, 1,816 hours, Table 8) would increase available Croatian data from 8,161 hours to ~10,000 hours and, critically, provide high-quality parliamentary speech with human-generated transcripts rather than relying on pseudo-labeled VoxPopuli data. For Slovenian, ARTHUR 1.0 (CC-BY-SA, 884 hours) and GOS (CC-BY-SA-NC, 120 hours) collectively represent more than 1,000 hours of labeled data compared to the 32 labeled hours currently in MOSEL. The practical output of this follow-up would be a prioritized list of datasets, contact information for their creators (where available), draft re-licensing agreements, and a public tracker of re-licensing progress — essentially turning the paper's licensing diagnosis into a roadmap for community action. The paper's MOSEL GitHub repository provides a natural platform for this tracker.

YouTube-Commons cleaning and validation to determine the realistic usable fraction. The paper includes 443,396 hours of YouTube-Commons data in the headline 950k total but explicitly excludes it from pseudo-labeling due to quality issues: non-speech content, inaccurate language tags, and unsegmented transcripts (Section 2). A critical follow-up would process a representative subset of YouTube-Commons data through a cleaning pipeline — language identification (using Whisper's LID capability, as done for VoxPopuli Maltese in Table 5) to remove mislabeled segments, voice activity detection to filter out music-only videos, and automatic segmentation (e.g., using WebRTC VAD or speaker diarization) to split long videos into utterance-length chunks — and then measure: (a) what fraction of the original 443k hours survives cleaning (the paper's VoxPopuli Maltese LID analysis found 23% mislabeled, so perhaps 60-80% survives, but this will vary by language), (b) what Whisper WER the cleaned subset achieves compared to the VoxPopuli data for the same languages, and (c) whether training on cleaned YouTube-Commons data plus the same labeling scheme improves downstream ASR compared to the existing MOSEL recipe. This experiment would transform YouTube-Commons from an aspirational line item into a quantified resource (or a documented liability) and would directly inform whether the 950k hour headline figure is remotely realistic. Given the scale, an initial experiment might target 3 languages (a high-resource language like English, a medium-resource language like Bulgarian, and a low-resource language like Maltese) with 100 hours each of cleaned YouTube-Commons data, evaluating downstream WER improvement over the MOSEL baseline without YouTube-Commons.

Training and releasing the first EU-OSSFM — the explicit goal MOSEL is designed to enable. The paper frames itself as "the first step toward filling this gap" (Section 1), with the gap being the absence of any OS-compliant speech foundation model covering EU languages. The most direct follow-up is to actually train that model using the MOSEL data. A credible first attempt would train a multilingual Conformer-Transformer or Whisper-style encoder-decoder model (500M–1B parameters, somewhere between the paper's 116M proof-of-concept and Whisper large v3's ~1.55B) on all available MOSEL data: the 505.7k labeled hours plus the 441k pseudo-labeled hours (excluding YouTube-Commons for the initial version, with the option to add cleaned YouTube-Commons data later). The model would use language-ID tokens (following the Whisper paradigm) to handle all 23 languages (Irish would be omitted or trained on the 17 available hours with heavy data augmentation and transfer learning from related Celtic languages if available). The key deliverables would be: (a) model weights released under Apache 2.0 or CC-BY, (b) full training code and configuration, (c) a data manifest mapping every training example to its MOSEL source and license, establishing an unbroken chain of OS compliance from raw data to trained model, and (d) evaluation on standard ASR benchmarks (CommonVoice, FLEURS) for all 23 languages, with comparison to Whisper large v3 (the non-OS-compliant baseline) to quantify the "cost of compliance" — how much WER is sacrificed by restricting to OS-compliant data. This training run would be computationally expensive (likely 10,000–50,000 GPU hours depending on model scale) but would close the loop on MOSEL's promise: the paper collected the data and demonstrated it works for Maltese; this follow-up would demonstrate it works at scale, for all languages, producing the artifact that MOSEL was designed to enable.

Exploring the legal and practical boundaries of "open-source" when teacher models are trained on proprietary data. The paper's pseudo-labeling pipeline uses Whisper large v3 — an Apache 2.0-licensed model trained on 5 million hours of proprietary data — to generate OS-compliant transcripts under CC-BY. This is legally permitted because Apache 2.0 does not claim ownership over model outputs, but it introduces a philosophical tension: the OS compliance of the training data (the pseudo-labels) depends on a model whose own training data is non-OS-compliant. A follow-up research question — more legal/policy than technical — would investigate whether this dependency is problematic under emerging AI governance frameworks. Specifically: (a) Does the EU AI Act's transparency requirements mandate disclosure of the teacher model's training data provenance, even if the downstream model only uses the teacher's outputs? (b) Could a model trained on CC-BY pseudo-labels generated by a proprietary-trained teacher be considered "contaminated" by the teacher's non-OS-compliant training data under certain legal theories (e.g., if the teacher model memorized and reproduced portions of its training data in the pseudo-labels)? (c) What would a fully bootstrapped OS-compliant pipeline look like — where an initial, small ASR model is trained on purely OS-compliant labeled data, then used to pseudo-label unlabeled data, then a larger model is trained on the expanded dataset, iterating until performance saturates? The paper's Maltese experiment provides an existence proof that this bootstrapping is possible in principle (Whisper 80.8 WER → student 39.4 WER → filtered student 23.8 WER is a two-hop improvement chain), but a dedicated investigation would quantify how many iterations are needed, whether performance plateaus below the proprietary teacher's level, and whether the final OS-compliant model can close the gap to models trained on non-OS-compliant data. This direction matters because it determines whether the open-source speech AI ecosystem can achieve independence from proprietary foundation models, or whether it will remain structurally dependent on them for pseudo-labeling.

Practical Applications and Downstream Use Cases

Rapid prototyping of ASR for underserved EU languages without legal review. Before MOSEL, a startup or research lab wanting to build a speech application for, say, Latvian would need to either (a) use a proprietary API (Google, Azure) with per-request costs and no model access, (b) use an open-weights model like Whisper (which supports Latvian but was trained on non-public data, so the resulting application cannot claim open-source compliance), or (c) attempt to assemble and license-clear their own training data — a process requiring legal expertise to evaluate CC license variants, distinguish wrapper licenses from underlying data licenses, and determine whether SA or NC terms apply to model training. The paper estimates this process consumed the authors' effort across 49 datasets (18 accepted, 31 rejected) and required interpreting license terms at the level of individual CC variants (the SA clause in footnote 6, the collection exception in footnote 7, the YouTube license exclusion for GigaSpeech). MOSEL eliminates this burden: a developer can consult the inventory's per-language table (Table 2), download the CC-BY pseudo-labels from HuggingFace, combine them with the 173 labeled Latvian hours, and train a model using the paper's open-source recipe (Conformer-Transformer, 116M parameters, CTC+CE loss) — all without legal consultation, because every data source has been pre-vetted against the OSI standard. The 23.8 WER achieved for Maltese (the worst case) suggests that Latvian (where Whisper achieves 16.7–19.4 WER on Table 7, and where 173 labeled hours are available) would likely achieve substantially better performance, making the resulting model practically usable for transcription, voice search, or meeting summarization in a language where commercial options are limited and expensive.

Cost-efficient batch transcription for EU institutions and public services. EU institutions (the European Parliament, the European Commission, the Court of Justice) have a legal obligation to provide services in all 24 official languages, creating a massive demand for speech transcription across parliamentary sessions, committee meetings, and public hearings. Currently, this demand is met by a combination of human interpreters and proprietary ASR systems, both of which are expensive and not auditable (the training data and error profiles of proprietary ASR systems are opaque, making it impossible to verify fairness across languages or to identify systematic errors for specific demographic groups). The paper's MOSEL inventory provides a path to building an OS-compliant, auditable, in-house ASR system covering all EU languages. Using the 505.7k labeled hours and 441k pseudo-labeled hours (total: ~947k hours, excluding YouTube-Commons), an institution could train a single multilingual model and deploy it at zero marginal cost per transcription hour (only inference compute, no API fees). The Maltese result (23.8 WER after filtering) establishes a performance floor: even for the hardest language, the model is usable for gisting and keyword search, and for higher-resource languages (Spanish at 4.7 WER Whisper, Italian at 5.5 WER, German at 5.7 WER), the model would likely approach human-level transcription quality. The paper's cost analysis for the pseudo-labeling step — $100,000+ on cloud, 25,500 GPU hours — is a one-time investment that amortizes over all future transcription volume, making it economically attractive compared to per-hour API pricing for institutions that transcribe thousands of hours annually. Moreover, the OS compliance means the resulting model can be shared across EU member state governments without licensing negotiations, export controls, or vendor lock-in — each member state can download, modify, and deploy the model for its own parliamentary transcription needs.

Self-improvement pipelines for continually expanding OS-compliant speech data in low-resource languages. The paper's Maltese experiment demonstrates a concrete bootstrapping loop: Whisper (80.8 WER) → student model trained on Whisper pseudo-labels + 16 labeled hours (39.4 WER) → student model trained on filtered pseudo-labels (23.8 WER). The natural extension — which the paper does not perform but which its data release enables — is to use the 23.8 WER model to generate a new round of pseudo-labels for the same 9,100 hours of VoxPopuli Maltese audio, then train a second-generation model on those (higher-quality) pseudo-labels plus the original 16 labeled hours. If the pattern holds — the teacher's WER drops from 80.8 to 23.8, producing substantially better pseudo-labels for the next iteration — the second-generation model might achieve 15–20 WER, and a third generation might approach 10–12 WER. This iterative self-improvement is directly enabled by MOSEL because the unlabeled audio is organized, OS-compliant, and persistently available (the paper releases it as a catalog, not as a one-time training set). A practical deployment scenario: a Maltese language technology NGO with limited GPU budget could download the MOSEL pseudo-labels (generation 0), train an initial model (generation 1, achieving ~24 WER), use that model to re-transcribe the VoxPopuli data overnight on a single GPU (generating generation 2 pseudo-labels), and train an improved model the next day — without ever needing to collect new audio, hire human transcribers, or access proprietary APIs. The key enabling factor is that the paper has already done the expensive first step (25,500 GPU hours of Whisper inference) and released both the pseudo-labels and the model training recipe, so the subsequent iterations require only local compute proportional to the language's data volume (~9,100 hours for Maltese, substantially less for other languages with more labeled data). This bootstrapping paradigm could be applied to any of the 23 languages where unlabeled data exists, with faster convergence expected for languages with lower initial Whisper WER (Estonian at 29.9, Finnish at 24.6, Lithuanian at 27.6).