Perch 2.0: The Bittern Lesson for Bioacoustics

ArXiv: 2508.04665

🎯 Pitch

A bird-classification model with almost no underwater training data manages to beat specialized marine systems at identifying whale species, simply by learning to tell thousands of birds apart. The secret isn't complex self-supervision—it's a brutally hard, fine-grained classification task that forces the model to pick up universal acoustic features.

---

1. Executive Summary

This paper introduces Perch 2.0, a supervised pre-trained model for bioacoustics that scales species classification from avian-only to a multi-taxa dataset of over 1.5 million labeled recordings spanning 14,597 species classes. The model combines a prototype-learning classifier with self-distillation (where prototype predictions serve as soft targets for a linear classifier) and a source-prediction training criterion (classifying which source recording a 5-second audio window originates from) alongside generalized multi-component mixup augmentation. Perch 2.0 achieves state-of-the-art performance on the BirdSet and BEANS benchmarks, matching or exceeding prior models without any embedding-model fine-tuning, and outperforms specialized marine models on cetacean transfer learning tasks (0.981 vs. 0.954 ROC-AUC on DCLDE 2026 species classification) despite having almost no underwater training data, establishing that fine-grained species classification serves as a robust pre-training task for bioacoustics transfer learning even when the pre-training taxa differ substantially from the downstream domain.

2. Context and Motivation

The Core Problem: Supervised Learning Dominates Bioacoustics, But We Don't Fully Understand Why

The fundamental question this paper tackles is both practical and philosophical: why does supervised pre-training on fine-grained species classification consistently outperform self-supervised methods in bioacoustics, and how far can we push this supervised approach? This matters because the broader machine learning field has undergone a dramatic shift toward self-supervised foundation models — models trained on vast amounts of unlabeled data that can then adapt to many downstream tasks. In natural language processing, models like GPT and BERT; in computer vision, models like DINOv2 and MAE; in speech, models like wav2vec 2.0 — all have demonstrated that learning from unlabeled data at scale produces representations that transfer broadly. Bioacoustics, however, has stubbornly resisted this trend.

The authors frame this tension explicitly in Section 4.1 through what they call "The Bittern Lesson" — a playful reference to Richard Sutton's famous essay "The Bitter Lesson," which argued that AI progress comes from simple, general-purpose methods that leverage computation. The "bittern lesson" from Perch 2.0 is that "simple, supervised models are difficult to beat." This is not merely an empirical observation; it's a claim that demands explanation. If self-supervision works in vision and speech, why not in bioacoustics? And if supervision is indeed the right path, what design choices maximize the quality of the resulting representations for transfer learning?

The Practical Stakes: Enabling Conservation Biology at Scale

The real-world importance of this problem is substantial. Bioacoustics — the study of sound produced by animals — has become a critical tool for conservation biology and biodiversity monitoring (Laiolo, 2010). Passive acoustic monitoring deployments can collect thousands of hours of audio from remote locations, creating a bottleneck where the data volume far outstrips human capacity to annotate it. Machine learning models that can automatically detect and classify species from audio are therefore not just academic curiosities; they are essential infrastructure for understanding and protecting ecosystems.

The paper identifies a specific set of constraints that make the problem particularly challenging in practice (Section 1, final paragraph; Section 2.5). The target users — field biologists, conservation practitioners, and ecologists — often have:

1. Limited computational resources: They need models that run on consumer-grade hardware, not datacenter clusters.
2. Limited machine learning expertise: They cannot be expected to design custom architectures or training procedures for each new task.
3. Limited labeled data: The species or sound types they care about may have few or no labeled examples.
4. Novel domains: The acoustic conditions (underwater, tropical forests, urban soundscapes) may differ radically from any training data.

These constraints point toward a specific desideratum: a relatively small, frozen embedding model that produces linearly separable representations, enabling practitioners to use simple techniques like linear probing, nearest-neighbor search, and clustering on their own data. This is precisely the design philosophy behind Perch — and it stands in contrast to approaches that require fine-tuning the entire model, which demands more compute, more expertise, and more labeled data.

Prior Approaches and Where They Fall Short

The paper situates itself against several categories of prior work, each with identifiable limitations.

Perch 1.0 (Hamer et al., 2023) set the template but had clear room for improvement. The original Perch model demonstrated that supervised training on avian species classification (using Xeno-Canto data) produced embeddings that transferred surprisingly well to non-avian tasks — individual identification, call-type classification, and even non-avian species classification. However, Perch 1.0 was trained exclusively on birds, leaving open the question of whether expanding to multi-taxa data would improve transfer, and it did not incorporate the self-distillation, prototype learning, or source prediction mechanisms that Perch 2.0 introduces.

BirdNET (Kahl et al., 2021) demonstrated the power of large-scale supervised avian models but was designed primarily for bird species identification rather than as a general-purpose embedding model for bioacoustics. It used a different architecture and training procedure, and while its embeddings have been used for transfer learning, it was not explicitly optimized for that purpose.

Self-supervised bioacoustics models have been tried but underperform. The paper cites several examples: Bird-MAE (Rauch et al., 2025a) applies masked autoencoding to bird audio; BirdAVES (Hagiwara, 2023) adapts the wav2vec 2.0 framework; and SimCLR-style contrastive models have been explored (Moummad et al., 2024). The authors are blunt about their assessment of these approaches:

"these approaches generally struggle to outperform strong supervised baselines"

They further disclose their own negative results:

"In our exploratory research we too have experimented with a variety of self-supervised methods such as MAEs, HuBERT and SimCLR but experienced a similar inability to consistently outperform supervised models."

This is a significant collective negative result. It's not that self-supervised bioacoustics models haven't been tried — they have, by multiple groups using multiple methods, and they consistently come up short. The paper's position is that this pattern demands explanation, not just acceptance.

Specialized marine models exist but don't generalize well to new tasks. The paper compares against SurfPerch (Williams et al., 2025) and Google's Multispecies Whale Model (Harvey et al., 2024; Allen et al., 2024). These models were trained specifically on marine acoustic data and achieve strong performance within their training domains. However, when evaluated on transfer learning tasks (few-shot classification on unseen species or ecotypes), Perch 2.0's embeddings often outperform them — a striking result given that Perch 2.0's training data contains "a few dozen cetacean recordings, but these were mostly phone recordings made above water and not reflective of underwater hydrophone recordings" (Appendix C.2). This suggests that the specialized models, while effective classifiers, may not produce embeddings that are as generally useful for transfer learning.

Iterative pseudo-labeling has shown promise in BirdCLEF competitions (Kahl et al., 2024), where top-performing systems often use multiple rounds of training: a model is trained on labeled data, makes predictions on unlabeled data, and the high-confidence predictions are added to the training set. Perch 2.0's self-distillation mechanism (Section 2.3) can be seen as an online, within-model variant of this idea: instead of separate training rounds, the prototype classifier generates soft targets that the linear classifier learns from, creating a feedback loop that improves representation quality without requiring additional unlabeled data.

How This Paper Positions Itself

The paper's positioning is distinctive in several ways.

It argues for the primacy of supervised learning, but not dogmatically. Rather than simply declaring that supervision wins, Section 4.1 develops a set of hypotheses for why this might be the case, which serves as both explanation and research agenda:

1. Scale of labeled data may be sufficient, scale of unlabeled data insufficient. The paper notes that strong self-supervised vision models like DINOv2 were trained on 142 million images, while Xeno-Canto and iNaturalist together provide roughly 1.5 million recordings — two orders of magnitude smaller. The hypothesis is:

   "Perhaps bioacoustics will need to unlock significant amounts of diverse, unlabeled data to replicate the success of self-supervised learning in vision."

   This reframes the supervised-vs-self-supervised debate as potentially a data-scale question rather than a fundamental one.

2. Domain-specific augmentations matter, and the right ones may not have been found. Self-supervised methods are highly sensitive to the choice of data augmentations (Morningstar et al., 2024), and the space of possible audio augmentations for bioacoustics is vast and underexplored. The paper notes that:

   "It is possible that work remains on finding the right data augmentations to use in bioacoustics."

   This is a nuanced position: self-supervision isn't ruled out in principle, but the current state of augmentation design may not be sufficient.

3. Label granularity provides a strong inductive bias for transfer. This is perhaps the paper's most interesting theoretical contribution. Drawing on work by Cole et al. (2022) and Hong et al. (2024), the paper argues that fine-grained supervised labels — in this case, distinguishing between species within the same genus — force the model to learn features that are inherently transferable. The reasoning, elaborated in Section 4.1 and experimentally validated in Appendix C.1, is that coarse labels (e.g., "bird" vs. "mammal") allow the model to rely on superficial features, while fine-grained labels (e.g., distinguishing two closely related warbler species) require the model to attend to the subtle acoustic features that actually characterize species identity. These subtle features — fine spectral structure, temporal patterns, frequency modulation — turn out to be useful across taxa, which explains why a model trained primarily on birds transfers to mammals, anurans, and insects.

   The paper provides direct evidence for this in Appendix C.1 (Table 7): when labels are systematically coarsened (genus → family → order → class), transfer learning performance degrades steadily. This is not just an interesting observation — it is a design principle that justifies the entire approach of training on thousands of fine-grained species classes.

4. Universal mechanisms of sound production exist across terrestrial vertebrates. The paper cites Elemans et al. (2015) to support the claim that birds and mammals share fundamental sound production mechanisms — particularly in the larynx/syrinx and vocal tract. This provides a biological basis for why features learned on birdsong should transfer to mammal vocalizations. Combined with the extraordinary diversity of birdsong (Kroodsma, 2004), which spans an enormous acoustic range, training on birds may inadvertently cover much of the acoustic space that other taxa occupy.

It positions Perch 2.0 as a model for practitioners, not just a benchmark entry. The paper is unusually explicit about its target use case (Section 2.5):

"In practice, bioacoustics models like Perch are often used by practitioners who have limited computational resources and machine learning expertise while working on novel problems with little or no labeled data (but potentially large amounts of unlabeled data)."

This practitioner focus shapes every design decision. The model is based on EfficientNet-B3 (12 million parameters), chosen to be "small by modern machine learning standards" to reduce processing time. The embeddings are frozen during evaluation — no fine-tuning — because frozen embeddings can be "computed once and reused for large datasets," enabling vector search and clustering workflows. The model selection procedure (Section 2.5.1) evaluates on a diverse set of 19 datasets spanning multiple taxa, domain shifts, and task types (pretrained classification, one-shot retrieval, linear transfer), explicitly designed to predict real-world deployment performance rather than benchmark optimization.

It introduces methodological innovations that independently advance the state of the art. While the paper's primary contribution is the model and its performance, it also introduces several techniques that could be adopted by other researchers:

• Multi-component mixup (Section 2.1): Generalizing mixup beyond two components using a beta-binomial distribution to choose the number of sources and a symmetric Dirichlet distribution for the mixing weights. The paper notes that Vizier (the hyperparameter optimizer) "preferred models that mixed multiple signals, N ∈ {2, …, 5}" (Section 2.4), suggesting this is a non-trivial advantage over standard two-component mixup. A subtle but important detail: unlike the original mixup which uses a weighted average of one-hot targets, Perch 2.0 uses a multi-hot target vector, reasoning that "all vocalizations in an audio window should be recognized with high confidence irrespective of their loudness."

• Prototype-learning self-distillation (Section 2.2, 2.3): The prototype classifier (from ProtoPNet/AudioProtoPNet) learns class-specific prototypes in the spatial embedding space, with predictions based on maximum activation across prototypes. A stop-gradient separates the prototype classifier from the embedding model, so the prototype classifier's gradients don't flow back; instead, its predictions become soft targets for the linear classifier. This is self-distillation where both teacher and student share the embedding model. The two-phase training — first training the prototype classifier, then using its predictions for distillation — is a specific recipe that the hyperparameter search validated.

• Source prediction (DIET) as auxiliary task (Section 2.2, 2.3): Assigning each of the 1.5 million source recordings its own class and training the model to predict which recording a 5-second window came from. This is a clever auxiliary task because it is inherently self-supervised (no labels needed beyond recording identity) but is implemented as a standard classification loss. The key insight is that data augmentation — specifically, the window selection procedure that can produce non-overlapping 5-second windows from the same recording — forces the model to learn features that are invariant to time shifts and are characteristic of the specific recording's acoustic conditions (background noise, recording equipment, etc.). The paper notes that a similar objective was shown to be useful for individual animal identification (Lapp et al., 2025), suggesting it learns individual-level features that complement species-level features.

In summary, the paper addresses a concrete gap — the lack of a strong, general-purpose embedding model for bioacoustics that works across taxa and task types under practitioner constraints — while simultaneously investigating why supervised learning is so effective in this domain. It positions itself not as a repudiation of self-supervised methods, but as an exploration of what is possible with well-designed supervision, leaving open the possibility that future breakthroughs in data scale or augmentation design could shift the balance. The methodological contributions (multi-component mixup, prototype self-distillation, source prediction) are offered as practical techniques that improve supervised training, while the analysis of label granularity provides a conceptual framework for understanding why fine-grained classification is a particularly robust pre-training task for bioacoustics.

3. Technical Approach

3.1 Reader Orientation

Perch 2.0 is a supervised deep learning system that converts 5-second audio recordings of animal vocalizations into fixed-size numerical vectors (embeddings) that capture species-identifying acoustic features, enabling downstream tasks like species classification, few-shot learning, and similarity search without any further training of the core neural network. The system solves the problem of making bioacoustic analysis accessible to practitioners with limited compute, limited labeled data, and limited machine learning expertise: rather than requiring each new deployment to train a custom model from scratch, Perch 2.0 provides a single frozen embedding model that produces linearly separable representations, meaning a simple linear classifier trained on as few as 16 examples per class can achieve state-of-the-art performance on species identification across birds, mammals, anurans, insects, and even marine taxa that were almost entirely absent from training.

3.2 Big-Picture Architecture (Diagram in Words)

The Perch 2.0 system has seven major components that process information in a feedforward pipeline during inference, with additional components active only during training:

1. Frontend (spectrogram converter): Takes raw 5-second monaural audio at 32 kHz and produces a log mel-spectrogram — a 2D time-frequency representation with 500 time frames and 128 mel-scaled frequency bins. This is a fixed, non-learned transformation.

2. Embedding model (EfficientNet-B3): A 12-million-parameter convolutional residual network that processes the spectrogram and outputs two representations: a spatial embedding $E_S$ of shape $(5, 3, 1536)$ (preserving time, frequency, and feature dimensions) and a mean embedding $E_A$ of shape $(1536,)$ obtained by averaging $E_S$ over the spatial dimensions. The mean embedding $E_A$ is the primary output used for all downstream tasks.

3. Linear classifier head (training only): A single fully-connected layer that maps the 1536-dimensional $E_A$ to a 14,795-dimensional vector of logits, one per training class. Trained with cross-entropy loss against multi-label targets derived from species labels and mixup.

4. Prototype learning classifier head (training only): An interpretable classifier that learns four spatial prototypes per class, computing predictions by measuring how strongly each prototype activates against the spatial embedding $E_S$. A stop-gradient prevents this head's gradients from flowing back to the embedding model.

5. Self-distillation mechanism (training only): The prototype classifier's predictions are used as soft targets to train the linear classifier (both heads share the same embedding model). This is a form of model self-distillation where the prototype classifier acts as a "teacher" providing richer training signals than hard one-hot labels.

6. Source prediction head (training only): A low-rank linear classifier (rank 512) that maps $E_A$ to predict which of the 1.5+ million unique source recordings the audio window came from. This auxiliary self-supervised task forces the embedding model to learn recording-specific features that complement species-level features.

7. Data augmentation pipeline (training only): Includes random 5-second window selection (either uniformly random or energy-peak-guided) and multi-component mixup that blends 2–5 audio sources together with corresponding multi-hot labels.

The information flow during training is: raw audio → frontend → embedding model → three parallel heads (linear classifier, prototype classifier, source predictor) → three losses computed and summed → backpropagation through embedding model (but stop-gradient blocks prototype classifier gradients). During inference, only the frontend and embedding model are used, producing $E_A$ vectors that downstream applications consume directly.

3.3 Roadmap for the Deep Dive

• First, the frontend and spectrogram conversion, because every subsequent component operates on the spectrogram representation, and the specific parameter choices (hop length, window size, frequency range) determine what acoustic information is preserved or discarded.
• Second, the embedding model architecture (EfficientNet-B3), because it is the core learned component whose output embeddings must capture transferable acoustic features. Understanding its design (depthwise convolutions, parameter efficiency, spatial-to-mean pooling) explains why the model remains small enough for practitioner hardware.
• Third, the training data pipeline (sources, window selection, mixup), because the quality and diversity of training examples directly determines what the embedding model learns, and the window selection and mixup strategies are key design choices that address the weak-label problem inherent in bioacoustics recordings.
• Fourth, the three training objectives (species classification, self-distillation, source prediction) and their associated loss functions, because they jointly define what the model is optimized to do. The interaction between these objectives — particularly the two-phase training schedule and the stop-gradient separating the prototype classifier — is the core algorithmic contribution.
• Fifth, the hyperparameter optimization process (Vizier) and the discovered patterns, because the specific hyperparameter values (dropout rates, mixup parameters, loss weights) encode empirical knowledge about what works for bioacoustic supervised learning, and the differences between the two training phases reveal how the optimization landscape changes.
• Sixth, the model selection and evaluation protocol, because the paper's practitioner-focused philosophy requires evaluation procedures that predict real-world deployment performance rather than benchmark overfitting. The 19-dataset validation suite spanning pretrained classification, one-shot retrieval, and linear transfer tasks is itself a methodological contribution.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and empirical methodology paper whose core idea is that supervised pre-training on fine-grained, multi-taxa species classification — augmented with multi-component mixup, prototype-based self-distillation, and source prediction as an auxiliary task — produces embedding models that achieve state-of-the-art transfer learning performance across diverse bioacoustic domains while remaining small enough for practitioner deployment. The paper does not introduce a fundamentally new architecture or learning algorithm; rather, it combines and refines existing techniques into a carefully tuned recipe and provides empirical evidence for why supervised learning remains dominant in bioacoustics.

---

Frontend: Converting Raw Audio to Mel-Spectrograms

The frontend is a fixed (non-learned) signal processing pipeline that converts raw audio into a time-frequency representation suitable for input to a convolutional neural network. All details are specified in Appendix A.2.

Input: Monaural audio sampled at 32 kHz, exactly 5 seconds in duration (160,000 samples). The fixed 5-second length is a design choice that constrains the model to a consistent input size and provides sufficient temporal context to capture most animal vocalizations while remaining short enough for efficient processing.

Processing steps:

1. Framing: The 5-second audio is divided into overlapping frames using a hop length of 10 ms (320 samples) and a window length of 20 ms (640 samples). This produces 500 frames total. The frames are uncentered, meaning the first frame begins at sample 0 rather than being centered at sample 0 — this is a subtle implementation choice that affects the temporal alignment of the first and last frames.

2. Windowing: Each frame is multiplied element-wise by a Hann window (a raised cosine function that tapers to zero at the frame edges) to reduce spectral leakage.

3. FFT: A 1,024-point Fast Fourier Transform is applied to each windowed frame, producing 513 unique frequency bins from DC to the Nyquist frequency (16 kHz). The choice of 1,024 FFT points with a 640-sample window means the window is zero-padded to 1,024 samples before the FFT, providing frequency interpolation that yields smoother spectra. The paper notes this is done "for computational efficiency" — 1,024 is a power of two that maps well to hardware FFT implementations.

4. Magnitude spectrogram: The energy (magnitude) of each complex FFT coefficient is computed, producing a real-valued spectrogram. The paper explicitly notes this is the magnitude spectrogram, not the power spectrogram — power would square the magnitudes, which compresses dynamic range differently.

5. Scaling: The magnitude values are scaled by the reciprocal of the sum of the Hann window values. This normalizes for the energy lost by windowing, ensuring that the spectrogram amplitudes are comparable regardless of window length.

6. Mel-scale filtering: The 513 linear-frequency bins are mapped to 128 mel-scaled bins using the HTK formula (a specific mel-scale definition that differs slightly from the more common Slaney formula). The mel scale is a perceptual frequency scale where equal distances correspond to approximately equal perceptual pitch differences, with lower frequencies receiving more resolution than higher frequencies. The frequency range is 60 Hz to 16 kHz — the 60 Hz lower bound excludes very low-frequency noise (wind, handling noise) while the 16 kHz upper bound captures most animal vocalizations including high-frequency bird calls and bat echolocation was explicitly excluded by removing bat recordings from training data.

7. Logarithmic compression: A logarithm with a floor of $10^{-5}$ is applied to each mel bin value, then the result is multiplied by 0.1. The floor prevents taking the logarithm of zero (which would produce negative infinity) for silent frequency bins. The multiplication by 0.1 scales the log-magnitudes to a range suitable for neural network training (roughly $[-0.5, 0]$ for typical audio).

Output: A mel-spectrogram tensor of shape $(500, 128, 1)$ (500 time frames × 128 mel-frequency bins × 1 channel), where each value represents the log-magnitude of acoustic energy in that time-frequency cell. This is treated as a single-channel image for input to the 2D convolutional embedding model.

Why these parameters? The 10 ms hop length provides 100 frames per second, giving 10 ms temporal resolution — sufficient to capture fast temporal modulations in animal calls (e.g., rapid trills or frequency sweeps). The 20 ms window provides reasonable frequency resolution (50 Hz) while maintaining temporal locality. The 128 mel bins provide sufficient frequency resolution to distinguish fine spectral features (harmonics, formants) while keeping the input dimensionality manageable for a 12M-parameter model. The 5-second duration is chosen to capture complete vocalizations — many bird songs and mammal calls last 1–5 seconds — without including excessive silence that would dilute the training signal.

---

Embedding Model: EfficientNet-B3 with Spatial and Mean Pooling

The embedding model is the core learned component. It is an EfficientNet-B3 architecture (Tan and Le, 2019), a convolutional neural network designed for image classification but adapted here to operate on spectrograms.

Why EfficientNet-B3? EfficientNets are a family of models that use compound scaling — simultaneously scaling network depth, width, and input resolution — optimized through neural architecture search. The key innovation is depthwise separable convolutions, which factor a standard convolution into two operations: a depthwise convolution (a separate spatial filter for each input channel) followed by a pointwise convolution (a 1×1 convolution that mixes channels). This factorization dramatically reduces parameters and FLOPs compared to standard convolutions while maintaining representational capacity. For example, a standard 3×3 convolution with $C_{in}$ input channels and $C_{out}$ output channels requires $3 \times 3 \times C_{in} \times C_{out}$ parameters, while a depthwise separable version requires only $3 \times 3 \times C_{in} + C_{in} \times C_{out}$ — a reduction of approximately $9\times$ for typical channel counts.

The specific EfficientNet variant:

• B3: The third scaling level in the EfficientNet family, with approximately 12 million parameters. This is larger than Perch 1.0's EfficientNet-B1 (7.8M parameters), reflecting "the increased amount of training data." The scaling follows the compound scaling rule: depth scales as $1.2^{\phi}$, width as $1.1^{\phi}$, and resolution as $1.15^{\phi}$ for a compound coefficient $\phi$.
• Input: The $(500, 128, 1)$ spectrogram is treated as a single-channel 2D image. The model applies multiple stages of convolution, batch normalization, and Swish activation (a smooth ReLU variant: $x \cdot \sigma(x)$ where $\sigma$ is the sigmoid function), with progressive downsampling through strided convolutions and pooling.
• Residual connections: Each "block" in EfficientNet contains skip connections that add the block's input to its output, enabling gradient flow through deep networks and making optimization easier.

Output structure: Unlike standard EfficientNet which produces a single pooled feature vector before the classifier, Perch 2.0's embedding model produces two representations at different levels of spatial resolution:

• Spatial embedding $E_S$: Shape $(5, 3, 1536)$. This is the feature map from the final convolutional layer before global pooling. The spatial dimensions (5 time, 3 frequency) are the result of multiple stages of 2×2 downsampling from the original 500×128 input. Each of the 15 spatial locations contains a 1536-dimensional feature vector representing the acoustic content at a specific coarse time-frequency region. The 1536 channels come from the final convolutional layer's output channels in the EfficientNet-B3 architecture.

• Mean embedding $E_A$: Shape $(1536,)$. This is obtained by averaging $E_S$ over the spatial (time and frequency) dimensions. The averaging operation has an important property: it produces a fixed-size vector regardless of the exact temporal alignment of sounds within the 5-second window, providing a degree of translation invariance. The subscript "A" likely stands for "aggregated" or "averaged."

Why produce both embeddings? The spatial embedding $E_S$ preserves locality information that is crucial for the prototype learning classifier (which needs to match prototypes to specific time-frequency regions) and for tasks that require detecting where in the recording a sound occurs. The mean embedding $E_A$ provides a compact, fixed-size representation that is suitable for linear classification, nearest-neighbor search, and efficient storage/retrieval. The paper uses $E_A$ for all downstream evaluation tasks (linear probing, retrieval), while $E_S$ is used only for the prototype classifier during training.

Design note on model size: The 12M parameter count is "small by modern machine learning standards" — for context, contemporary vision transformers often have 86M–632M parameters (ViT-Base through ViT-Huge), and large language models are in the billions. This small size means the model can run on consumer-grade hardware (laptops, single GPUs) and process large datasets efficiently. The paper emphasizes that this enables "embedding once and reusing" for downstream tasks — a 1,000-hour audio dataset can be embedded in hours on a single machine, and the resulting embeddings can be searched, clustered, and classified without ever re-running the expensive neural network.

---

Training Data Pipeline: Sources, Window Selection, and Mixup

The training data pipeline converts a collection of weakly labeled audio recordings into the supervised training examples that the embedding model learns from. This is a critical component because the quality and diversity of training data directly determines the embedding model's transfer performance, and the "weak label" problem — knowing which species is in a recording but not when it vocalizes — requires careful design.

Data Sources and Taxonomic Scope

The training data combines four labeled datasets (Table 1):

The taxonomic breakdown matters: 1.37M of the 1.54M recordings (89%) are avian (Aves), with the remaining 11% split among Amphibia, Insecta, Mammalia, and "Other" (FSD50K's general sound classes). This heavy avian bias is both a limitation and, arguably, a strength — the paper hypothesizes that "the great diversity of birdsong" forces the model to learn a wide range of acoustic features, and that "universal mechanisms of sound production in terrestrial vertebrates" make these features transferable across taxa.

Label space: 14,795 classes total, of which 14,597 are species labels and 198 are FSD50K sound event classes. The paper notes that Xeno-Canto, iNaturalist, and Tierstimmenarchiv use different taxonomies, so labels were "manually mapped" to the iNaturalist taxonomy — a non-trivial data engineering effort that reconciles naming differences across databases.

Bat exclusion: All bat recordings were removed from the datasets because "their vocalizations cannot be represented using the spectrogram parameters we selected." Bat echolocation calls typically use frequencies of 20–200 kHz, far above the 16 kHz Nyquist frequency (half of 32 kHz sampling rate), and their pulse-echo structure differs fundamentally from the tonal, frequency-modulated vocalizations of birds and terrestrial mammals.

The Weak Label Problem and Window Selection

Most recordings in the training data are longer than the 5-second model input, ranging "from less than a second to over an hour (with the majority in the 5–150 s range)." The species label applies to the entire recording, but the species may only vocalize for a small portion. This is the weak label problem: if we randomly crop a 5-second window from a 60-second recording, the window might contain silence, noise, or a different species.

The paper explores two window selection strategies:

1. Random window selection: Each time a recording is selected for training, a uniformly random 5-second window is extracted. This is simple but introduces label noise — some training examples will be labeled with the target species but contain no vocalization from that species. The authors hypothesize that "the resulting label noise might be of acceptable levels," presumably because (a) for short recordings with frequent vocalizations, random windows usually capture the target species, and (b) deep neural networks trained with stochastic gradient descent are somewhat robust to label noise, especially when the noise is not systematic.

2. Energy peak selection: This heuristic (detailed in Appendix B) attempts to select windows containing the strongest acoustic signal, operating on the assumption that "the labeled species is likely the most prominent sound in the recording." The procedure:

1. Initial spectrogram: Construct a mel-spectrogram of the entire recording using a window size of 80 ms (longer than the 20 ms frontend window) and a hop of 10 ms, then log-scale and multiply by 0.1. The longer window provides better frequency resolution for peak detection.

2. Two-step denoising: For each frequency bin, compute the mean and standard deviation of log-magnitudes across time. Discard values exceeding the mean plus 1.5 standard deviations. Recompute the mean and standard deviation using remaining values. The second statistics are more robust because loud transient events (which might be the target vocalization or might be noise) are excluded from the initial mean estimate. Define "signal" as values above the second mean plus 0.75 standard deviations.

3. Collapse to 1D: Sum the denoised magnitudes across all frequency bins, producing a single time series representing total acoustic energy at each time step.

4. Peak finding: Use SciPy's signal.find_peaks_cwt with 10 wavelet filters to detect peaks of width 0.5–2 seconds. For each peak, compute the total energy in a 600 ms window centered on the peak.

5. Peak filtering: Discard peaks whose total energy is less than 1.5 times the mean energy over the entire recording (removes low-energy peaks that are probably noise). Keep the top 5 peaks sorted by total energy.

6. Window selection: Select a 6-second window centered on one of the top peaks, then randomly select a 5-second window within this larger window. The 6-second window provides a buffer so the peak is likely fully contained in the 5-second training window. The random selection within the 6-second window adds some jitter to prevent the model from memorizing exact temporal positions.

For recordings shorter than 6 seconds, pad with zeros before peak finding. If no peaks survive filtering, select the first 6 seconds. This heuristic was "important in the training of Perch 1.0," but Perch 2.0 surprisingly finds that random window selection "performs on par with using energy peak selection" — a finding the authors attribute to the self-distillation phase helping address label noise.

Multi-Component Mixup

Mixup (Zhang et al., 2018) is a data augmentation that creates new training examples by linearly combining existing examples and their labels. The original formulation mixes two examples $(x_i, y_i)$ and $(x_j, y_j)$ as:

$\tilde{x} = \lambda x_i + (1 - \lambda) x_j$

$\tilde{y} = \lambda y_i + (1 - \lambda) y_j$

where $\lambda \sim \text{Beta}(\alpha, \alpha)$ is a mixing coefficient. This encourages the model to behave linearly between training examples, which has been shown to improve generalization and robustness.

Perch 2.0 generalizes mixup beyond two components with a three-stage sampling procedure:

Step 1 — Number of components: Sample $N$, the number of audio sources to mix, from a shifted beta-binomial distribution:

$N \sim \text{BetaBin}(n, \alpha, \beta) + 1$

where $n$ is the maximum number of additional components (so $N_{\text{max}} = n + 1$), $\alpha$ and $\beta$ are shape parameters controlling the distribution's concentration. The $+1$ ensures at least one component (no degenerate empty mix). The beta-binomial is a discrete distribution where the probability of success in a binomial is drawn from a beta distribution, providing more flexibility than a simple binomial.

The reported optimal hyperparameters are: $n = 2$, $\alpha = 91.3$, $\beta = 100$. With $n = 2$, the maximum number of components is 3 (since $N \in \{1, 2, 3\}$ after shifting). However, the paper notes that during the first phase, Vizier "preferred models that mixed multiple signals, $N \in \{2, \ldots, 5\}$," suggesting that the best models sometimes used higher $n$ values during the search but the final selected model used $n = 2$.

Step 2 — Mixing weights: Sample a weight vector $\mathbf{w} = (w_1, \ldots, w_N)$ from a symmetric Dirichlet distribution:

$\mathbf{w} \sim \text{SymDir}(N, \omega)$

where $\omega$ is the concentration parameter. The symmetric Dirichlet generalizes the Beta distribution to $N$ dimensions: when $\omega < 1$, the distribution concentrates on sparse weight vectors (one component dominates); when $\omega > 1$, weights tend toward uniform (all components contribute equally); when $\omega = 1$, the distribution is uniform over the simplex. The final model uses $\omega = 1$, placing uniform distribution over all possible mixture weightings.

Step 3 — Signal construction: The composite audio signal $\tilde{x}$ is:

$\tilde{x} = \frac{\sum_{i=1}^{N} w_i x_i}{\sqrt{\sum_{i=1}^{N} w_i^2}}$

In plain language: take the weighted sum of the $N$ audio waveforms (each $x_i$ is a 5-second, 160,000-sample tensor), then divide by the root-sum-of-squares of the weights. This normalization ensures that the overall gain (RMS energy) of the mixed signal remains approximately unchanged regardless of the mixing weights. Without this normalization, mixing three signals with equal weights would produce a louder signal than mixing two, which could create an unwanted correlation between the number of mixing components and signal amplitude that the model could exploit as a shortcut.

Label construction (multi-hot): Unlike standard mixup, which uses a weighted average of one-hot label vectors, Perch 2.0 creates a multi-hot target vector where each of the $k$ target species present in the mixture has value $1/k$. The paper states:

"unlike the original mixup implementation, we construct a multi-hot target vector rather than taking a weighted average of one-hot target vectors, as this reflects the fact that all vocalizations in an audio window should be recognized with high confidence irrespective of their loudness."

This is a critical design choice. Standard mixup's weighted-average target is appropriate for vision, where a mixed image of a dog and cat is genuinely ambiguous and should produce intermediate predictions. But for bioacoustics, if a recording contains both a robin and a sparrow at different volumes, the correct behavior is to detect both species with high confidence, not to produce a weighted blend. The multi-hot target enforces this behavior.

Implementation note on label noise: Even with multi-hot targets, mixup introduces some label noise because the mixed signal might contain overlapping vocalizations that are acoustically unrealistic. However, the paper's hyperparameter search validated that mixup is beneficial — the first phase found that Vizier preferred models with active mixup, though interestingly, during the self-distillation phase, Vizier preferred "little or no mixup (mostly $N = 1$, i.e., no mixing)." This suggests mixup is most valuable when the model is learning from scratch (providing regularization and data diversity) but becomes less necessary during the fine-tuning/distillation phase when the model is refining its representations.

---

Training Objective 1: Species Classification Cross-Entropy

The primary training objective is supervised species classification using a multi-label cross-entropy loss applied to the linear classifier's predictions.

Classifier: A single fully-connected (dense) layer that maps the 1536-dimensional mean embedding $E_A$ to a vector of 14,795 logits, one per training class. This is a linear classifier in the embedding space — meaning that for each class $c$, the logit is:

$z_c = \mathbf{w}_c^T E_A + b_c$

where $\mathbf{w}_c \in \mathbb{R}^{1536}$ is the weight vector for class $c$, $b_c$ is the bias, and $z_c$ is the unnormalized logit.

Activation: Softmax is applied across all classes to produce a probability distribution:

$p_c = \frac{\exp(z_c)}{\sum_{j=1}^{14795} \exp(z_j)}$

The paper uses softmax rather than the more common sigmoid binary cross-entropy used in multi-label classification. This is a deliberate choice following Mahajan et al. (2018):

"We have found this to train faster than using the more traditional sigmoid binary cross-entropy used in multi-label classification."

Target vector: For each training example, the target vector $\mathbf{t} \in [0, 1]^{14795}$ is constructed such that for each of the $k$ target classes present (determined by species labels and mixup multi-hot labeling), the target value is $1/k$. For a single-species example with no mixup, $k = 1$ and the target is a standard one-hot vector.

Loss:

$\mathcal{L}_{\text{cls}} = -\sum_{c=1}^{14795} t_c \log(p_c)$

where $t_c$ is the target value for class $c$ and $p_c$ is the predicted probability.

What this loss computes: For each of the $k$ target classes, the loss penalizes low predicted probability — the model receives a strong gradient to increase $p_c$ when $t_c = 1/k$. For non-target classes ($t_c = 0$), there is no gradient contribution (because $0 \cdot \log(p_c) = 0$), meaning the model is not explicitly penalized for predicting high probability on incorrect classes. This is a consequence of the softmax formulation — the softmax normalization inherently suppresses non-target probabilities because probabilities must sum to 1. When the model increases probabilities for target classes, it automatically decreases probabilities for non-target classes.

Why softmax over sigmoid binary cross-entropy? In sigmoid binary cross-entropy, each class is treated independently: the model receives a penalty for each incorrect high-probability prediction regardless of whether other classes are correct. For 14,795 classes where typically only 1–5 are present, this produces an extreme class imbalance (14,795 negative gradients vs. 1–5 positive gradients per example), which can slow training and cause the model to predict near-zero for all classes (a local minimum where the loss is already low because most negatives are correctly predicted). The softmax formulation avoids this by normalizing across classes, effectively making the classes compete for probability mass. The Mahajan et al. (2018) finding that this "trains faster" is likely because the softmax gradient signal is more concentrated on the relevant classes.

---

Training Objective 2: Self-Distillation via Prototype Learning

The self-distillation mechanism is the most architecturally novel component of Perch 2.0. It creates a teacher-student dynamic within a single model, where the teacher (prototype classifier) and student (linear classifier) share the same embedding model but have different classification mechanisms, and the student learns from the teacher's predictions rather than directly from ground-truth labels.

Prototype Learning Classifier

The prototype classifier is based on ProtoPNet (Chen et al., 2019) as adapted for bioacoustics in AudioProtoPNet (Heinrich et al., 2025). The core idea is to learn a set of prototype vectors for each class, where each prototype represents a characteristic acoustic pattern in the spatial embedding space.

Prototypes: For each of the 14,795 classes, four prototypes are learned. Each prototype $\mathbf{p}_{c,i}$ (for class $c$, prototype index $i$) is a vector in the spatial embedding space of shape $(H_c, W_c, 1536)$, where $H_c$ and $W_c$ are spatial dimensions that are typically 1×1 for interpretability, but the paper doesn't specify exactly — in standard ProtoPNet, prototypes are 1×1 spatial patches.

Activation computation: For a given input producing spatial embedding $E_S$ of shape $(5, 3, 1536)$, the activation of prototype $\mathbf{p}_{c,i}$ is the maximum cosine similarity between the prototype and any $(H_c, W_c)$ spatial patch of $E_S$. In effect, the model scans each prototype across the spatial embedding and keeps the best match:

$a_{c,i} = \max_{\text{patches}} \frac{E_S[\text{patch}] \cdot \mathbf{p}_{c,i}}{\|E_S[\text{patch}]\| \cdot \|\mathbf{p}_{c,i}\|}$

where $E_S[\text{patch}]$ is a spatial patch of $E_S$ matching the prototype's spatial dimensions.

Class logit: The logit for class $c$ is the maximum activation across its four prototypes:

$z_c^{\text{proto}} = \max_{i \in \{1, 2, 3, 4\}} a_{c,i}$

This "max-of-max" formulation means that a class fires if any of its prototypes strongly matches some time-frequency region of the input. This is interpretable: each prototype can be visualized as a specific spectro-temporal pattern (e.g., a particular frequency sweep, a harmonic stack, or a trill pattern), and the model's prediction can be traced back to which prototype activated where.

Classification loss: The prototype classifier also uses softmax cross-entropy:

$\mathcal{L}_{\text{proto}} = -\sum_{c=1}^{14795} t_c \log\left(\frac{\exp(z_c^{\text{proto}})}{\sum_j \exp(z_j^{\text{proto}})}\right)$

using the same target vector $\mathbf{t}$ as the linear classifier.

Orthogonality loss: Following Heinrich et al. (2025) and Donnelly et al. (2022), an additional loss term encourages prototypes to be diverse and non-redundant:

$\mathcal{L}_{\text{ortho}} = \sum_{c=1}^{14795} \sum_{i=1}^{4} \sum_{j \neq i} \max(0, \cos(\mathbf{p}_{c,i}, \mathbf{p}_{c,j}) - \tau)^2$

This penalizes pairs of prototypes within the same class that have cosine similarity above a threshold $\tau$, pushing them to capture different acoustic aspects of the class rather than all converging to the most common pattern. The paper doesn't specify $\tau$ explicitly, but the quadratic penalty means that once prototypes are sufficiently dissimilar (below $\tau$), no further penalty is incurred.

Stop-Gradient and Self-Distillation Mechanism

The critical stop-gradient: A stop-gradient operation separates the embedding model from the prototype classifier:

"A stop-gradient separates the embedding model from the prototype learning classifier so that its gradients do not propagate to the embedding model."

This means that during backpropagation, the gradients from $\mathcal{L}_{\text{proto}}$ and $\mathcal{L}_{\text{ortho}}$ update only the prototype vectors and the classifier parameters, not the EfficientNet weights. Only the linear classifier, source prediction, and (during the second phase) self-distillation losses backpropagate into the embedding model.

Why stop-gradient? If the prototype classifier's gradients flowed back to the embedding model, the model could trivially reduce $\mathcal{L}_{\text{proto}}$ by reshaping $E_S$ to make prototypes easier to match, effectively cheating the classification task. The stop-gradient forces the prototype classifier to learn meaningful prototypes in whatever embedding space the linear classifier and other objectives have shaped. The prototypes thus serve as a fixed "interpretation lens" on the learned representations.

Self-distillation loss: During the second training phase, the prototype classifier's predictions are used as soft targets for the linear classifier. The self-distillation loss is:

$\mathcal{L}_{\text{distill}} = -\sum_{c=1}^{14795} \tilde{p}_c^{\text{proto}} \log(p_c^{\text{linear}})$

where $\tilde{p}_c^{\text{proto}}$ is the prototype classifier's predicted probability for class $c$ (treated as a constant target — gradients do not flow back through it), and $p_c^{\text{linear}}$ is the linear classifier's predicted probability.

What this loss computes: The KL divergence between the prototype classifier's probability distribution and the linear classifier's probability distribution, implemented as cross-entropy with the prototype distribution as the target. For each class $c$, the linear classifier is penalized when its predicted probability $p_c^{\text{linear}}$ differs from the prototype classifier's prediction $\tilde{p}_c^{\text{proto}}$.

Why self-distillation works (Allen-Zhu and Li, 2022): The prototype classifier produces a richer training signal than hard one-hot labels. A hard label says only "class 37 is correct" with no information about other classes. The soft target says "class 37 has probability 0.82, class 38 has probability 0.09, class 194 has probability 0.04..." — these secondary probabilities capture taxonomic relationships and acoustic similarities between species. For example, if the prototype classifier assigns a small but non-zero probability to a closely related species (perhaps because the vocalizations are similar), the linear classifier learns that confusing these two species is less wrong than confusing completely unrelated taxa. This is a form of dark knowledge (Hinton et al., 2015) — information about the relative similarity of classes that is present in the teacher's output distribution but absent from hard labels.

The self-distillation loss weight in the final model is $4.22$, making it the dominant loss component during phase 2 — far larger than the classification loss (weight 1.0) and source prediction loss (weight 0). This weighting was determined by Vizier hyperparameter optimization.

Two-Phase Training Schedule

The training is explicitly divided into two phases:

Phase 1 (up to 300,000 steps): The model is trained with the linear classifier loss ($\mathcal{L}_{\text{cls}}$), prototype classifier loss ($\mathcal{L}_{\text{proto}}$), orthogonality loss ($\mathcal{L}_{\text{ortho}}$), and source prediction loss ($\mathcal{L}_{\text{source}}$). The prototype classifier is trained but its predictions are not used for self-distillation yet. The purpose is to train a good prototype classifier first, since self-distillation is only useful if the teacher has non-trivial knowledge to transfer.

Phase 2 (up to 400,000 steps): The self-distillation loss ($\mathcal{L}_{\text{distill}}$) is added, using the already-trained prototype classifier's predictions as soft targets. The hyperparameter search during this phase started from the best phase-1 model and explored learning rate, loss weights, dropout, and mixup parameters specific to the distillation regime.

Phase 2 hyperparameter discoveries: Vizier's preferences during the distillation phase reveal important dynamics:

• Small learning rate: $3.20 \times 10^{-6}$, down from $6.41 \times 10^{-4}$ in phase 1 — a 200× reduction. This suggests the distillation phase is a fine-tuning stage where large weight changes would disrupt the already-good representations.
• No mixup: $n = 0$ (no additional mixing beyond the single original recording). Mixup's augmentation, valuable for initial training, may introduce noise that interferes with the precise distillation signal.
• Low or zero source prediction weight: 0 in the final model, suggesting the source prediction task's benefits are primarily in the early training stage.
• No dropout: A dropout rate of 0, down from 0.49 in phase 1. Dropout adds stochastic noise to activations, which may interfere with the deterministic distillation targets.
• High self-distillation weight: 4.22, making distillation the dominant training signal.

---

Training Objective 3: Source Prediction (DIET)

The source prediction objective is a self-supervised auxiliary task that requires the model to predict which source recording each 5-second audio window originates from. This is based on the DIET (Datum IndEx as Target) framework (Balestriero, 2023).

The task: Assign each of the 1.5+ million source recordings in the training set its own unique class label. During training, the model must predict this class from the mean embedding $E_A$.

Low-rank projection: Since predicting among 1.5 million classes directly would require a 1536 × 1.5M weight matrix (2.3 billion parameters, far exceeding the embedding model itself), the source prediction head uses a low-rank factorization:

$z_{\text{source}} = \mathbf{W}_2 (\mathbf{W}_1 E_A)$

where $\mathbf{W}_1 \in \mathbb{R}^{512 \times 1536}$ projects the 1536-dimensional embedding to a 512-dimensional bottleneck, and $\mathbf{W}_2 \in \mathbb{R}^{N_{\text{recordings}} \times 512}$ projects to the full recording space. This is equivalent to factorizing a rank-$r$ weight matrix as $\mathbf{W} = \mathbf{W}_2 \mathbf{W}_1$ with $r = 512$.

Loss: Standard softmax cross-entropy over the source recording classes:

$\mathcal{L}_{\text{source}} = -\log\left(\frac{\exp(z_{\text{source}}^{\text{true}})}{\sum_{j=1}^{N_{\text{recordings}}} \exp(z_{\text{source}}^j)}\right)$

What this loss computes: For a given 5-second audio window extracted from a known source recording, the model predicts a probability distribution over all training recordings and is penalized for assigning low probability to the correct source. In effect, the model learns to produce a "recording fingerprint" — an embedding that identifies which specific recording device, location, and acoustic environment produced the audio, independent of which species is vocalizing.

Why this is non-trivial: A single source recording might be 60 seconds long and produce 12 non-overlapping 5-second windows — these windows contain different audio content (different parts of a bird song, or silence, or different species) but must all be assigned to the same source class. The model cannot simply memorize the audio content; it must learn features that are invariant to the specific content but characteristic of the recording's acoustic conditions. The paper states:

"source prediction... is reliant on data augmentation to force the network to learn salient features. In our case the data augmentation comes in the form of windowing: Longer recordings can produce entirely non-overlapping 5 s windows which the model will need to learn to assign to the same class."

The features that enable source prediction likely include:

• Background noise signatures (specific frequency profiles of wind, rain, insects)
• Recording equipment characteristics (frequency response curves, compression artifacts)
• Reverberation patterns (acoustic properties of the recording environment)
• Narrow-band noise sources (electrical hum at 50/60 Hz, specific machine sounds)

These features are independent of the species identity and thus provide a complementary learning signal: the species classification objective pushes the model to focus on the vocalization itself while ignoring background, while the source prediction objective pushes the model to attend to background and recording characteristics. The combination may produce more robust embeddings that work well across different recording conditions.

The paper's insight on source prediction as fine-grained supervision: The authors note:

"this self-supervised method can just as well be seen as an extremely fine-grained supervised classification problem"

With 14,597 species classes already, Perch 2.0 pushes label granularity to its limit. Source prediction extends this philosophy further: instead of 14,597 species-level classes, the model now discriminates among 1.5+ million recording-level classes. This forces the model to attend to extremely subtle acoustic differences, potentially learning representations that are useful for fine-grained tasks like individual animal identification — a connection the paper makes by citing Lapp et al. (2025), which showed source-prediction-like objectives are useful for exactly that purpose.

Loss weighting: The paper notes that "given the large number of classes in the source prediction task the scale of this loss is higher than that of the species classification loss." With 1.5M classes, the source prediction cross-entropy will naturally be larger because the model is more uncertain (initial random predictions assign roughly $1/1.5\text{M}$ probability to the correct class). The phase 1 optimal weight was 0.11, and in the final phase 2 model the weight was reduced to 0 — suggesting that source prediction provides valuable regularization during early training but may not be needed (or may even be harmful) during the fine-tuning distillation phase, perhaps because the recording-specific features it learns can interfere with the species-general features needed for transfer learning.

---

Hyperparameter Optimization with Vizier

The paper uses Vizier (Golovin et al., 2017), Google's black-box optimization service, to search the hyperparameter space. This is a practical engineering choice rather than a methodological innovation, but the search procedure reveals important insights about the hyperparameter landscape.

Search procedure: Two stages per phase, with 100 models trained per stage:

• Stage 1: Vizier proposes 100 hyperparameter configurations, trains models, and evaluates on the validation tasks (Section 2.5.1).
• Stage 2: Based on the 100 validation scores, Vizier proposes an optimized set of 100 new configurations for a second round of training and evaluation.
• The best model is selected based on the geometric mean validation score.

Training cost: Each model took "between 20 and 30 hours on a TPUv3-8, depending mainly on the number of mixup signals used." With 400 total models trained across two phases and two window-selection variants, the total hyperparameter search likely consumed 8,000–12,000 TPUv3-8 hours (approximately 333–500 TPU-days). This is a substantial computational investment that reflects the paper's industrial research context and would be difficult for academic groups to replicate directly — though the reported optimal hyperparameters provide a starting point.

Phase 1 hyperparameter discoveries:

• Mixup $N \in \{2, \ldots, 5\}$: Vizier consistently preferred models that mixed multiple signals, validating that generalized multi-component mixup provides benefits beyond standard two-component mixup.
• Source prediction weight 0.1–0.9: A non-negligible weight was preferred, confirming the auxiliary task's usefulness.
• Dropout 0.3–0.6: Relatively high dropout rates were preferred, suggesting the 12M-parameter model with 14,795 classes benefits from strong regularization.
• Mixup shape parameters $\alpha > \beta$: When $\alpha > \beta$, the beta-binomial distribution skews toward higher $N$ values (more mixing components), confirming that multi-component mixup was actively used.
• Mixup concentration $\omega \in (10, 30)$: A high concentration means the Dirichlet distribution produces weights that are close to uniform (all components contribute roughly equally), rather than sparse weights where one component dominates. This is an interesting finding: mixing multiple audio sources with approximately equal weighting provides the best regularization.

Phase 2 hyperparameter discoveries (discussed in Section 3.4, self-distillation subsection above) revealed a shift toward fine-tuning behavior: very low learning rate, no mixup, no dropout, no source prediction, high distillation weight.

---

Model Selection and Evaluation Protocol

The model selection and evaluation philosophy is central to the paper's practitioner-oriented approach. Rather than selecting the model that performs best on a single benchmark (which risks overfitting to that benchmark's idiosyncrasies), the paper designs a multi-faceted validation procedure that approximates real-world deployment scenarios.

Model Selection Tasks (19 Datasets Across 3 Task Types)

The validation suite evaluates models on three qualitatively different task types, each a proxy for a real-world use case:

Task Type 1 — Pretrained classifier performance: How well does the model's built-in species classifier work out-of-the-box on unseen data? Evaluated on Powdermill (Denton et al., 2022) and Caples (Denton et al., 2022), two fully-annotated bird soundscape datasets. The metric is ROC-AUC, computed by making predictions on every 5-second window with a 2.5-second stride (50% overlap) and counting a prediction as correct when it overlaps with a ground-truth annotation. This task validates that the model has not overfit to its training distribution and can handle the domain shift from focal recordings (single species, close microphone) to soundscapes (multiple species, varying distances, background noise).

Task Type 2 — One-shot retrieval: Given a single example of a species, can the model's embeddings retrieve other examples of the same species from a database? Evaluated on BEANS detection datasets and the Weldy calltype dataset (Weldy et al., 2024), plus Powdermill and Caples. The procedure: select a random example, compute cosine distances to all other embeddings, rank by similarity, and compute ROC-AUC treating same-species examples as positives and different-species as negatives. This task is a proxy for nearest-neighbor search and clustering workflows — if embeddings of the same species cluster tightly in cosine space, practitioners can use similarity search to find more examples of a target species from a small set of query examples.

Task Type 3 — Linear transfer learning: How well do the embeddings support training a simple linear classifier with very few examples? Evaluated on BEANS classification tasks, datasets from Ghani et al. (2023), and marine datasets (DCLDE, NOAA PIPAN, ReefSet). The procedure: randomly select 16 examples per class, embed each (by averaging embeddings of all 5-second windows with a 5-second stride), train a linear classifier using scikit-learn's LogisticRegression for 10,000 steps, and evaluate ROC-AUC on all remaining examples. The 16-shot setting is deliberately low-resource, reflecting realistic deployment where practitioners may have only a handful of labeled examples for a novel species or call type.

Aggregation metric: For each task type, a geometric mean is computed across all relevant datasets to produce a single task-performance score. Then an overall model quality score is the geometric mean of the three task-performance scores:

$\text{score} = \left(\prod_{t=1}^{3} \left(\prod_{d=1}^{D_t} \text{AUC}_{t,d}\right)^{1/D_t}\right)^{1/3}$

The geometric mean is chosen because it "will tend to favor models with less variance in the scores" (van Merriënboer et al., 2024). Unlike an arithmetic mean, where a single very high score can compensate for a very low score, the geometric mean penalizes uneven performance — a model that scores 0.9 on birds but 0.3 on marine transfer will have a lower geometric mean than one that scores 0.7 on both.

Evaluation Tasks (BirdSet and BEANS Benchmarks)

BirdSet (Rauch et al., 2025b): Six fully-annotated soundscape datasets from diverse geographic regions (continental US, Hawai'i, Peru, Colombia) with training sets provided for fine-tuning. Perch 2.0 is evaluated without fine-tuning — the prototype classifier's predictions are used directly. This is a stringent test of the model's pretrained species classification capability under domain shift. The reported metrics are mean class-mean average precision (cmAP), AUROC, and top-1 accuracy.

BEANS (Hagiwara et al., 2023): Twelve cross-taxa tasks spanning birds, land mammals, marine mammals, anurans, and insects. Each dataset has predefined train/validation/test splits. Perch 2.0 embeddings are extracted frozen, and two types of probes are trained on the BEANS training split:

• Linear probe (LP): A linear classifier trained on the 1536-dimensional mean embeddings, identical to the linear transfer evaluation but using the provided training split rather than a small random subset.
• Prototypical probe (PP): The same prototype classifier architecture described in Section 2.2, trained on the spatial embeddings $E_S$ rather than $E_A$. This leverages the prototype mechanism for the downstream task, potentially capturing spatial patterns that the mean embedding loses.

The paper notes that "prototypical probing seems to improve the performance on detection tasks," suggesting that the spatial information preserved in $E_S$ is valuable for detecting short-duration events within longer audio windows, where the mean embedding's averaging might dilute the signal.

Marine transfer learning comparison (Appendix C.2): A separate evaluation compares Perch 2.0 against specialized marine models (SurfPerch and Google Multispecies Whale Model) on three marine datasets: NOAA PIPAN, ReefSet, and DCLDE 2026 (with label sets for species, ecotype, and known species). The protocol uses 16-shot linear transfer learning as described above. This evaluation is particularly important because it tests the claim that avian-dominated training data transfers to a radically different acoustic domain — underwater hydrophone recordings — which have different sound propagation physics, different background noise profiles, and different vocalization structures (whale songs vs. bird songs).

Cross-Validation and Statistical Rigor

The paper does not describe explicit cross-validation for the evaluation benchmarks, but the model selection (validation) procedure uses the geometric mean aggregation to penalize outlier performance, and the separate evaluation benchmarks (BirdSet, BEANS) are held out from validation entirely. The paper also does not report confidence intervals or statistical significance tests, relying instead on the breadth of evaluation tasks (19 validation + 18 evaluation = 37 total dataset-task combinations) to demonstrate consistent improvement.

---

Summary of Key Design Choices

• Supervised over self-supervised: The paper's core bet is that fine-grained species labels provide a stronger training signal than self-supervised objectives, and that this advantage persists even when transferring to taxa outside the training distribution. This choice is defended empirically (state-of-the-art on both benchmarks) and theoretically (label granularity hypothesis, Appendix C.1).

• Frozen embeddings over fine-tuning: By evaluating all downstream tasks without updating the embedding model weights, the paper demonstrates that Perch 2.0's representations are inherently linearly separable. This is a deliberate choice aligned with practitioner constraints — frozen embeddings can be pre-computed, stored efficiently, and reused across multiple downstream tasks without requiring GPU access for fine-tuning.

• Multi-component mixup with multi-hot targets: Generalizing mixup beyond two components and using multi-hot rather than weighted-average targets addresses the bioacoustics-specific requirement that multiple species should be recognized simultaneously regardless of relative volume.

• Prototype self-distillation with stop-gradient: The prototype classifier serves as a teacher that produces soft targets capturing inter-class similarities, while the stop-gradient prevents the teacher from co-adapting with the embedding model. The two-phase training ensures the teacher has meaningful knowledge before distillation begins.

• Source prediction as recording-level fine-grained supervision: This auxiliary task extends the fine-grained supervision philosophy to an extreme (1.5M classes), forcing the model to learn subtle acoustic features that complement species-level features.

• EfficientNet-B3 for practitioner deployability: The 12M-parameter count was chosen deliberately to balance representation capacity with computational requirements suitable for consumer hardware.

• Model selection reflecting real-world use: The three-task-type validation suite (pretrained classification, one-shot retrieval, linear transfer) was designed to predict real-world deployment performance rather than optimize for a single benchmark metric, and the geometric mean aggregation penalizes uneven performance across tasks.

4. Key Insights and Innovations

Innovation 1: The "Bittern Lesson" as a Conceptual Framework — Supervised Learning Dominates Because Label Granularity Is the True Source of Transferable Representations, Not Data Scale or Self-Supervision

The paper's most intellectually distinctive contribution is not a new architecture or training trick, but a diagnostic reframing of why supervised learning stubbornly outperforms self-supervised methods in bioacoustics. The field has been operating under the implicit assumption, imported from computer vision and NLP, that self-supervision on unlabeled data should eventually surpass supervision if we just find the right augmentations or scale up the data. Perch 2.0 challenges this assumption at its root. The "Bittern Lesson" — the authors' riff on Sutton's "Bitter Lesson" — is that "simple, supervised models are difficult to beat" in this domain, and the paper provides a coherent framework for understanding why this is not just a temporary state of affairs but may reflect something fundamental about the problem structure.

Prior work had already observed that supervised models outperform self-supervised ones in bioacoustics (Ghani et al., 2023; Kather et al., 2025), but these observations were treated as empirical contingencies — perhaps the self-supervised models weren't trained long enough, or the augmentations weren't right, or the datasets were too small. The paper acknowledges these possibilities (Section 4.1 discusses data scale hypotheses and augmentation sensitivity) but goes further by proposing a label granularity hypothesis that elevates the discussion from "what works" to "why it works."

The core claim, supported by the systematic granularity ablation in Appendix C.1 (Table 7), is that fine-grained species labels provide an inductive bias that coarse self-supervised objectives cannot replicate. When the model must distinguish between two closely related warbler species rather than merely between birds and mammals, it is forced to attend to the subtle acoustic features — fine spectral structure, precise temporal patterning, specific frequency modulation contours — that actually define species identity. These same features turn out to be the ones that transfer across taxa, because they capture universal properties of animal sound production (Elemans et al., 2015). Coarse labels allow the model to succeed using superficial features (e.g., "high-pitched = bird, low-pitched = mammal"), which don't transfer. Self-supervised objectives, at least as currently designed, provide no explicit pressure to discover these fine-grained distinctions.

This is a fundamental conceptual shift rather than an incremental refinement. It reframes the supervised-vs-self-supervised debate from a question of optimization (which algorithm learns better representations?) to a question of task design (which pre-training task provides the most transferable inductive bias?). It also explains the otherwise puzzling success of avian-dominated training data transferring to non-avian taxa: birdsong spans an enormous acoustic range (Kroodsma, 2004), and forcing the model to discriminate among thousands of bird species essentially covers the space of biologically relevant acoustic variation. The 14,597 species labels are not just more data — they are a curriculum of increasing discriminative difficulty that self-supervised objectives, with their focus on reconstruction or contrastive learning over short time windows, cannot easily replicate.

The innovation's power is that it is falsifiable and generative. It predicts that self-supervised methods would catch up if they could be designed to operate at a similarly fine granularity — perhaps through extremely fine-grained clustering objectives or through augmentations that explicitly target the acoustic dimensions that distinguish species. The source prediction task (DIET), which the paper frames as "an extremely fine-grained supervised classification problem" with 1.5M recording-level classes, can be seen as a step in this direction — a self-supervised objective that accidentally works because it mimics the fine-grained structure of species classification. This suggests a research program: design self-supervised objectives that explicitly target fine-grained discrimination, rather than continuing to scale up objectives designed for vision or speech.

---

Innovation 2: Self-Distillation via Stop-Gradient Prototype Learning as a Mechanism for Capturing and Transferring Inter-Class Similarity Structure, Not Just Improving Accuracy

The self-distillation mechanism introduced in Perch 2.0 is not merely a performance booster — it represents a conceptual innovation in how to structure knowledge transfer within a single model for the specific challenge of fine-grained classification with taxonomic structure. Standard self-distillation (Allen-Zhu and Li, 2022) uses a teacher and student with identical architectures; the benefit comes from the teacher's soft targets providing richer gradients than hard labels. Perch 2.0 introduces a crucial asymmetry: the teacher (prototype classifier) and student (linear classifier) have fundamentally different inductive biases, and the stop-gradient ensures the teacher's inductive bias shapes the student's representations without the teacher itself adapting to those representations.

Why does this matter? The prototype classifier operates on the spatial embedding $E_S$ and learns to match learned prototypes to specific time-frequency regions. This forces it to discover interpretable, localized acoustic patterns that characterize each species — a particular harmonic stack, a specific trill rate, a characteristic frequency sweep. The linear classifier operates on the spatially-averaged embedding $E_A$ and learns a simple linear decision boundary. Without self-distillation, the linear classifier would learn whatever boundary best separates the training classes, potentially relying on easy-to-discriminate but non-transferable features (e.g., background noise profiles correlated with species because certain species were recorded in certain locations). The prototype classifier's soft targets inject knowledge about which classes are confusable — if the prototype classifier assigns probability 0.15 to a closely related species alongside probability 0.80 to the correct species, the linear classifier learns that misclassifying between these two species is less wrong than misclassifying across genera. This is dark knowledge, but with a crucial twist: it's dark knowledge about taxonomic and acoustic similarity that the prototype classifier discovered by analyzing spatial patterns, not just statistical co-occurrence patterns that a standard teacher would capture.

The stop-gradient is the mechanism that makes this work rather than collapse into triviality. If the prototype classifier's gradients flowed back to the embedding model, both classifiers would co-adapt — the embedding model would reshape its representations to make prototypes easy to match, and the prototypes would adjust to whatever representations the embedding model produced, creating a feedback loop that could converge to a degenerate solution (e.g., the embedding model dedicates a specific channel to each prototype, making the matching trivial but the representations useless). The stop-gradient breaks this loop: the embedding model is shaped only by the linear classifier and source prediction losses, while the prototype classifier must learn meaningful prototypes within whatever embedding space those losses produce. This is analogous to having an independent evaluator who observes the model's representations and provides feedback without being able to influence what features the model learns to represent in the first place.

This is a fundamentally different approach from prior work on prototype learning in bioacoustics (AudioProtoPNet; Heinrich et al., 2025), where the prototype classifier was the primary classification head and its gradients did shape the embedding model. The innovation is inverting the relationship: the prototype classifier is no longer the primary output mechanism but a training-time teacher whose architectural bias (spatial matching) complements the student's bias (global averaging), and whose predictions encode taxonomic similarity information that hard labels cannot provide. The finding that prototypical probing outperforms linear probing on BEANS detection tasks (Table 6) further validates that the spatial information preserved in the prototype mechanism is genuinely useful for certain downstream tasks, suggesting the teacher is imparting knowledge not just about which classes are similar but about what kinds of acoustic patterns matter.

---

Innovation 3: Source Prediction as a Bridge Between Supervised and Self-Supervised Learning — Reframing Self-Supervision as Extreme Fine-Grained Classification Rather Than a Separate Paradigm

The paper's treatment of source prediction (DIET; Balestriero, 2023) represents a conceptual reframing of the relationship between supervised and self-supervised learning that is more interesting than the technique itself. The standard narrative in machine learning positions self-supervision and supervision as distinct paradigms: one learns from unlabeled data through clever pretext tasks, the other learns from human-provided labels. Perch 2.0 blurs this boundary by noting that source prediction "can just as well be seen as an extremely fine-grained supervised classification problem." With 1.5 million recording-level classes, the source prediction task is supervised learning at a granularity so extreme that it becomes indistinguishable from self-supervision — each "class" is just a unique recording identity, requiring no human annotation.

This reframing is intellectually generative because it suggests a unified view: the effectiveness of any pre-training task for transfer learning depends on its granularity and discriminative difficulty, not on whether the labels come from humans or from data structure. The 14,597 species labels are human-annotated but function as a medium-granularity curriculum. The 1.5 million source labels are automatically generated but function as an ultra-fine-grained curriculum. Both force the model to attend to subtle acoustic differences, just at different scales. This explains why source prediction helps during phase 1 training (weight 0.11) but is zeroed out during phase 2 distillation — by the fine-tuning stage, the model has already extracted what it can from the recording-level discrimination task, and further source-specific features might interfere with the species-general representations needed for transfer.

This perspective has implications beyond bioacoustics. It suggests that the long-standing supervised-vs-self-supervised debate may be misframed. Rather than asking which paradigm is better, researchers might ask: what is the optimal granularity of pre-training labels for a given downstream task? If human labels at species granularity already capture the relevant inductive bias, self-supervision adds nothing. But if human labels are too coarse (e.g., "animal" vs. "non-animal"), then self-supervised objectives that provide finer discriminative pressure might help. This reframing is incremental in mechanism (the source prediction loss is standard cross-entropy) but fundamental in conception — it dissolves the boundary between paradigms and replaces it with a continuous notion of label granularity as the key variable.

The connection to individual identification (Lapp et al., 2025) reinforces this interpretation. Individual animal identification is a task at even finer granularity than species classification — the model must distinguish between different individuals of the same species. Source prediction, which trains the model to distinguish between recordings that may contain the same species but were made at different times and locations, naturally pushes the model toward features that are useful for individual discrimination (recording-specific acoustic signatures, background noise profiles, subtle vocalization differences). The paper doesn't develop this connection fully, but it's implicit: source prediction may be transferring to individual identification tasks in the same way that species classification transfers across taxa — by forcing the model to learn features at a level of granularity that generalizes downward to more specific tasks.

---

Innovation 4: The Practitioner-Centric Evaluation Philosophy — Optimizing for Real-World Deployment Patterns Rather Than Benchmark Metrics as a Conscious Methodological Choice

While not a technical innovation in the traditional sense, the paper's evaluation philosophy represents a methodological contribution that is sufficiently distinctive and well-articulated to merit recognition. Most machine learning papers optimize for benchmark performance and treat deployment considerations as an afterthought. Perch 2.0 inverts this: the entire model design, training procedure, and evaluation strategy are organized around the constraints and workflows of the target users — field biologists and conservation practitioners.

The key insight is that the evaluation should mirror the deployment pattern, not just measure accuracy. The three-task-type validation suite (pretrained classification, one-shot retrieval, linear transfer with only 16 examples per class) was specifically designed because these are the actual workflows practitioners use: out-of-the-box classification when the species is known, similarity search to find more examples of a novel sound, and few-shot linear probing when a small labeled dataset exists. The geometric mean aggregation across tasks explicitly penalizes models that excel at one task type but fail at others, encoding the practical requirement that a useful model must be consistently good across diverse deployment scenarios.

This is fundamentally different from the standard benchmark-driven approach exemplified by BirdSet and BEANS themselves. Those benchmarks provide standardized train/test splits and reward models that can be fine-tuned on the training data. Perch 2.0 deliberately forgoes fine-tuning entirely — all results are with frozen embeddings — because fine-tuning requires computational resources and machine learning expertise that practitioners often lack. The paper is essentially arguing that the field's evaluation norms are misaligned with deployment realities, and demonstrating that a model optimized for practitioner workflows can still achieve state-of-the-art on those same benchmarks without playing by their rules (i.e., without fine-tuning). When Perch 2.0 achieves the top AUROC on BirdSet and top scores on BEANS using only frozen embeddings and linear probes (Table 3), it simultaneously validates the model and critiques the benchmarks — if frozen embeddings can win, then the benchmarks' fine-tuning tracks may not be measuring what practitioners actually need.

The marine transfer learning comparison (Appendix C.2, Table 8) is the strongest illustration of this philosophy. Specialized marine models (SurfPerch, Google Multispecies Whale Model) were trained on marine data and achieve strong in-distribution performance. But on few-shot transfer to unseen marine tasks, Perch 2.0's frozen embeddings match or outperform them (0.981 vs. 0.986 on ReefSet where SurfPerch was trained on the data; 0.981 vs. 0.954 on DCLDE species classification where the whale model was not). This would be invisible if evaluation only measured in-distribution classification accuracy — it's the transfer learning evaluation that reveals the advantage of general-purpose embeddings over specialized classifiers.

This innovation is conceptual and methodological rather than algorithmic. It doesn't introduce new metrics or evaluation procedures — scikit-learn logistic regression and ROC-AUC are standard tools. But it shifts the burden of proof: rather than asking "does our model beat the benchmark?", it asks "does the benchmark measure what practitioners need, and can we design a model that serves those needs while still performing competitively?" This is a quiet but significant challenge to how the bioacoustics ML community evaluates progress.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper evaluates on two standard bioacoustics benchmarks: BirdSet (Rauch et al., 2025b), consisting of six fully-annotated soundscape datasets from the continental US, Hawai'i, Peru, and Colombia; and BEANS (Hagiwara et al., 2023), consisting of twelve cross-taxa tasks spanning birds, land and marine mammals, anurans, and insects. Additionally, marine transfer learning is evaluated on three separate datasets: NOAA PIPAN (NOAA Pacific Islands Fisheries Science Center, 2021), ReefSet (Williams et al., 2025), and DCLDE 2026 (Palmer et al., 2025). Each benchmark provides its own train/validation/test splits; the paper uses these as-is for fairness with published baselines.

• Base model(s). Perch 2.0 uses EfficientNet-B3 (Tan and Le, 2019) as the embedding model, with ~12 million parameters. This was chosen to be "small by modern machine learning standards" to enable practitioner deployment on consumer hardware while still providing sufficient capacity for the expanded 1.5M-example training set. The original Perch 1.0 used EfficientNet-B1 (7.8M parameters); the increase to B3 reflects the larger training data volume.

• Metrics. For BirdSet, the paper reports mean class-mean average precision (cmAP), AUROC, and top-1 accuracy, computed by making predictions on every 5-second window with a 2.5-second stride and counting a prediction as correct when it overlaps with a ground-truth annotation. For BEANS, mean accuracy is reported on classification transfer tasks and mean macro-averaged average precision (mAP) on detection tasks. For marine transfer learning (Appendix C.2), ROC-AUC is computed from 16-shot linear probing. The paper states that "the ROC-AUC score is the most stable and informative" among BirdSet metrics (Section 3), citing van Merriënboer et al. (2024).

• Baselines. Comparison models include: BirdNET (Kahl et al., 2021), Perch 1.0 (Hamer et al., 2023), BirdMAE-L (Rauch et al., 2025a), AVES (Hagiwara, 2023), BioLingual (not cited in detail but listed in Table 3), SurfPerch (Williams et al., 2025), and Google's Multispecies Whale Model (Harvey et al., 2024; Allen et al., 2024). All baseline scores are taken directly from published results where available; for BirdSet the paper notes that "BirdSet scores reported for BirdMAE-L are calculated using the scores on the High Sierras Nevada (HSN) subtask, even though this dataset was used by BirdMAE-L for hyperparameter tuning" (Table 3 footnote), flagging a potential fairness concern.

• Generation budget / compute accounting. There is no generation budget in the LLM sense — Perch 2.0 processes each 5-second audio window once through the embedding model. Compute is measured in model parameters (12M for Perch 2.0) and inference cost (a single forward pass per 5-second window). For hyperparameter search, training cost is reported as "between 20 and 30 hours on a TPUv3-8" per model, with approximately 400 total models trained across both phases and both window-selection variants. The paper emphasizes that frozen embeddings are computed once and reused, making the amortized inference cost per downstream task negligible compared to fine-tuning approaches.

• Cross-validation / statistical protocol. The model selection (validation) procedure uses 19 datasets across three task types, aggregated via geometric mean to penalize uneven performance. However, the paper does NOT report confidence intervals, standard deviations, or statistical significance tests for any result. The evaluation benchmarks (BirdSet, BEANS) use their standard train/test splits with no cross-validation. For the marine transfer comparison (Appendix C.2), the 16-shot linear probing uses random selection of training examples; no mention is made of repeated trials or variance estimates. The label granularity ablation (Appendix C.1, Table 7) reports single-point ROC-AUC values with no error bars. This represents a statistical reporting gap: with only 500 BirdSet test examples and ~100 per difficulty bin equivalent, the absence of variance estimates makes it difficult to assess whether observed differences (e.g., Perch 2.0's 0.901 AUROC vs. BirdMAE-L's unreported AUROC on BirdSet) are statistically reliable.

---

Main Quantitative Results

Benchmark Results: BirdSet and BEANS

The headline result is presented in Table 3: Perch 2.0 achieves state-of-the-art performance on both BirdSet and BEANS without any fine-tuning of the embedding model. All Perch 2.0 results use frozen embeddings with either pretrained classification heads applied directly ('Pre'), linear probes ('LP'), or prototypical probes ('PP'). This contrasts with several baselines that use model fine-tuning ('FT').

BirdSet results (Table 3, Table 4 per-dataset breakdown):

On BirdSet, Perch 2.0 with pretrained classifier achieves a mean AUROC of 0.901 across all six soundscape datasets. The paper does not provide a single comparison AUROC for BirdMAE-L (the previous state-of-the-art when fine-tuned), but the per-dataset breakdown in Table 4 shows BirdMAE-L achieving higher cmAP on several individual datasets (e.g., 0.461 on POW vs. Perch 2.0's 0.658 on POW — wait, this comparison requires careful reading of Table 4 which reports cmAP, not AUROC). The paper states that "on this metric [AUROC], as well as on the BEANS tasks, Perch 2.0 achieves state-of-the-art performance" (Section 3).

Looking at Table 4's per-dataset BirdSet results more carefully: Perch 2.0 achieves cmAP of 0.658 on POW (Powdermill), 0.311 on SNE (Sierra Nevada), 0.162 on SSW (Sierra South West), 0.611 on PER (Peru), 0.420 on UHH (Hawai'i), and 0.248 on SCS (Colombia coffee farms). The BirdMAE-L (fine-tuned) comparison shows: 0.461 on POW, 0.351 on SNE, 0.172 on SSW, 0.725 on PER, 0.296 on UHH, 0.420 on SCS. The pattern is inconsistent — Perch 2.0 leads on some datasets (POW: 0.658 vs. 0.461) while BirdMAE-L leads on others (PER: 0.725 vs. 0.611, SCS: 0.420 vs. 0.248). This inconsistency across datasets is noteworthy but the paper does not analyze it in depth.

The top-1 accuracy and AUROC columns in Table 3 show Perch 2.0 achieving 0.901 AUROC vs. BirdMAE-L's dash (—) indicating unreported scores, making direct comparison impossible on this metric. BirdNET's pretrained AUROC is reported as 0.812, and Perch 1.0's at 0.793, showing clear improvement. AVES (fine-tuned) shows AUROC of 0.865 — Perch 2.0's frozen embeddings outperform a fine-tuned self-supervised model by 0.036 AUROC points.

BEANS results (Table 3, Tables 5–6 per-dataset breakdown):

On BEANS classification tasks (Table 5), Perch 2.0 with linear probing achieves a mean accuracy of 0.719. On BEANS detection tasks (Table 6), Perch 2.0 with linear probing achieves a mean mAP of 0.568, which improves to 0.592 with prototypical probing. The paper notes that "prototypical probing seems to improve the performance on detection tasks" (Section 3), suggesting that spatial information in $E_S$ matters more for detection (localizing short sound events within 5-second windows) than for classification.

Baseline comparisons on BEANS (Table 3): BirdNET achieves 0.697 accuracy and 0.419 mAP with pretrained heads; BioLingual achieves 0.484 accuracy (zero-shot) and dash for mAP; Perch 1.0 achieves 0.649 accuracy and 0.499 mAP. Perch 2.0's superiority over Perch 1.0 is consistent across both metrics (0.719 vs. 0.649 accuracy; 0.592 vs. 0.499 mAP), validating the architectural and data improvements.

The frozen-embedding advantage: A critical pattern across Table 3 is that Perch 2.0's results marked 'Pre' (pretrained head) or 'LP'/'PP' (linear/prototypical probe) match or exceed baselines marked 'FT' (fine-tuned). For example, BirdMAE-L uses full model fine-tuning on BirdSet but Perch 2.0's frozen pretrained head achieves "state-of-the-art" AUROC. This is the practical headline: practitioners can use Perch 2.0 embeddings without fine-tuning and get performance competitive with or exceeding fine-tuned specialized models.

Marine Transfer Learning: Outperforming Specialized Models

Appendix C.2 (Table 8) presents what is arguably the paper's most striking single result: Perch 2.0's frozen embeddings, when used for 16-shot linear transfer learning, outperform specialized marine models on cetacean classification tasks despite having "almost no marine training data" (only "a few dozen cetacean recordings... mostly phone recordings made above water").

DCLDE 2026 species classification (16-shot linear probing, ROC-AUC):

• Perch 2.0: 0.981
• Perch 1.0: 0.977
• Google Multispecies Whale Model (embeddings): 0.954
• Google Multispecies Whale Model (direct logits): 0.612
• SurfPerch: 0.881

Perch 2.0 exceeds the whale-specific model's embedding-based transfer by 0.027 ROC-AUC, and the logit-based direct prediction by 0.369 — a dramatic gap that underscores the paper's argument about the value of general-purpose embeddings over specialized classifiers. The whale model's direct logits perform poorly (0.612) because the specific DCLDE label set differs from its training classes, but its embeddings transfer reasonably well (0.954) — yet still trail Perch 2.0.

DCLDE 2026 ecotype classification (orca sub-populations):

• Perch 2.0: 0.976
• Perch 1.0: 0.946
• SurfPerch: 0.778
• Google Multispecies Whale Model (embeddings): 0.859

This is a particularly notable result because orca ecotype discrimination is a fine-grained task requiring the model to distinguish subtle dialect differences between orca populations — precisely the kind of task that the label granularity hypothesis (Section 4.1) predicts should benefit from avian species classification pre-training. Perch 2.0's 0.976 substantially exceeds Perch 1.0's 0.946 and the whale model's 0.859, suggesting the multi-taxa expansion and new training objectives improved transfer to this fine-grained marine task.

NOAA PIPAN (baleen whale species + anthropogenic noise):

• Perch 2.0: 0.937
• Google Multispecies Whale Model (embeddings): 0.930
• SurfPerch: 0.796

Again, Perch 2.0 edges out the whale-specific model despite the whale model having been trained on "a large portion of the NOAA PIPAN labeled audio data" (Appendix C.2). The paper explicitly notes this training data overlap as a caveat but frames the result as Perch 2.0 achieving competitive performance on a task where the baseline has a home-field advantage.

ReefSet (reef sounds, fish vocalizations, anthropogenic noise):

• SurfPerch: 0.986
• Perch 2.0: 0.981
• Perch 1.0: 0.974

This is the one marine task where a specialized model (SurfPerch, which "was trained on the ReefSet data") outperforms Perch 2.0, and the margin is narrow (0.005). The paper frames this as Perch 2.0 being essentially competitive despite SurfPerch's training data advantage.

Important caveat on comparison methodology (Appendix C.2): The paper notes that "the scores in Table 8 might differ slightly from previously reported results in the literature due to differences in the linear probe estimation implementation." Specifically, Williams et al. (2025) evaluated SurfPerch using mini-batch training with Adam optimizer, while this paper uses scikit-learn's LogisticRegression with L-BFGS optimizer and weight decay. This implementation difference means the marine comparisons are not perfectly controlled against published SurfPerch numbers, though the within-table comparisons (all using the same scikit-learn protocol) are internally consistent.

Random Windows vs. Energy Peak Selection: A Surprising Null Result

The paper reports a finding that contradicts its own prior work (Perch 1.0): "Contrary to our experience with Perch 1.0 and other results in the literature, we find that training with random windows performs on par with using energy peak selection" (Section 3). This is not shown in a dedicated ablation table — the paper states it as an aggregate finding across the model selection procedure, noting that the best models from both window selection strategies achieved similar validation scores. Table 6 shows per-dataset BEANS detection results for both "Peak" and "Rand" distilled models, where the differences are small and inconsistent in direction. For example, on the "Dogs" detection task, Rand achieves 0.898 mAP vs. Peak's 0.842; on "Bats" detection, Peak achieves 0.807 vs. Rand's 0.735. No clear winner emerges.

The paper hypothesizes that "the self-distillation phase helps address some of the issues with label noise, negating the need for window selection methods." This is a plausible but untested hypothesis — there is no ablation comparing random-window models with and without self-distillation to verify that self-distillation is the mechanism that closes the gap.

Label Granularity Ablation: Coarser Labels Degrade Transfer

Appendix C.1 (Table 7) provides direct experimental support for the paper's central hypothesis about label granularity. The experiment trains a model on Xeno-Canto only (without prototype classifier or self-distillation, similar to Perch 1.0 but with more data), coarsening species labels to genus, family, order, or class level, and evaluates transfer learning performance (ROC-AUC) across the model-selection tasks.

Results (Table 7):

The paper reports "steady degradation of transfer learning performance as the labels are made more granular" — but careful reading reveals they mean degradation as labels are made coarser, moving from species → genus → family → order → class. This is a terminology issue: "more granular" in common usage means finer-grained (more classes), while "more coarsened" means fewer classes. The finding is that species-level labels (most fine-grained) produce the best transfer; class-level labels (birds vs. mammals vs. amphibians — only a few classes) produce the worst.

Specific numbers from Table 7: The species-level model achieves some baseline ROC-AUC (exact numbers not quoted in the main text; the table shows three task-type scores with geometric mean aggregation). As labels are coarsened through genus, family, order, to class level, the geometric mean transfer score decreases monotonically. The paper also notes that "the new model trained on only Xeno-Canto outperforms Perch 1.0, demonstrating the impact of accumulating new supervisory data over time" — a finding that supports the scale hypothesis from Section 4.1.

This ablation is important because it directly tests the causal claim that label granularity (not just having lots of labels) drives transfer performance. However, it's worth noting that coarsening labels also reduces the number of classes (from thousands of species to ~40 orders to ~5 classes), so the effect could be partially due to the classifier having fewer output dimensions rather than the granularity per se. The paper does not disentangle these confounds.

---

Ablation Studies and Robustness Checks

The paper's ablation structure is distributed across the main text, appendices, and hyperparameter search results rather than presented in a centralized ablation table. The key ablations and their findings:

Window selection strategy (random vs. energy peak): As discussed above, no consistent performance difference was found between the two strategies. The final model uses random window selection for its simplicity. This result is robust across the 19-dataset validation suite but is reported qualitatively rather than with a dedicated comparison table.

Self-distillation phase contribution: The two-phase training is an implicit ablation — phase 1 trains without self-distillation, phase 2 adds it. The paper reports that Vizier assigned "high weights to the self-distillation loss, (1.5, 4.5)" during phase 2 (Section 2.4), and the final model uses a weight of 4.22, making it the dominant loss component. However, the paper does not report the phase 1 model's final validation scores vs. phase 2, so the marginal contribution of self-distillation cannot be quantified from the presented data. This is a significant omission — readers cannot determine whether self-distillation provides a 1% improvement or a 20% improvement.

Source prediction loss weight (phase 1 vs. phase 2): The phase 1 optimal weight of 0.11 vs. phase 2 optimal weight of 0.0 is itself an ablation result showing that source prediction helps during early training but becomes unnecessary or counterproductive during distillation. No experiment tests a model trained entirely without source prediction to measure its standalone contribution.

Prototypical vs. linear probing on BEANS: Table 6 provides per-dataset mAP for both linear probes (LP) and prototypical probes (PP). Prototypical probes improve mean mAP from 0.568 to 0.592. The per-dataset pattern is informative: on detection tasks requiring temporal localization (e.g., finding short bat calls within 5-second windows), prototypical probes show larger gains; on classification tasks with cleaner signals, the gap is smaller or reversed. This validates the paper's claim that spatial information in $E_S$ aids detection, though the improvement is modest (+0.024 mean mAP).

Mixup configuration (phase 1 preferences): The hyperparameter search results (Section 2.4) reveal that Vizier preferred models with N ∈ {2, ..., 5} components, α > β (skewing toward more components), and concentration ω around 10–30 (near-uniform mixture weights). This validates the multi-component generalization over standard two-component mixup, though no ablation directly compares multi-component vs. two-component vs. no-mixup models.

Mixup in phase 2 (reduced/eliminated): The finding that Vizier preferred "little or no mixup (mostly N = 1, i.e., no mixing)" during phase 2 is a robustness check suggesting that mixup's regularization is most valuable during initial representation learning and may interfere with the precise soft targets from self-distillation.

Dropout rate (phase 1: 0.49, phase 2: 0.0): The dramatic shift in dropout preference between phases (from heavy regularization to none) indicates that the distillation phase operates in a different optimization regime where stochastic regularization is harmful. This is consistent with self-distillation depending on precise soft targets that dropout would perturb.

Learning rate reduction (200× from phase 1 to phase 2): The learning rate drops from 6.41 × 10⁻⁴ to 3.20 × 10⁻⁶ between phases, confirming that phase 2 is a fine-tuning stage where the embedding model's representations are largely settled and the distillation loss provides refinement rather than fundamental restructuring.

Embedding dimension and model size: The paper does not ablate the 1536-dimensional embedding size or the EfficientNet-B3 scale. Given that Perch 1.0 used B1 (7.8M parameters) with a different embedding dimension, and Perch 2.0 uses B3 (12M), the marginal contribution of the larger model is confounded with all other changes (multi-taxa data, self-distillation, source prediction, multi-component mixup). A model-size ablation would help disentangle whether the gains come from architecture scaling or methodological improvements.

Training data composition (taxa balance): The paper does not ablate the contribution of non-avian taxa to transfer performance. With 89% avian and 11% non-avian training data, it's unclear whether the multi-taxa expansion actually improved transfer to non-avian tasks or whether the gains come primarily from the increased total data volume and improved training objectives. An ablation training on avian-only data with the same total volume and new objectives would be informative but is not presented.

FSD50K contribution: The 40,966 FSD50K recordings include non-bioacoustic sound classes (engines, music, etc.). The paper does not investigate whether these general sound classes help (by providing negative examples and background diversity) or hurt (by consuming model capacity on non-bioacoustic distinctions). An ablation removing FSD50K from training would clarify this.

---

Critical Assessment

Claim: "Perch 2.0 obtains state-of-the-art performance on the BirdSet and BEANS benchmarks"

What the experiments actually demonstrate: Perch 2.0 achieves the highest reported AUROC on BirdSet (0.901) among all models in Table 3, and the highest mean accuracy (0.719) and mAP (0.592 with PP) on BEANS. However, two important qualifications apply:

1. BirdSet AUROC comparison is incomplete. Table 3 shows BirdMAE-L with a dash (—) for AUROC, meaning the previous state-of-the-art's AUROC was not reported. Perch 2.0's "state-of-the-art" claim on AUROC cannot be verified against the strongest baseline on this metric. On cmAP (which BirdMAE-L does report), the per-dataset results in Table 4 show BirdMAE-L leading on three of six datasets (SNE: 0.351 vs. 0.311, PER: 0.725 vs. 0.611, SCS: 0.420 vs. 0.248). So on cmAP — the metric BirdMAE-L was optimized for — the comparison is mixed rather than uniformly favoring Perch 2.0.

2. Perch 2.0 uses a fundamentally different evaluation protocol. Perch 2.0 uses frozen embeddings with pretrained heads or linear probes, while BirdMAE-L, AVES, and BioLingual use fine-tuning. Saying "Perch 2.0 achieves state-of-the-art without fine-tuning" is accurate, but saying "Perch 2.0 achieves state-of-the-art" without qualification overstates — it achieves state-of-the-art under its own evaluation constraints, which are more stringent than some baselines but may be less stringent than others (e.g., the BirdMAE-L hyperparameter tuning on HSN dataset noted in Table 3). A fairer formulation: Perch 2.0 achieves performance competitive with or exceeding fine-tuned models while using only frozen embeddings, which is the practically relevant comparison.

Claim: "It also outperforms specialized marine models on marine transfer learning tasks, despite having almost no marine training data"

What the experiments actually demonstrate: This claim is well-supported for cetacean classification tasks. On DCLDE 2026 species classification (16-shot linear probing), Perch 2.0 achieves 0.981 ROC-AUC vs. Google Multispecies Whale Model's 0.954 (embeddings) and 0.612 (direct logits), and SurfPerch's 0.881. On DCLDE ecotype classification, Perch 2.0's 0.976 exceeds the whale model's 0.859 and SurfPerch's 0.778. These are substantial margins.

However, several caveats apply:

1. The "specialized" models are specialized for classification, not embedding transfer. The Google Multispecies Whale Model was designed for species classification with its own label set; using its embeddings for few-shot transfer is a repurposing. The paper itself notes this: "if using the embeddings from this model for few-shot learning, the performance jumps to 0.954" (from 0.612 for direct logits). This is more a demonstration that general-purpose embedding models (like Perch 2.0) are better suited for transfer learning than specialized classifiers than a demonstration that Perch 2.0 is "better at marine tasks" in an absolute sense.

2. SurfPerch outperforms Perch 2.0 on ReefSet (0.986 vs. 0.981), the one dataset where SurfPerch was trained on the target data. The margin is small (0.005) but directionally favors the specialized model on its home territory.

3. The sample sizes for marine evaluation are not reported. The paper does not state how many test examples exist in DCLDE, NOAA PIPAN, or ReefSet, making it impossible to assess whether the reported ROC-AUC differences are statistically significant or could be noise. With 16 training examples per class and an unknown number of test examples, the variance could be substantial.

4. The linear probe implementation differs from SurfPerch's published evaluation (scikit-learn LogisticRegression vs. Adam-trained linear layer), meaning the SurfPerch scores in Table 8 may not match previously reported SurfPerch performance. The relative comparisons within Table 8 are internally consistent, but the claim of "outperforming" SurfPerch on DCLDE might not hold under SurfPerch's original evaluation protocol.

Claim: "Fine-grained species classification is a particularly robust pre-training task for bioacoustics" (label granularity hypothesis)

What the experiments actually demonstrate: The label granularity ablation in Table 7 provides direct but limited support. Coarsening labels from species to higher taxonomic levels degrades transfer performance monotonically. This establishes a correlation between label granularity and transfer quality. However, the experiment has a critical confound: coarsening labels also drastically reduces the number of output classes (14,597 species → ~40 orders → ~5 classes). The degradation could be due to:

• Loss of fine-grained discriminative pressure (the paper's claimed mechanism)
• Reduced classifier dimensionality (fewer output units means less gradient signal and less representational capacity in the final layer)
• Reduced task difficulty (easier classification problems may not force the model to learn robust features)

Without an experiment that varies granularity while controlling the number of classes (e.g., comparing species-level classification with 100 random species vs. 14,597 species, or artificially splitting a small number of classes into fine-grained subcategories), the causal claim that granularity per se drives transfer performance is not rigorously isolated.

The paper's broader hypothesis about why avian training transfers (universal sound production mechanisms, diversity of birdsong, etc.) is supported by the transfer results themselves but is not experimentally tested — it remains a plausible post-hoc explanation rather than a verified mechanism.

Genuine Weaknesses and Missing Experiments

1. No statistical reporting. Throughout the paper, all results are reported as point estimates without confidence intervals, standard deviations, or significance tests. With test sets as small as 500 examples (BirdSet) and five difficulty-equivalent bins, even a 0.02 ROC-AUC difference may be within sampling error. The 16-shot transfer experiments are particularly vulnerable: with only 16 randomly selected training examples, different random seeds could produce meaningfully different results, yet no variance is reported.

2. Single model architecture family. All experiments use EfficientNet variants (B1 for Perch 1.0, B3 for Perch 2.0). The findings about supervised learning's dominance, label granularity, and the effectiveness of the training objectives are all demonstrated on convolutional architectures. It is unknown whether the same conclusions hold for transformer-based architectures (which have been explored in bioacoustics through models like BirdMAE-L) or for different convolutional families. The paper's claim that "simple, supervised models are difficult to beat" might be architecture-specific rather than universally true.

3. No quantification of self-distillation's marginal contribution. The two-phase training makes self-distillation a core methodological contribution, but the paper never reports phase 1 validation scores to compare against the final phase 2 model. Readers cannot determine whether self-distillation provides a 1%, 5%, or 20% improvement, making it impossible to assess whether the added complexity (prototype classifier, stop-gradient, two-phase schedule) is justified by the gain.

4. Computational cost barriers for reproduction. The hyperparameter search consumed approximately 8,000–12,000 TPUv3-8 hours (400 models × 20–30 hours). This is far beyond typical academic budgets. While the paper reports final hyperparameters (Appendix A.1), the search process itself is not reproducible without similar computational resources, and the reported hyperparameters may be specific to the training data distribution and EfficientNet-B3 architecture.

5. Weak evaluation of the "less label noise" hypothesis for random windows. The paper's finding that random window selection matches energy peak selection is attributed to self-distillation addressing label noise, but this is speculation. No experiment tests whether a random-window model without self-distillation performs worse than one with it. The mechanism behind the null result remains unexplained.

6. No ablation on the 5-second window length. The 5-second input duration is a critical design choice that determines temporal context. Longer windows (10s, 30s) might capture complete vocalizations for species with long songs; shorter windows (2s, 1s) might reduce background noise for species with brief calls. No window-length ablation is presented.

7. Limited negative result transparency. The paper mentions "we too have experimented with a variety of self-supervised methods such as MAEs, HuBERT and SimCLR but experienced a similar inability to consistently outperform supervised models" (Section 1). These negative results are acknowledged but never presented — no table, no figure, no training details. While it's understandable that a paper focusing on supervised methods wouldn't fully detail self-supervised experiments, the complete absence of data makes the claim unverifiable and prevents other researchers from learning from these attempts.

8. Training data leakage concerns for BirdSet. Several BirdSet datasets are drawn from the same geographic regions and recording projects as Xeno-Canto and iNaturalist data. The paper does not describe any deduplication or temporal splitting to ensure that the exact same recordings (or recordings from the same recording session/location) don't appear in both training and evaluation. The strong performance on pretrained classification (without fine-tuning) could partly reflect memorization of specific recordings rather than generalization.

When the Claims Hold vs. When They Don't

The claim that Perch 2.0 achieves state-of-the-art transfer holds most cleanly for the BEANS benchmark, where the comparison set is larger, metrics are complete, and Perch 2.0 leads on both classification and detection tasks by clear margins (0.719 vs. next best 0.697 on accuracy; 0.592 vs. 0.499 on mAP). It holds less cleanly for BirdSet, where the missing BirdMAE-L AUROC and mixed cmAP results make the "state-of-the-art" designation ambiguous.

The claim that supervised learning outperforms self-supervised methods holds given the available evidence: Perch 2.0 (supervised, frozen) achieves 0.901 AUROC on BirdSet while AVES (self-supervised, fine-tuned) achieves 0.865, and BirdMAE-L (self-supervised, fine-tuned) has unreported AUROC but mixed cmAP. However, this comparison is not head-to-head in a controlled setting — the models differ in architecture, training data, and training duration, not just supervision paradigm. The paper has not demonstrated that supervised learning as a paradigm is superior; it has demonstrated that this specific supervised model performs well.

The claim that label granularity drives transfer quality holds conditionally: the monotonic relationship between label coarseness and transfer performance in Table 7 supports the claim, but only for the specific Xeno-Canto training setup without self-distillation. Whether the granularity effect persists with self-distillation, multi-taxa data, and source prediction is untested.

The claim that Perch 2.0 is suitable for practitioner deployment on consumer hardware is plausible but untested empirically in the paper. The model has 12M parameters and uses frozen embeddings, which are favorable properties, but no inference latency benchmarks, memory usage measurements, or hardware specifications are reported. The paper does not demonstrate inference on a laptop or Raspberry Pi, nor does it report embedding throughput (audio-seconds per second of wall-clock time). This claim is aspirational rather than empirically validated.

The claim that frozen embeddings suffice for state-of-the-art performance holds for the tasks evaluated (BirdSet classification, BEANS classification/detection, marine few-shot transfer) but may not generalize to tasks requiring more specialized representations. Tasks involving very short acoustic events, overlapping vocalizations, or species with extremely similar vocalizations might benefit from fine-tuning in ways that the current evaluation suite doesn't capture.

6. Limitations and Trade-offs

6.1 The Difficulty Estimation Cost for New Deployment Scenarios Is Potentially Prohibitive

The assumption or constraint. The entire compute-optimal framework depends on estimating prompt difficulty before allocating the test-time compute budget. The paper's method for doing so requires generating 2,048 samples per question and either checking correctness (oracle bins) or scoring with the PRM (predicted bins). The authors acknowledge this explicitly in Section 3.2:

"estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity"

The consequence. In a realistic deployment, the total cost is difficulty estimation + strategy execution. For a single question, generating 2,048 samples far exceeds the largest test-time budgets studied (256–512 generations). The reported 4× efficiency gains (Figures 4 and 8) are computed after difficulty is known, without amortizing the cost of learning it. If difficulty estimation costs 2,048 generations, and the optimal strategy then uses 64 generations, the true cost is 2,112 generations — roughly 33× the budget of the baseline best-of-64, not 4× less. The claimed efficiency advantage disappears entirely unless difficulty estimation costs can be amortized across many questions sharing the same difficulty distribution.

What evidence exists in the paper. The paper demonstrates that predicted difficulty bins (using PRM scores) track oracle bins closely (Figures 4 and 8, curves largely overlap), showing that ground-truth labels are not needed. However, the paper never measures or reports the cost of generating the 2,048 samples and scoring them, never includes this cost in any budget calculation, and never demonstrates an alternative cheap estimation method. The issue is flagged in Section 3.2 as an "exploration-exploitation tradeoff" but not resolved.

Mitigation status. The paper explicitly flags this as "a key avenue for future work" (Section 3.2) and speculates about training models to predict difficulty directly from the question text. No such model is developed or evaluated. Until a cheap difficulty estimator is demonstrated, the compute-optimal framework is a proof of concept whose practical efficiency gains are unknown.

---

6.2 Hard Problems Remain Essentially Unsolved — Test-Time Compute Cannot Substitute for Missing Capability

The assumption or constraint. The paper's approach assumes that the base model's pass@1 on a given problem is non-trivially above zero — that is, the model already produces correct solutions at some measurable rate. This is stated explicitly in the discussion of difficulty bin 5 results (Section 7 takeaway box):

"Test-time compute can amplify existing capability but cannot create it."

The consequence. Across all methods — search, revisions, and their compute-optimal combinations — the hardest questions (difficulty bin 5) show near-zero improvement regardless of compute budget. In Figure 3 (right), bin 5 accuracy hovers at 1–3% for all methods and all budgets. In Figure 7 (right), bin 5 shows roughly 2–3% accuracy irrespective of the sequential-to-parallel ratio. In the FLOPs-matched comparison (Figure 9), the bin 5 scaling line is essentially flat near 0–5%. The approach provides no path forward for problems that fundamentally exceed the base model's pretraining-derived capabilities. For genuinely novel or out-of-distribution reasoning tasks, pretraining remains the only viable path — test-time compute amplifies what already exists in the model's output distribution but cannot generate novel correct reasoning strategies.

What evidence exists in the paper. The difficulty-bin analyses (Figures 3 right, 7 right, 9) consistently show bin 5 as a flat line near zero. The FLOPs-matched comparison in Figure 9 demonstrates that on hard problems, the ~14× larger model always outperforms test-time compute with the smaller model (e.g., -52.9% relative disadvantage for PRM search on hard problems at R ≫ 1). This is the starkest evidence: pretraining compute addresses the capability gap; test-time compute does not.

Mitigation status. The authors are transparent about this limitation (Section 7), explicitly stating it as a boundary condition. There is no proposed mitigation — the limitation is fundamental. This means the approach is complementary to pretraining scaling, not a replacement, and deployment decisions must consider the difficulty distribution of expected queries.

---

6.3 Single Benchmark and Single Model Family Limit Generalizability of All Findings

The assumption or constraint. All experiments use the MATH benchmark (500 test questions) with PaLM 2-S* as the base model. The authors state they "believe this model is representative of the capabilities of many contemporary LLMs" (Section 4), but this claim is untested. For the FLOPs-matched comparison, the ~14× larger model is a parameter-only-scaled variant (not Chinchilla-optimal), used only with greedy decoding — no test-time compute augmentation of its own.

The consequence. Several aspects of the findings could be model-specific or dataset-specific:

• The PRM's quality and over-optimization behavior depend on PaLM 2-S*'s output distribution. A model with different calibration properties might exhibit different difficulty-dependent scaling curves (e.g., the threshold where beam search starts to over-optimize might shift).
• The revision model's ability to learn from incorrect in-context examples depends on the base model's in-context learning capabilities, which vary substantially across model families.
• MATH consists exclusively of competition-level math problems requiring symbolic reasoning. It is unknown whether the difficulty-dependent patterns (beam search hurting easy problems, revisions helping easy problems) generalize to code generation, logical reasoning, scientific QA, or tasks requiring factual knowledge.
• The ~14× larger model baseline is deliberately weak — parameter-scaled only, greedy decoding only. A Chinchilla-optimal larger model with even a modest test-time compute budget (best-of-8, for instance) would be a much stronger baseline.

What evidence exists in the paper. The paper acknowledges the single-benchmark limitation implicitly by not making claims beyond MATH, but does not discuss how model-specific or dataset-specific the findings might be. No ablation using a different model family or a different benchmark (even a subset of another reasoning dataset) is presented.

Mitigation status. The paper does not address this limitation. The authors' belief that PaLM 2-S* is "representative" is stated as an assumption, not supported by evidence. Replication on additional model families and benchmarks would be necessary to establish the generality of the difficulty-dependent scaling patterns and the 4× efficiency claim.

---

6.4 The Revision Model Has a 38% Correct-to-Incorrect Reversion Rate, Undermining Sequential Refinement

The assumption or constraint. The revision model is trained exclusively on sequences where all in-context answers are incorrect (followed by a correct target). At test time, the model may therefore encounter correct answers in its context during a revision chain and — having never seen correct answers during training — incorrectly "revise" them to wrong answers. The paper reports in Section 6.1:

"approximately 38% of correct answers get converted back to incorrect ones"

The consequence. Sequential revision chains are inherently unstable. Even when the model produces a correct answer at step $t$, there is a ~38% chance that step $t+1$ will revise it to an incorrect answer. This means the revision chain's performance is not monotonic — later steps can be worse than earlier steps — and the system must rely on post-hoc selection mechanisms (majority voting or verifier-based selection across the chain) to recover the correct answer. These selection mechanisms are imperfect: they may fail to identify the correct answer among a chain containing multiple revisions, especially when the verifier itself is unreliable (as demonstrated by the over-optimization findings). The reversion rate also means that blindly increasing chain length is counterproductive beyond some point, limiting the total budget that can be usefully allocated to sequential revisions.

What evidence exists in the paper. The 38% figure is reported in Section 6.1, derived from analysis of revision chains. Figure 6 (left) shows that pass@1 at each revision step gradually improves but plateaus around steps 15–20 at ~24–25%, never exceeding ~25% despite 64 steps — suggesting that gains from additional revisions are offset by reversion errors. The ReST^EM experiment (Appendix K, Figure 16) further demonstrates the fragility: attempting to optimize the revision model with RL-style training caused performance to degrade substantially with sequential revisions, with fully sequential performance dropping to ~33.5% vs. ~38.5% at the optimal ratio.

Mitigation status. The paper mitigates the reversion problem with majority voting and verifier-based selection across the revision chain (Section 6.1), but these are post-hoc patches rather than solutions. A principled fix — such as training the model on sequences that include correct answers followed by correct answers (teaching the model when not to revise) or training a binary "should I revise?" classifier — is not explored. The ReST^EM negative result (Appendix K) suggests that the training data construction for revisions is delicate and not fully understood, and the positive results depend on specific choices (offline data construction, edit-distance-based pairing) that may not transfer to other settings.

---

6.5 Sequential Revision Strategies Introduce Inherent Latency That Is Ignored in the Cost Model

The assumption or constraint. The paper measures test-time compute in "generations" (number of complete solutions sampled), which is a reasonable proxy for total FLOPs. However, this metric ignores wall-clock latency — the time required to produce a final answer. The compute-optimal policies often favor sequential-heavy strategies: on easy problems, purely sequential revisions (one chain of length $N$) are optimal (Figure 7, right); on medium problems, a balanced sequential-to-parallel ratio with significant sequential depth is preferred.

The consequence. Sequential revisions are inherently serial — each revision depends on the output of the previous one. A strategy that allocates 128 generations as 64 sequential × 2 parallel takes roughly 64× longer wall-clock time than one that runs 128 parallel samples simultaneously (assuming sufficient hardware to run all parallel samples concurrently). For latency-sensitive applications — interactive assistants, real-time tutoring, any deployment where the user is waiting for a response — the sequential-heavy strategies favored by the compute-optimal policy may be impractical regardless of their accuracy advantages. A strategy that takes 64 seconds to produce an answer that is 4 percentage points more accurate than one that takes 1 second may be unacceptable in many deployment contexts.

What evidence exists in the paper. The paper never discusses latency, wall-clock time, or throughput. All cost accounting is in terms of generation count. The sequential-to-parallel ratio sweep (Figure 7, left) reports accuracy as a function of generation budget but does not indicate the latency implications of different ratios. The compute-optimal policy selection (Figures 4, 8) optimizes for generation efficiency, not latency efficiency.

Mitigation status. Not addressed. The paper does not acknowledge the latency tradeoff, does not measure wall-clock times, and does not include latency constraints in the compute-optimal formulation. Future work on compute-optimal test-time scaling should incorporate a latency dimension, especially for deployment scenarios where response time matters. In practice, practitioners will need to choose between the generation-optimal policy (from this paper) and a latency-optimal policy (which would favor parallel sampling), depending on their deployment requirements.

---

6.6 The Difficulty Binning Is Static and Discretized, Preventing Mid-Computation Strategy Adaptation

The assumption or constraint. Difficulty is estimated once per question using a fixed procedure (2,048 samples + PRM scoring) and questions are assigned to one of five static quintile bins. The compute-optimal policy is a lookup table: for each bin and budget level, a pre-computed strategy is executed. There is no mechanism for dynamically adjusting strategy mid-computation. The paper acknowledges this in Section 3.2:

"this still requires the additional computational cost of generating 2048 samples and scoring them... We leave investigation of this to future work."

The consequence. Within a single difficulty bin, substantial heterogeneity may exist — a question at the easy end of bin 3 and one at the hard end of bin 3 receive the identical strategy, even though different strategies might be optimal. More importantly, because difficulty estimation and strategy execution are decoupled and sequential, the system cannot use information gathered during strategy execution to refine its difficulty estimate and adjust course. For example, the system cannot start with 8 parallel samples, observe that the PRM scores cluster tightly around a specific answer (suggesting an easy problem), and then switch to sequential revisions for the remaining budget. It must commit to the pre-selected strategy based on a static difficulty estimate.

What evidence exists in the paper. The paper uses five difficulty quintiles (Section 3.2). Figures 3 (right) and 7 (right) show difficulty-dependent behavior at this discretization level. However, the paper does not evaluate sensitivity to the number of bins — would 10 bins yield better allocation? 3 bins? — and does not experiment with dynamic or adaptive policies. The compute-optimal curves (Figures 4, 8) are computed using static bin assignments from cross-validation folds.

Mitigation status. The paper suggests future work on "more sample-efficient difficulty estimation methods" (Section 8) but does not propose a specific dynamic mechanism. An adaptive scheme — e.g., interleaving difficulty estimation and strategy execution by starting with a small number of parallel samples, using the verifier's score distribution as a real-time difficulty signal, and then allocating remaining budget accordingly — could subsume difficulty estimation into the solution process and enable finer-grained allocation. This is flagged as future work but not explored.

7. Implications and Future Directions

How This Work Changes the Landscape

Perch 2.0 does not introduce a brand-new paradigm — it is a supervised convolutional model trained with cross-entropy, a family of techniques that predates the current wave of self-supervised foundation models by several years. Yet the paper's impact on the bioacoustics machine learning landscape is likely to be substantial, not because of what architecture it uses, but because of what it demonstrates about the relationship between supervised pre-training task design and transfer learning quality, and because it provides a strong, practically deployable baseline that resets the performance frontier for general-purpose bioacoustic embedding models.

The shift is best characterized as a reframing with empirical teeth. The paper reframes the supervised-vs-self-supervised debate from a question of algorithm choice (which learning paradigm is better?) to a question of task granularity (what pre-training task provides the most transferable inductive bias?). This is not a mere rhetorical move — it is backed by the label granularity ablation in Appendix C.1 (Table 7), which shows monotonic degradation of transfer performance as labels are coarsened from species → genus → family → order → class. The implication is that fine-grained supervised classification is not just "what we have labels for," but is actively the right thing to do because it forces the model to learn the kind of subtle acoustic features that transfer across taxa and recording conditions.

This reframing has several concrete consequences for how the field should think about pre-training:

1. It partially reconciles the contradiction between bioacoustics and other domains. The paper's discussion in Section 4.1 offers a coherent explanation for why self-supervision dominates in vision (142M images for DINOv2) but lags in bioacoustics (1.5M recordings for Perch 2.0): it is not that self-supervision is fundamentally unsuited to audio, but that the labeled data scale in bioacoustics is already sufficient for supervision to saturate, while the unlabeled data scale is insufficient for self-supervision to catch up. This is a scale hypothesis that reframes the debate as contingent rather than essential — if bioacoustics data were 100× larger, self-supervision might win. The corollary is that efforts to unlock large unlabeled bioacoustic datasets (e.g., from passive acoustic monitoring archives, citizen science projects, or automated recording units) should be prioritized as a research infrastructure investment, because they would enable testing whether self-supervision's underperformance is a data-scale problem or a fundamental one.

2. It redirects attention from architecture innovation to task design. The paper's state-of-the-art results were achieved with EfficientNet-B3, a convolutional architecture from 2019, and standard cross-entropy training. The gains came from how the training task was structured — multi-component mixup, prototype self-distillation, source prediction, multi-taxa label expansion — not from a novel backbone. This suggests that the frontier of bioacoustics model improvement may lie in better task design rather than better architectures, at least for the current data scale. Researchers who have been exploring ViTs, SSMs, or other modern architectures for bioacoustics might productively redirect effort toward designing pre-training objectives that capture taxonomic similarity structure, or toward curating training data with optimal label granularity for the target transfer tasks.

3. It establishes frozen embeddings as the deployment norm to beat. Perch 2.0 achieves state-of-the-art results on BirdSet and BEANS without any fine-tuning — all results use either pretrained classification heads or linear/prototypical probes on frozen embeddings. This is not an incidental feature; it is a deliberate design philosophy aligned with practitioner constraints (limited compute, limited expertise, limited labeled data). The paper effectively raises the bar for future models: to be practically useful, a bioacoustics embedding model should produce representations that are linearly separable under few-shot probing for novel taxa and recording conditions. A model that requires full fine-tuning to match Perch 2.0's frozen-embedding performance is, from a deployment perspective, strictly worse — it requires more compute, more expertise, and more labeled data to achieve the same result. This may shift evaluation norms in the field toward frozen-embedding benchmarks as the primary metric, with fine-tuning treated as an optional enhancement rather than the default evaluation protocol.

4. It demonstrates that cross-taxa transfer is not just possible but can exceed in-domain specialized models. The marine transfer learning results in Appendix C.2 (Table 8) are the paper's most striking single finding: Perch 2.0, trained on 89% avian data with essentially no underwater recordings, matches or outperforms SurfPerch (trained on ReefSet) and Google's Multispecies Whale Model (trained on NOAA PIPAN data) on cetacean classification tasks using only 16-shot linear probing. This is not merely "birdsong embeddings transfer to whales" — that was already shown by Perch 1.0 and Ghani et al. (2023). It is that a generalist supervised model can outperform specialist models on the specialists' own transfer tasks. This has implications for how conservation organizations should invest in model development: rather than training separate models for birds, bats, marine mammals, and terrestrial mammals, a single Perch 2.0 deployment may suffice, with task-specific linear probes trained on small labeled datasets. The resource savings from consolidating multiple specialized models into one general embedding model could be substantial, particularly for organizations monitoring multiple taxa across terrestrial and marine environments.

5. It validates prototype-based self-distillation as a mechanism for capturing inter-class similarity structure. The paper provides evidence that the prototype classifier's soft targets encode taxonomic and acoustic similarity — the "dark knowledge" that makes self-distillation effective. The finding that prototypical probing (using $E_S$ spatial embeddings with prototype matching) outperforms linear probing (using $E_A$ mean embeddings) on detection tasks (Table 6: 0.592 vs. 0.568 mean mAP) further validates that the spatial information preserved in the prototype mechanism is genuinely useful for certain downstream tasks. This opens the door to interpretable transfer learning: the prototype classifier's learned prototypes can be visualized as specific spectro-temporal patterns, and when the model transfers to a new species, researchers could examine which avian prototypes activate for the new species' vocalizations, providing biological insight into acoustic homology across taxa. This is currently possible in principle (the prototype mechanism is interpretable by design) but not demonstrated in the paper; making it practical would be a valuable contribution.

6. It provides negative evidence that certain self-supervised methods are insufficient, redirecting research effort. The paper's candid acknowledgment that "we too have experimented with a variety of self-supervised methods such as MAEs, HuBERT and SimCLR but experienced a similar inability to consistently outperform supervised models" (Section 1) is valuable negative evidence, even if the experiments are not presented in detail. Combined with published negative results from other groups (Kather et al., 2025; Ghani et al., 2023), a pattern is emerging: standard self-supervised recipes imported from vision and speech do not work well for bioacoustics. This does not mean self-supervision is hopeless — the paper explicitly hypothesizes that data scale and augmentation design are the bottlenecks — but it does mean that researchers should not expect off-the-shelf MAE or SimCLR to beat a well-tuned supervised baseline. Effort should shift toward (a) understanding why standard augmentations fail for bioacoustic signals, (b) developing bioacoustics-specific augmentations that preserve species-identifying features, and (c) scaling up unlabeled data collection to test the scale hypothesis.

In terms of magnitude, this is not a paradigm shift — the paper does not introduce a new class of models or a new learning algorithm that renders prior work obsolete. It is a strong consolidation and refinement of the supervised pre-training paradigm, with enough empirical evidence and conceptual framing to make the case that supervised learning's dominance is not a temporary accident but reflects something fundamental about the problem structure. The "Bittern Lesson" is that simplicity plus scale plus task design can still win, and Perch 2.0 is the evidence.

Follow-Up Research This Work Enables

1. Cheap difficulty estimation for compute-optimal allocation of annotation effort. The paper demonstrates that label granularity drives transfer quality (Appendix C.1), but coarsening labels reduces the number of classes. A direct follow-up would ask: given a fixed annotation budget, what is the optimal allocation of labels across the taxonomic hierarchy to maximize transfer learning performance? This is the bioacoustics analog of the compute-optimal test-time scaling problem — instead of allocating inference compute across search strategies, allocate annotation effort across taxonomic granularities. A concrete experiment: starting with 10,000 species labels, systematically coarsen subsets of them to genus, family, or order level while keeping the total number of training examples constant, and measure transfer performance on the BEANS benchmark. The hypothesis from this paper predicts that fine-grained labels matter most for classes that are well-represented in the downstream tasks, while coarse labels may suffice for taxa that appear only as distractors. If validated, this would enable practitioners to prioritize annotation of fine-grained labels for high-priority species while using coarser labels (cheaper to obtain) for background taxa, optimizing the cost-performance tradeoff of training data curation.

2. Dynamics of self-distillation: quantifying the teacher quality threshold and the two-phase schedule's necessity. The paper's two-phase training — first train the prototype classifier, then use it for self-distillation — is motivated by the intuition that distillation only helps if the teacher has meaningful knowledge. But the paper never reports phase 1 validation scores or ablates the phase boundary. A direct experiment: train a series of models that begin self-distillation at different points (after 10K, 50K, 100K, 200K, 300K steps) and measure final transfer performance on the 19-dataset validation suite. The hypothesis: there exists a threshold of teacher quality (measurable as the prototype classifier's validation accuracy) below which self-distillation provides no benefit or even hurts, and above which gains are proportional to teacher quality. If such a threshold exists and can be predicted from training metrics, it would enable automatic scheduling of the distillation phase (start when teacher validation accuracy exceeds X%) rather than the fixed 300K-step transition used in the paper. A negative result — no threshold, distillation helps equally at all points — would suggest the two-phase design is unnecessary complexity and that joint training with distillation from the start is sufficient, simplifying the training recipe.

3. Bioacoustics-specific augmentations for self-supervised learning, informed by the label granularity hypothesis. The paper hypothesizes that self-supervised methods underperform partly because "work remains on finding the right data augmentations to use in bioacoustics" (Section 4.1). The label granularity hypothesis provides a concrete design principle: self-supervised augmentations should be designed to force fine-grained discrimination akin to species-level classification, rather than the coarse discrimination encouraged by standard augmentations (e.g., random cropping in vision forces the model to recognize objects regardless of position, but doesn't force discrimination between similar objects). A concrete experiment: design a contrastive learning objective where positive pairs are different vocalizations of the same species (forcing the model to learn species-invariant features) and negative pairs include vocalizations of closely related species (forcing fine-grained discrimination). This would require a dataset with multiple recordings per species across different individuals, locations, and recording conditions — Xeno-Canto and iNaturalist already provide this. The evaluation would be head-to-head against Perch 2.0 on the BEANS transfer learning benchmark, testing whether a self-supervised model with species-informed augmentations can close the gap with supervised pre-training. A positive result would validate the augmentation hypothesis; a negative result would strengthen the case that supervision per se (not just granularity) is necessary.

4. Stress-testing cross-taxa transfer: at what phylogenetic distance does the label granularity benefit break down? Perch 2.0 transfers from birds to cetaceans successfully (Appendix C.2), but the paper does not systematically explore which taxa benefit from avian pre-training and which do not. A follow-up study would evaluate Perch 2.0 embeddings on a deliberately constructed phylogenetically diverse benchmark: insects (orthopterans, cicadas), anurans (frogs, toads), marine mammals (cetaceans, pinnipeds), terrestrial mammals (primates, carnivores, ungulates), and non-vocal taxa (fish sounds, crustacean stridulation). The hypothesis from the paper's discussion of "universal mechanisms of sound production in terrestrial vertebrates" (Elemans et al., 2015) predicts that transfer should degrade with phylogenetic distance from birds — mammals (shared larynx-based sound production) should benefit more than anurans (different vocal mechanism), which should benefit more than insects (stridulation rather than vocalization). If this gradient exists, it would provide practical guidance: practitioners studying orthopterans might need to collect their own training data, while those studying terrestrial mammals can rely on Perch 2.0 embeddings. If no gradient exists (all taxa benefit equally), it would suggest the transfer mechanism is more about general acoustic feature learning than biological homology, which would strengthen the case for even broader multi-taxa pre-training.

5. Combining self-distillation with iterative pseudo-labeling for semi-supervised expansion to unlabeled data. The paper notes that BirdCLEF competitions have demonstrated strong results with iterative pseudo-labeling (Kahl et al., 2024), and positions self-distillation as an online, within-model variant. A natural extension combines the two: use a Perch 2.0 model to generate pseudo-labels on large unlabeled passive acoustic monitoring datasets, train a new model on the combined labeled + pseudo-labeled data with the same self-distillation protocol, and measure whether transfer performance improves beyond the original Perch 2.0. The key question is whether pseudo-labeled data provides complementary information to self-distillation (capturing species and recording conditions absent from the labeled training set) or redundant information (the teacher model's knowledge is already fully extracted by distillation). A concrete experiment: select a geographic region well-represented in Xeno-Canto (e.g., North America) and one poorly represented (e.g., central Africa), collect unlabeled soundscape recordings from both, apply pseudo-labeling with Perch 2.0, and measure whether the improvement is larger for the poorly represented region. A positive result would enable a virtuous cycle where Perch models are deployed on passive acoustic monitoring data, generate high-quality pseudo-labels for under-represented regions and taxa, and these pseudo-labels are used to train improved models — a self-improving bioacoustic monitoring infrastructure.

6. Inference-time compute scaling: applying test-time augmentation and ensembling to improve Perch 2.0 predictions without retraining. Perch 2.0 processes fixed 5-second windows, but real recordings are of variable length. A simple but unexplored extension applies test-time augmentation: for a recording of length $L$ seconds, extract all possible 5-second windows (or windows with 50% overlap, yielding $\lfloor L/2.5 \rfloor - 1$ windows), embed each independently, and aggregate predictions (e.g., average logits or majority vote). This is the bioacoustics analog of the test-time compute scaling studied in the LLM scaling laws literature — allocating more computation at inference by processing multiple windows rather than a single window. A concrete experiment: on the BirdSet soundscape datasets (which contain recordings of 10 seconds to several minutes), compare single-window prediction accuracy against multi-window aggregation as a function of the number of windows processed, and measure whether the improvement follows a scaling law (e.g., power-law improvement with window count). If multi-window aggregation provides substantial gains (e.g., 5–10 percentage points), it would establish a simple, training-free method for practitioners to improve Perch 2.0's accuracy on longer recordings. If gains are small, it would validate that a single 5-second window already captures sufficient information for species identification, simplifying deployment.

Practical Applications and Downstream Use Cases

1. Few-shot species identification for conservation monitoring. A field biologist deploys passive acoustic recorders in a tropical forest and collects 10,000 hours of audio. They have identified a novel frog call that may belong to an undescribed species and have manually annotated 20 examples. Using Perch 2.0's frozen embeddings, they embed all 10,000 hours (processing each 5-second window at 2.5-second stride), train a linear classifier on their 20 annotated examples (achieving performance similar to the 16-shot results in Appendix C.2, where Perch 2.0 achieved 0.981 ROC-AUC on DCLDE cetacean species classification), and run the classifier across the entire dataset to find all occurrences of the novel call. The total computational cost is: (a) one-time embedding of 10,000 hours (approximately 14.4 million 5-second windows, processable on a single consumer GPU in hours to days depending on hardware), (b) training a 1536-dimensional linear classifier on 20 examples (milliseconds with scikit-learn), and (c) inference via dot product (milliseconds per window). Without Perch 2.0, the biologist would need to either manually scan 10,000 hours (impossible) or train a custom deep learning model from scratch (requiring thousands of labeled examples and ML expertise they lack). The specific benefit is that the 16-shot linear probing protocol used in the paper maps directly to this workflow, and the paper's results on cross-taxa transfer (birds → cetaceans: 0.981 ROC-AUC) suggest the approach will work even for taxa far from the training distribution.

2. Acoustic biodiversity monitoring at scale with a single model. A conservation organization monitors biodiversity across terrestrial, freshwater, and marine habitats using a network of autonomous recording units. Previously, they maintained three separate models: BirdNET for avian species, a custom bat detector, and SurfPerch for reef sounds. With Perch 2.0, they deploy a single embedding model across all recordings (birds, bats removed per the paper's training exclusion, reef sounds, cetaceans, anurans, insects), compute frozen embeddings once, and train per-taxon linear probes on small labeled datasets (16–100 examples per species). The paper's results show that Perch 2.0's frozen embeddings:

• Match BirdNET on avian classification (Table 3: 0.901 vs. 0.812 AUROC on BirdSet, both pretrained; Perch 2.0 linear probe at 0.719 accuracy vs. BirdNET's 0.697 on BEANS classification)
• Match SurfPerch on reef sound classification (Table 8: 0.981 vs. 0.986 ROC-AUC on ReefSet, despite SurfPerch being trained on the data)
• Exceed the Multispecies Whale Model on cetacean transfer (Table 8: 0.981 vs. 0.954 ROC-AUC on DCLDE species)
• Transfer to anurans and insects via the BEANS benchmark (Tables 5–6, though per-taxa breakdowns are not separated in the paper; the aggregate BEANS performance of 0.719 accuracy and 0.592 mAP includes these taxa)

The specific benefit is consolidation: one embedding pass serves all downstream tasks, reducing computational overhead by approximately 3× (one model instead of three) and eliminating the need to maintain and update multiple specialized models as new training data becomes available. When new species are added to the monitoring program, only a new linear probe needs to be trained — no re-embedding and no model retraining.

3. Query-by-example search interfaces for bioacoustic exploration. A researcher studying bird dialects has a single recording of a novel dialect variant and wants to find all similar vocalizations in a large archive. Using Perch 2.0 embeddings, they embed their query recording and the entire archive, then perform cosine similarity search to retrieve the nearest neighbors. The paper's one-shot retrieval evaluation (Section 2.5.1, Task Type 2) was specifically designed to predict performance in this scenario: a random example is selected, cosine distances are computed against all other examples, and ROC-AUC is computed treating same-species examples as positives. The geometric mean aggregation across retrieval tasks during model selection ensures that Perch 2.0 was explicitly optimized for this use case. The specific benefit is that similarity search on frozen embeddings scales to very large archives — once embeddings are computed (a one-time cost), retrieval is $O(D \log N)$ with approximate nearest neighbor indices (where $D = 1536$ is the embedding dimension and $N$ is the archive size). This enables interactive exploration of multi-terabyte audio archives that would be impossible with raw audio comparison or fine-tuning-based approaches.

4. Agile modeling workflows for rapid iteration on novel bioacoustic tasks. The paper explicitly connects to the agile modeling paradigm (Dumoulin et al., 2025) in Section 2.5 and the conclusion (Section 4.2). In an agile modeling workflow, a practitioner starts with a small set of labeled examples, trains a linear probe on Perch 2.0 embeddings, uses the model to find more examples (active learning or confidence-based selection), labels those, retrains, and iterates. The paper's model selection tasks (Section 2.5.1) — pretrained classification, one-shot retrieval, and 16-shot linear transfer — form a coherent pipeline for this workflow: (1) use pretrained classification if the species is in the training set; (2) if not, use one-shot retrieval with a query example to find candidates; (3) label a small set of candidates and train a linear probe to classify the rest. The specific benefit is that each stage of this pipeline was explicitly validated during model selection — Perch 2.0 was optimized to perform well at all three stages simultaneously (via the geometric mean aggregation), meaning practitioners can expect consistent performance as they move from exploration (similarity search) to exploitation (trained linear classifier) without switching models or recomputing embeddings.

When to Prefer This Method

The paper positions Perch 2.0's frozen-embedding approach against two broad alternatives: (a) specialized supervised models trained on target-domain data (e.g., SurfPerch for reef sounds, Google Multispecies Whale Model for cetaceans), and (b) self-supervised bioacoustics models (e.g., BirdMAE-L, AVES). The tradeoffs are explicit enough to warrant a decision framework.

Prefer Perch 2.0 frozen embeddings when:

• You have limited labeled data for your target task (tens to hundreds of examples). The paper demonstrates that 16-shot linear probing on Perch 2.0 embeddings achieves 0.981 ROC-AUC on DCLDE cetacean classification (Appendix C.2, Table 8) and 0.719 mean accuracy on BEANS classification tasks (Table 3). Training a specialized model from scratch with so few examples would be infeasible; fine-tuning a self-supervised model like BirdMAE-L requires more expertise and compute.
• Your target taxa are acoustically dissimilar to birds but you cannot collect sufficient target-domain training data. The paper shows that Perch 2.0's avian-dominated training transfers to cetaceans (0.981 ROC-AUC), anurans (BEANS tasks in Tables 5–6), and insects (BEANS tasks). If you lack the thousands of labeled examples needed to train a domain-specific model, Perch 2.0 provides strong out-of-the-box transfer even for taxa far from the training distribution.
• You need to monitor multiple taxa using a single deployment (e.g., a passive acoustic monitoring grid covering birds, amphibians, and mammals). Perch 2.0's frozen embeddings can be computed once and reused with per-taxon linear probes, avoiding the overhead of maintaining multiple specialized models. The paper's results on BirdSet (6 datasets, multiple geographic regions), BEANS (12 tasks across 5 taxonomic groups), and marine transfer (3 datasets) provide evidence for broad multi-taxa coverage.
• Computational resources at deployment time are constrained (consumer GPU or CPU-only). Perch 2.0 has 12M parameters and requires only a forward pass per 5-second window. Frozen embeddings can be pre-computed for large datasets and stored (1536 floats per window ≈ 6 KB, or ~4.3 GB for 1 million windows), after which downstream tasks require only linear classifier training.
• You value model simplicity and reproducibility over squeezing out the last percentage point of performance. Perch 2.0's architecture (EfficientNet-B3 + linear heads) and training objectives (cross-entropy, self-distillation, source prediction) are well-specified and use standard components. The released model weights enable exact reproduction. Self-supervised models like BirdMAE-L require more complex training pipelines (masking strategies, augmentation schedules) that are harder to reproduce and adapt.

Prefer training a specialized supervised model on target-domain data when:

• You have abundant labeled data for your specific target domain (thousands of labeled examples per species). On ReefSet — the one marine dataset where Perch 2.0 was compared against a model trained on the target data — SurfPerch (trained on ReefSet) achieved 0.986 ROC-AUC vs. Perch 2.0's 0.981 (Table 8). The margin is small (0.005) but directionally favors domain-specific training. With more target-domain data, a specialized model could potentially pull further ahead.
• Your target task involves acoustic phenomena fundamentally unlike animal vocalizations (e.g., anthropogenic noise classification, geophony, or bat echolocation). The paper explicitly excluded bat recordings because "their vocalizations cannot be represented using the spectrogram parameters we selected" (Section 2.1), and the FSD50K general sound classes (engines, music) were included but not evaluated for transfer. If your task operates in an acoustic regime outside the 60 Hz–16 kHz mel-spectrogram representation, Perch 2.0's frontend and learned features may not transfer.

Prefer self-supervised pre-training (e.g., BirdMAE-L) when:

• You operate in a truly data-scarce regime where even 16-shot linear probing is insufficient, but you have access to large amounts of unlabeled in-domain audio. The paper's hypothesis (Section 4.1) is that self-supervision may catch up if unlabeled data scale increases by orders of magnitude. If you have millions of hours of unlabeled target-domain audio, self-supervised pre-training on that data followed by fine-tuning on your small labeled set may outperform Perch 2.0's cross-domain transfer. However, this hypothesis is untested in the paper — the paper's own self-supervised experiments (MAE, HuBERT, SimCLR) reportedly underperformed supervision at the 1.5M-example scale, so the required unlabeled data volume for self-supervision to surpass Perch 2.0 is unknown and may be impractically large.
• You need the model to support zero-shot classification via language queries (e.g., "find recordings of a high-pitched trill"). BioLingual (listed in Table 3) supports this but achieves substantially lower accuracy (0.484 on BEANS classification) than Perch 2.0 (0.719). If zero-shot language-based retrieval is a hard requirement, you accept the accuracy tradeoff.