Vision2025~112 minarXiv:2509.19087

Zero-Shot Multi-Spectral Learning: Reimagining a Generalist Multimodal Gemini 2.5 Model for Remote Sensing Applications

score 3Ganesh Mallya, Yotam Gigi, Dahun Kim +4↑ 2Sep 23, 2025cs.CV

🎯 Pitch

Generalist vision models like Gemini 2.5 can be tricked into understanding satellite sensor data they were never trained on—without any fine-tuning. Just by converting multi-spectral bands into custom false-color images and explaining them in plain language prompts, accuracy on land-cover classification benchmarks jumps significantly, matching or beating dedicated remote sensing models.

1. Executive Summary

This paper proposes a training-free Zero-Shot approach that enables generalist multimodal models—trained exclusively on RGB inputs—to understand unfamiliar multi-spectral satellite imagery by converting spectral bands into pseudo-color images (false-color composites, NDVI, NDWI, NDMI maps) and pairing each image with an extensive descriptive prompt that explains the sensor bands, their physical meaning, and how each pseudo-image was constructed. Evaluating Gemini 2.5 on the BigEarthNet and EuroSat benchmarks, the method improves Zero-Shot land-cover classification F1 score by approximately +0.04 on BigEarthNet (both 43-class and 19-class variants) and boosts accuracy by +3% on EuroSat, establishing new state-of-the-art Zero-Shot results while requiring no model retraining, fine-tuning, or domain-specific data collection—establishing that generalist multimodal models can exploit specialized sensing modalities effectively without architectural modification, provided the new inputs are translated into the model’s native visual and textual reasoning space.

2. Context and Motivation

The Core Problem: Generalist Multimodal Models Cannot Ingest Specialized Sensor Data

The fundamental problem this paper tackles is a mismatch between two powerful but disconnected forces in remote sensing: the rapidly advancing capabilities of generalist large multimodal models (LMMs) trained on massive internet-scale RGB data, and the specialized, multi-sensor nature of Earth observation data that carries rich physical information beyond what three-channel RGB can capture. This problem is not merely a technical inconvenience—it represents a structural barrier that prevents the remote sensing community from benefiting from the most powerful visual understanding systems ever built.

To understand why this matters, consider what multi-spectral imagery provides. Sentinel-2, one of the most widely used Earth observation satellite platforms, captures 12 spectral bands spanning visible light (blue, green, red), vegetation-sensitive red-edge bands, near-infrared (NIR), narrow NIR, water vapor, coastal aerosol, and two short-wave infrared (SWIR) bands. Each of these channels correlates with distinct physical properties on the ground: NIR reflectance is strongly indicative of photosynthetically active vegetation, SWIR bands detect moisture content in soil and plant matter, and the red-edge channels capture subtle transitions in leaf chlorophyll absorption. An RGB image, by contrast, represents only three of these twelve dimensions, discarding approximately 75% of the spectral information available from a single Sentinel-2 acquisition. This discarded information is precisely what enables remote sensing specialists to distinguish between visually similar land cover types—a coniferous forest and a broad-leaved forest may look nearly identical in visible light but have dramatically different spectral signatures in the NIR and SWIR bands.

The gap is not academic. The remote sensing community invests enormous resources in building domain-specific models precisely because this extra spectral information translates directly into analytical accuracy. As the paper notes, "multi-spectral bands capture additional frequencies which can detect various aspects of the material properties on the ground" and "combining multiple sensors can lead to more accurate and detailed responses, whereas individual sensors, e.g. RGB, have blind spots and limitations." When a model cannot access these bands, it operates with an information deficit that no amount of reasoning or visual sophistication can fully compensate for—the information simply is not present in the input.

At the same time, generalist multimodal models like Gemini 2.5 represent an unprecedented concentration of visual understanding capability. These models have been trained on enormous corpora of natural images, learning rich representations of objects, scenes, textures, spatial relationships, and contextual cues. They can reason about what they see, follow complex instructions, and generalize surprisingly well to domains they were not explicitly trained on—including, as the paper's RGB-only baselines demonstrate, overhead remote sensing imagery. The question the paper poses is whether this powerful generalist capability can be bridged to the specialized sensing modalities that make remote sensing data so valuable, without requiring the model to be retrained.

Why This Gap Is Important: Practical, Economic, and Strategic Dimensions

The cost and fragility of domain-specific model development. The dominant approach to handling multi-spectral data in remote sensing is to train specialized foundation models from scratch or fine-tune existing models on domain-specific data. This is expensive in multiple dimensions. First, data collection and curation for remote sensing is non-trivial: satellite imagery requires preprocessing (atmospheric correction, cloud masking, geometric alignment), annotations often require domain expertise, and the data volumes are substantial. Second, training specialized foundation models from scratch—models like SatMAE, Prithvi, SpectralGPT, and HyperSIGMA—requires significant computational resources, often comparable to the resources used for the generalist models they seek to replicate. Third, and perhaps most critically, these specialized models are brittle with respect to sensor evolution.

The paper explicitly calls out this brittleness: "new generation of sensors might extend the number of inputs or produce different or higher quality data than previous ones; new types of sensors can also become available." A model trained to ingest the specific 12-band configuration of Sentinel-2 cannot, without modification, handle a future satellite with 15 bands, or a legacy sensor with only 4 bands, or a combination of optical and SAR (Synthetic Aperture Radar) data. Each sensor change potentially requires retraining or architectural modification. In contrast, a generalist model that can be prompted to understand new inputs without retraining would be resilient to sensor evolution—a new band simply means a new pseudo-image and an updated prompt description, not a new training pipeline.

The opportunity cost of ignoring generalist capabilities. Generalist multimodal models are not just good at vision—they bring capabilities that specialized remote sensing models typically lack. These include free-form reasoning about visual content, the ability to answer questions and generate explanations in natural language, contextual understanding that draws on broad world knowledge, and the capacity to follow complex, multi-part instructions. A remote sensing analyst using a Gemini 2.5-like system could ask follow-up questions, request justifications for classifications, or specify task constraints in natural language—interaction patterns that specialized classification models, designed to output a single label from a closed set, cannot support. By enabling these generalist models to ingest multi-spectral data, the paper's approach potentially unlocks this richer mode of interaction for geospatial professionals.

Deployability and accessibility. Specialized foundation models for remote sensing, once trained, must be hosted and maintained. This creates barriers for smaller organizations, researchers in low-resource settings, and practitioners who need to adapt quickly to new sensors or tasks. The paper's approach, by contrast, requires no training infrastructure whatsoever—it is purely an inference-time adaptation that reorganizes existing inputs into a format the generalist model already understands. This dramatically lowers the barrier to entry: anyone with access to a generalist multimodal model (which are increasingly available via APIs) and a Sentinel-2 image can immediately benefit from multi-spectral analysis without training a single parameter.

Where Prior Approaches Fall Short

The paper identifies three broad categories of prior work, each with specific limitations that the proposed approach is designed to address.

Remote sensing-specific multimodal models remain RGB-only. A substantial body of work has adapted generalist vision-language models to remote sensing through continued pre-training or fine-tuning on domain-specific data. RemoteCLIP, SkyCLIP, RS-CLIP, SkySense, SatlasPretrain, EarthPT, GeoChat, Scale-MAE, GFM, and others represent this approach. These models inherit the vision-language alignment of their generalist backbones but are specialized for overhead imagery through exposure to remote sensing data during training. This is a natural and effective strategy, but it has a fundamental limitation: these adaptations are exclusively in the RGB domain. They learn to better understand overhead views, to recognize remote-sensing-specific objects and patterns, and to align remote sensing imagery with domain-specific language—but they do so using the same three-channel input format as the generalist models they extend. They cannot ingest the additional spectral bands that make multi-spectral analysis powerful, because their architecture and training process never expose them to such inputs. The paper's approach directly addresses this gap by providing a mechanism to feed multi-spectral information into models that were never designed for it.

Multi-spectral foundation models are powerful but rigid and expensive. A second line of work tackles the multi-spectral challenge head-on by training models specifically designed for multi-band input. SatMAE extends masked autoencoding pre-training to multi-spectral and temporal satellite imagery. The Prithvi model series ingests Blue, Green, Red, Narrow NIR, SWIR 1, and SWIR 2 as six input channels. SpectralGPT proposes a 3D Generalized Transformer for hyperspectral data. HyperSIGMA uses sparse attention to handle the high dimensionality of hyperspectral inputs. CROMA, OmniSat, and MMEarth explore self-supervised learning across multiple sensor modalities. These models represent substantial engineering efforts and achieve strong performance on their target tasks.

However, the paper identifies three critical limitations. First, training these models is resource-intensive: collecting the necessary multi-spectral training data, designing architectures that can handle variable numbers of input bands, and running the large-scale pre-training needed to learn useful representations all require significant investment. Second, these models are sensor-locked: a model trained on Sentinel-2's specific band configuration cannot trivially adapt to a new satellite with different spectral characteristics, resolution, or number of bands. The paper makes this point explicitly: "these foundational models are not flexible in terms of adding new input channels." Third, they replicate effort: each new multi-spectral foundation model essentially re-implements the visual representation learning pipeline that generalist models have already done at much larger scale, but constrained to a narrower data distribution. The paper's approach sidesteps all three limitations by leaving representation learning entirely to the generalist model and focusing instead on input adaptation.

Embedding-based approaches lack Zero-Shot flexibility. A third paradigm, exemplified by MOSAIKS, SatlasPretrain, EarthPT, S2Vec, and AlphaEarth Embeddings, treats remote sensing prediction as a two-stage process: first, a featurization step transforms multi-sensor inputs into a compact embedding vector, then a regression step maps these embeddings to task-specific outputs. These embeddings can integrate multiple sensor modalities and are useful for clustering, fine-tuning, and low-shot learning. However, as the paper notes, "they are not easy to use in zero-shot settings, as our model is." Embedding-based approaches require at least some labeled data to train the regression step, or they operate in a retrieval paradigm where similarity to labeled examples drives predictions. They do not support the kind of open-ended, instruction-following, zero-shot classification that a prompted multimodal model can achieve, where the model reasons about the input based solely on the class descriptions provided in the prompt—no examples, no training, no embeddings.

How This Paper Positions Itself

The paper's positioning is distinctive in several ways. It does not propose a new model architecture, a new training procedure, or a new dataset. Instead, it proposes a methodological shift: rather than adapting models to data (through training or fine-tuning), adapt data to models (through format conversion and descriptive prompting). This inversion of the usual paradigm is what enables the "training-free" and "Zero-Shot" claims—the model itself is completely unchanged, and all adaptation happens in the input preprocessing and prompting layer.

The paper explicitly targets a scenario that none of the prior work addresses: a user who already has access to a powerful generalist multimodal model (like Gemini 2.5) and wants to apply it to multi-spectral remote sensing tasks without any additional training, data collection, or model deployment. This is a pragmatic, user-centered motivation. It does not claim that the approach will outperform a carefully fine-tuned multi-spectral foundation model on every metric—the paper does not, in fact, compare against such models at all. Rather, it claims that the approach enables a generalist model to significantly improve over its own RGB-only baseline by incorporating multi-spectral information, and that this improvement comes at essentially zero additional cost beyond the inference-time effort of generating pseudo-images and composing a prompt.

The paper also positions itself as a demonstration of a general principle rather than a one-off solution for specific spectral bands or sensors. The choice of five specific pseudo-image combinations (false-color near-infrared, NDVI, NDWI, and two NDMI variants) is presented as illustrative, not exhaustive. The paper explicitly states that "out of 12 multi-spectral bands we only created five example image combinations, whereas many other useful combinations can be included. This can improve performance further, without needing to change the approach." This framing suggests that the method is extensible: new sensors, new spectral indices, or new band combinations can be incorporated simply by adding more pseudo-images and updating the prompt description, without touching the model.

Furthermore, the paper implicitly argues that generalist pre-training provides unseen multi-spectral reasoning capability. The model has never been trained on NDVI or NDWI images—these are spectral indices computed from band ratios, not web images. Yet when provided with a pseudo-color NDVI image and a text description explaining that "this image is an NDVI image which is a colormap with values Red, Yellow and Green generated using B08 and B04 bands," the model can apparently use this information to improve its classification accuracy. This is a subtle but important claim: it suggests that the model's visual understanding is abstract enough to reason about novel color mappings when given appropriate textual grounding, and that its reasoning capability can bridge the gap between the unfamiliar visual input and the semantic task.

The paper's relationship to the existing literature is thus not competitive but complementary: it does not replace specialized multi-spectral models for applications where maximum accuracy on a fixed sensor configuration is paramount, but it offers a lightweight, flexible, and immediately deployable alternative for scenarios where adaptability, Zero-Shot capability, and access to generalist reasoning are valued.

3. Technical Approach

3.1 Reader orientation (approachable technical breakdown)

The "system" here is not a trained model or a software pipeline — it is a data preparation protocol that converts specialized multi-spectral satellite data into a format that an unmodified, RGB-only generalist multimodal model (Gemini 2.5) can already understand and reason about. The problem it solves is that powerful generalist vision-language models cannot ingest the 12-band spectral imagery that remote sensing professionals rely on, because these models were trained exclusively on three-channel RGB images harvested from the web; the solution is to re-encode the multi-spectral information as a set of visually interpretable pseudo-color images and to pair those images with an extensive natural-language prompt that teaches the model what each image means in physical terms, enabling Zero-Shot inference without any training, fine-tuning, or architectural modification.

3.2 Big-picture architecture (diagram in words)

The approach has five logical components, though only two are "active" at inference time:

Multi-spectral source data — a 12-band Sentinel-2 image (or equivalent) providing the raw spectral measurements that contain the physical information the model needs but cannot natively ingest.
Band combination and pseudo-image generator — a deterministic preprocessing step that takes the raw multi-spectral channels and produces six output images: one standard RGB composite and five derived visualizations (false-color NIR composite, NDVI, NDWI, and two NDMI variants). These are saved as standard RGB images, which is the only format the model can accept.
Prompt constructor — a hand-crafted template that describes, in detail, what each of the six images represents, what satellite bands were used to create it, what physical properties each band measures, and how the derived indices (NDVI, NDWI, NDMI) should be interpreted. This prompt also includes the task instruction and the list of possible answer classes.
Frozen generalist multimodal model (Gemini 2.5) — a large vision-language model trained on web-scale RGB data that receives the six pseudo-images and the descriptive prompt together as a single multi-modal input, and produces a classification prediction by reasoning jointly over the visual content of all six images and the textual explanations of what each image encodes.
Answer parser — a simple post-processing step that extracts the model's output (in a structured format like "(1),(3)" for multi-label tasks or "(5)" for single-label) and maps it back to class names for evaluation.

The information flow is strictly feed-forward and single-pass: raw Sentinel-2 bands → band math and color mapping → six pseudo-color images saved to disk → images concatenated with prompt text → single Gemini 2.5 inference call → structured answer extraction. There is no training loop, no gradient computation, no model weight modification, and no iterative refinement. Every step except the Gemini 2.5 inference is deterministic preprocessing.

3.3 Roadmap for the deep dive

First, the core insight: representing spectral indices as pseudo-color images. Before anything else, we need to understand why a model trained on natural photographs can interpret an NDVI color map without having seen one — this is the central design hypothesis that the whole approach depends on, and it frames every subsequent decision.
Second, the specific band combinations and pseudo-image recipes. The paper creates exactly five derived images beyond RGB. We will walk through each one: what bands go in, what mathematical operation is performed, what color mapping is applied, and what physical property each image is designed to reveal.
Third, the prompt structure. The prompt is not just a task instruction — it is a miniature lecture on remote sensing delivered to the model at inference time. We will examine the full prompt template, what information it includes, and how that information bridges the gap between the unfamiliar visual inputs and the model's existing world knowledge.
Fourth, the task-specific prompt adaptation. The base prompt template is modified for each dataset and each task format (single-label vs. multi-label classification), and we will see how these modifications are handled.
Fifth, implementation details. We will cover the normalization procedure for true-color and false-color images, the resolution differences between bands, and how they are handled.
Sixth, what this approach is NOT doing. Understanding the negative space is important: no embedding extraction, no retrieval, no training on unlabeled data, no inductive inference from dataset statistics.

3.4 Detailed, sentence-based technical breakdown

This is primarily a methodological demonstration paper whose core idea is that a generalist multimodal model's native visual reasoning capabilities can be leveraged to understand unfamiliar sensing modalities when those modalities are translated into the model's established input format (RGB images) and grounded through detailed natural-language explanations of the physical meaning of the novel inputs.

The Core Insight: The Model Can Reason About Visual Encodings It Has Never Seen

The paper's entire approach rests on one non-obvious claim: that a large multimodal model trained on natural photographs, diagrams, charts, and other web imagery has learned a sufficiently abstract understanding of color coding as a representational convention that it can interpret a previously unseen spectral index visualization if given an adequate textual explanation. This is not a claim about the model understanding spectral physics — Gemini 2.5 has no concept of electromagnetic wavelengths or atmospheric absorption bands in any direct sense. Rather, it is a claim about transfer of visual reasoning: the model has seen thousands of examples where color encodes meaning (heatmaps, medical imaging with false color, weather radar maps, scientific visualizations, infrared photography with colormap overlays), and it has learned that when a text description says "red means X, green means Y, blue means Z," it can apply that mapping to interpret a novel image.

The paper does not prove or analyze this mechanism — it takes it as a working hypothesis and tests it empirically by measuring whether adding the pseudo-images and descriptive prompts improves classification accuracy. The positive results (Tables 1–3) are the evidence that this transfer of visual reasoning capability actually occurs. This is important to understand because it means the approach is not teaching the model about remote sensing in any durable sense — it is providing the model with enough context at inference time to apply its existing visual intelligence to an unfamiliar encoding scheme.

The implication is that the method's effectiveness depends on two factors: (1) the model's general visual reasoning capability (more capable models should benefit more), and (2) the quality and specificity of the prompt description (vague descriptions that do not explain the color mapping or the physical interpretation will not work as well).

The Multi-Spectral Input Encoding: From 12 Bands to 6 RGB Images

The raw input for the approach is a multi-spectral image with 12 bands, matching the Sentinel-2 sensor configuration. These bands are:

B01: Coastal Aerosol (60 m resolution, 443 nm) — sensitive to atmospheric particles and shallow water
B02: Blue (10 m resolution, 490 nm) — standard visible blue
B03: Green (10 m resolution, 560 nm) — standard visible green
B04: Red (10 m resolution, 665 nm) — standard visible red
B05: Red Edge 1 (20 m resolution, 704.1 nm) — transition between red and NIR, sensitive to chlorophyll absorption edge
B06: Red Edge 2 (20 m resolution, 740.5 nm) — further into the red-to-NIR transition
B07: Red Edge 3 (20 m resolution, 782.8 nm) — near the NIR plateau
B08: NIR (10 m resolution, 842 nm) — near-infrared, strongly reflected by healthy vegetation
B8A: Narrow NIR (20 m resolution, 865 nm) — narrower NIR band, less affected by water vapor
B09: Water Vapor (60 m resolution, 945 nm) — atmospheric water vapor absorption
B11: SWIR 1 (20 m resolution, 1613.7 nm) — short-wave infrared, sensitive to moisture and mineral composition
B12: SWIR 2 (20 m resolution, 2202.4 nm) — deeper SWIR, sensitive to geological features and moisture

The paper transforms these twelve channels into six output images, each an 8-bit RGB image (three channels, values in [0, 255]). The key design principle is that every output image is something the model has a chance of understanding: a natural-color photograph, a false-color composite (which looks like an oddly-colored photograph), or a pseudocolor map with a defined legend explained in the prompt. Below is the exact recipe for each.

Image 1: True-Color RGB Composite

Input bands: B04 (Red), B03 (Green), B02 (Blue).

Processing: Each band is individually normalized to bring its values into the range [0, 1]. The paper does not specify the normalization method in detail (whether min-max per-image, percentile clipping, or global statistics), but the stated goal is to produce values in [0, 1] that are then scaled to [0, 255] as integers. These three scaled values are stacked to form a standard 24-bit RGB image.

What it represents: This is a natural-color photograph approximating what a human would see from space. The model has been extensively trained on aerial and satellite photographs (Google Earth imagery, overhead views in web datasets), so this image is in the model's native visual domain.

Why it is included: This serves as the baseline visual input — the same information available to the model in the RGB-only setting. Including it alongside the derived images allows the model to correlate the familiar visual appearance with the unfamiliar spectral indices.

Image 2: False-Color Composite (NIR-Red-Green)

Input bands: B08 (NIR) mapped to the red channel of the output image, B04 (Red) mapped to the green channel, B03 (Green) mapped to the blue channel.

Processing: Same normalization as the true-color image — each band normalized to [0, 1], scaled to [0, 255], stacked.

What it represents: In this color scheme, healthy vegetation (which strongly reflects NIR while absorbing red) appears bright red or pink. Urban areas appear gray-blue. Bare soil appears in shades of cyan or light blue. Water appears dark blue or black. This is an extremely common visualization in remote sensing — the standard "false-color infrared" composite used in countless publications, news articles, and educational materials.

Why the model can understand it: The model has likely seen false-color infrared images in its web training data — they appear in geography textbooks, remote sensing tutorials, environmental journalism, and land-use documentation. Even if the specific band mapping is new, the visual appearance (bright red vegetation, dark water) is distinctive and learnable. The prompt reinforces this by explicitly stating which bands went into which channel.

Why this specific band combination: NIR-Red-Green is the most informative three-channel false-color composite for vegetation analysis. The NIR channel in the red position makes photosynthetically active vegetation visually salient, while the red and green channels maintain some continuity with natural color perception.

Image 3: NDVI Map

The Normalized Difference Vegetation Index (NDVI) is computed from the NIR and Red bands:

$\text{NDVI} = \frac{\text{B08} - \text{B04}}{\text{B08} + \text{B04}}$

where B08 is the NIR band (approximately 842 nm) and B04 is the Red band (approximately 665 nm).

What it computes: NDVI exploits the fact that healthy vegetation strongly reflects near-infrared light (due to the spongy mesophyll cell structure in leaves) while strongly absorbing red light (due to chlorophyll). The numerator $\text{B08} - \text{B04}$ produces a positive value for pixels where NIR reflectance exceeds red reflectance (vegetation) and a value near zero or negative where they are similar (bare soil, water, urban surfaces). The denominator $\text{B08} + \text{B04}$ normalizes the difference by total brightness, making the index relatively insensitive to illumination conditions (sun angle, cloud shadows, topographic shading). The output is a continuous value theoretically in the range $[-1, 1]$ , though in practice over land surfaces it typically ranges from approximately -0.1 (water) to 0.9 (dense, healthy vegetation).

Why this form: The normalized-difference formula $(a - b)/(a + b)$ is a standard vegetation index dating back to Rouse et al. (1973) because it provides two critical properties: (1) radiometric normalization — division by the sum cancels out multiplicative factors from illumination geometry and sensor calibration, making NDVI comparable across different acquisition dates and atmospheric conditions without elaborate correction, and (2) bounded output — the range [-1, 1] enables straightforward color mapping. Alternative indices that use only the difference (a - b) without normalization are sensitive to brightness variations. The specific band choice (NIR at ~842 nm, Red at ~665 nm) maximizes the spectral contrast between photosynthetically active vegetation and other surface types because these two wavelengths occupy opposite sides of the "red edge" — the sharp rise in vegetation reflectance that occurs between approximately 680 and 750 nm.

Color mapping: The paper states that the NDVI image is produced as "a colormap with values Red, Yellow and Green." The exact mapping function is not specified, but the standard convention (which the model may have encountered in web imagery) maps low NDVI values (near 0 or negative) to red, intermediate values to yellow, and high values (dense vegetation) to green. This is encoded as an RGB image where each pixel's color is determined by look-up from its NDVI value.

What it reveals: Vegetation presence, density, and health are immediately visible. Dense forests and healthy cropland appear bright green, sparse vegetation or senescent crops appear yellow-orange, and non-vegetated surfaces (water, bare soil, built-up areas) appear red.

Image 4: NDWI Map

The Normalized Difference Water Index (NDWI) is computed from the Green and NIR bands. The paper uses the formulation:

$\text{NDWI} = \frac{\text{B03} - \text{B08}}{\text{B03} + \text{B08}}$

where B03 is the Green band (560 nm) and B08 is the NIR band (842 nm). Note that this is the McFeeters (1996) formulation, not the Gao (1996) formulation (which uses NIR and SWIR). The McFeeters NDWI uses green minus NIR, making water bodies produce high positive values because water reflects more green light than NIR (water strongly absorbs NIR).

What it computes: Water surfaces have a distinctive spectral signature: reflectance is moderate in the visible spectrum (particularly in the blue and green) but drops sharply to near-zero in the near-infrared. The numerator $\text{B03} - \text{B08}$ is positive for water pixels (where green reflectance exceeds NIR) and negative for vegetation and soil pixels (where NIR reflectance exceeds green, often dramatically). The denominator $\text{B03} + \text{B08}$ provides the same illumination normalization as in NDVI. Output values range from approximately -1 to 1, with water bodies typically producing values above 0, and values above roughly 0.2–0.3 generally indicating open water.

Why this form: The same normalized-difference structure as NDVI, but with bands chosen to maximize the spectral contrast for water detection rather than vegetation. Using NIR in the denominator rather than red is crucial because water's reflectance difference between green and NIR is much larger than between green and red — water reflects some red light (making a red-based index less discriminative) but essentially no NIR. The alternative Gao NDWI (using NIR and SWIR, $\text{B08} - \text{B11})/(\text{B08} + \text{B11})$ ) is more sensitive to vegetation water content than surface water bodies, so the McFeeters formulation is the correct choice for this application.

Color mapping: The paper specifies that the NDWI image uses "a color map varying linearly as [(1, 1, 1), (1, 1, 1), (0, 0, 1)]" with values in the range [-0.8, 0.8]. This is an unusual specification that deserves careful parsing: the color map has three control points — (1, 1, 1) is white (R=1, G=1, B=1), (1, 1, 1) is white again, and (0, 0, 1) is blue. A linear interpolation between these points would produce: for values near the low end (-0.8) → white, transitioning to blue at the high end (+0.8). The double-white control point likely means that dry land pixels (low NDWI) appear white, while pixels with high water content transition to blue. This is visually intuitive: water bodies appear blue, land appears white/pale.

What it reveals: Surface water bodies (lakes, rivers, reservoirs, coastal waters) appear as distinct blue regions. This directly addresses one of the failure cases the paper visualizes in Figure 4, where an RGB model confuses a forest scene with water because the deep blue-green foliage resembles water in visible wavelengths — the NDWI image would show the forest as white (no free water) and an actual lake as blue.

Image 5 and 6: Two NDMI Variants

The Normalized Difference Moisture Index (NDMI) is computed twice, using two different SWIR bands in the denominator position:

For Image 5:

$\text{NDMI}_{\text{B11}} = \frac{\text{B8A} - \text{B11}}{\text{B8A} + \text{B11}}$

For Image 6:

$\text{NDMI}_{\text{B12}} = \frac{\text{B8A} - \text{B12}}{\text{B8A} + \text{B12}}$

where B8A is the Narrow NIR band (865 nm), B11 is SWIR 1 (1613.7 nm), and B12 is SWIR 2 (2202.4 nm).

What each computes: NDMI exploits the fact that water strongly absorbs short-wave infrared radiation — the longer the wavelength, the stronger the absorption. The Narrow NIR band (B8A at 865 nm) is relatively unaffected by water absorption, so it serves as a reflectance reference. The SWIR bands (B11 at ~1.6 µm, B12 at ~2.2 µm) are progressively more absorbed by water. The numerator $\text{B8A} - \text{B11}$ (or $\text{B8A} - \text{B12}$ ) produces high positive values for dry surfaces (where NIR reflectance greatly exceeds SWIR reflectance) and low or negative values for wet surfaces (where SWIR reflectance is suppressed by water absorption). The denominator provides normalization.

Why two variants: The two SWIR bands have different sensitivities. B11 (1.6 µm) is sensitive to moisture in vegetation canopies and surface soil layers — it is the standard band for agricultural drought monitoring. B12 (2.2 µm) penetrates deeper and is more sensitive to geological moisture, mineral composition, and the distinction between soil types. By including both, the model receives complementary moisture information: B11-based NDMI for vegetation water stress and surface moisture, B12-based NDMI for deeper moisture signals and mineral/geological context. The paper describes both images as "used for identifying wet and dry areas."

Why this form: NDMI is the moisture analog of NDVI — it normalizes a water-absorption band against a non-absorption band to create an illumination-invariant moisture index. The choice of Narrow NIR (B8A) rather than standard NIR (B08) as the reference band may be because B8A (865 nm, 20 m bandwidth) is less affected by water vapor absorption than B08 (842 nm, 115 m bandwidth). The broader bandwidth of B08 includes some water vapor influence that could confound the moisture signal.

Color mapping: Both NDMI images use "the colormap varying linearly as [(1, 0, 0), (0, 1, 0), (0, 0, 1)]" — a linear interpolation from red (dry) through green (intermediate) to blue (wet). This is a standard "rainbow" pseudo-color scheme that the model has likely encountered in scientific visualizations.

What they reveal: Together, the two NDMI images provide information about surface and subsurface moisture distribution, which is crucial for distinguishing between land cover classes that may look similar in visible wavelengths: a dry pasture and an irrigated agricultural field; a naturally dry grassland and a wetland at the edge of its moisture gradient; a coniferous forest on well-drained soil and one on waterlogged peat.

The Prompt Engineering: A Domain Mini-Lecture at Inference Time

The prompt is the second critical component of the approach — without it, the model would receive six images including unrecognizable pseudo-color visualizations with no way to map colors to physical meaning. The prompt serves as a real-time grounding mechanism that teaches the model the encoding scheme.

The full multi-spectral prompt (reproduced in Appendix A) has the following structure, which we will deconstruct:

Section 1 — Task framing: "Instructions: Answer the question asked after the given 6 images of the same scene." This establishes that all six images represent the same geographic location and should be considered jointly.

Section 2 — Sensor metadata: A complete listing of all 12 Sentinel-2 bands with their names, central wavelengths (where available), and spatial resolutions. This section says, in essence, "here are the raw materials that were available." Including wavelengths and resolutions gives the model physical context — it tells the model that Band B11 (1613.7 nm) measures fundamentally different physical properties from Band B02 (490 nm) because they are in different parts of the electromagnetic spectrum, and that some bands have coarser spatial resolution (60 m for B01 and B09, 20 m for B05–B07, B8A, B11, B12, 10 m for the others) which affects how much spatial detail they can capture.

Why include this metadata: The model has no built-in knowledge of Sentinel-2 bands, but it does have world knowledge about wavelengths, spatial resolution, and the electromagnetic spectrum. By providing the mapping from band numbers to physical properties, the prompt gives the model the vocabulary it needs to understand the subsequent image descriptions. The explicit listing of resolutions also signals that some derived images may have coarser spatial detail than others, which could affect how the model weights their contribution.

Section 3 — Image descriptions: For each of the six images, a one-to-two-sentence description stating what the image is, what bands were used, what color mapping was applied, and what physical interpretation to draw. The descriptions are factual and operational, not abstract:

"The first image is the RGB image generated using B04, B03 and B02 bands." — This maps the familiar RGB concept to the specific Sentinel-2 bands.
"The second image is the False Color Composite image generated using B08, B04 and B03 bands." — This explains the unusual coloration by stating the channel mapping.
"The third image is the NDVI image which is a colormap with values Red, Yellow and Green generated using B08 and B04 bands." — This provides the color key (Red → low NDVI → sparse vegetation, Yellow → intermediate, Green → high NDVI → dense vegetation) and the band formula.
"The fourth image is NDWI image whose values are in the range of -0.8 to 0.8 and using a color map varying linearly as [(1, 1, 1), (1, 1, 1), (0, 0, 1)]." — This provides the value range and the color mapping function explicitly.
"The fifth image is the NDMI images generated using B8A and B11 bands with the colormap varying linearly as [(1, 0, 0), (0, 1, 0), (0, 0, 1)]. This image is used for identifying wet and dry areas." — This provides the color key (Red → dry, Green → intermediate moisture, Blue → wet) and the band formula. The explicit statement of purpose ("identifying wet and dry areas") tells the model what to look for in this image.
The sixth image description mirrors the fifth with B12 instead of B11.

Why these descriptions are in the prompt rather than just the images: The model has no built-in NDVI or NDWI concept — these are domain-specific spectral indices that do not appear in natural language training data in their raw formulaic form. However, the model does understand the concepts encoded in the descriptions: "colormap with values Red, Yellow and Green," "identifying wet and dry areas," "False Color Composite." By translating the domain-specific encoding into a combination of (1) a visual representation the model can see and (2) a textual explanation of what it means, the prompt creates a temporary grounding that allows the model to reason about the novel visual input.

Section 4 — Output format constraint: The prompt ends with strict formatting instructions: "Output the option numbers corresponding to the correct answer in the format '(X)' where X is the correct number choice... In case of multiple correct answers, output the answer choices in the same format separated by commas." This is a standard technique for structured extraction from LLMs — by constraining the output format, the prompt ensures that the answer can be mechanically parsed and compared against ground truth without ambiguity. It also reduces the likelihood of the model generating verbose explanations that might drift from the correct classification.

Section 5 — Task question and class list: The prompt concludes with the specific classification question and the enumerated list of possible classes. For BigEarthNet (43-class version), this is a list of 43 land cover categories. For BigEarthNet (19-class version), the merged taxonomy. For EuroSat, 10 land use classes.

The multi-label handling: BigEarthNet is a multi-label dataset — a single image can belong to multiple classes (e.g., an agricultural landscape might contain both "Non-irrigated arable land" and "Pastures" and "Broad-leaved forest" in different parts of the 120×120 pixel scene). The prompt accommodates this with the instruction "Select all that apply" and the formatting allows multiple comma-separated answers, e.g., "(1),(3),(26)". The model is not told to limit the number of classes, which is appropriate since the number of ground-truth labels per image varies.

Task-Specific Prompt Adaptation

The prompt template is not universal — it is modified for each evaluation setting:

For RGB-only baselines on BigEarthNet: The prompt is substantially shortened. It removes the entire sensor metadata section, all the multi-spectral image descriptions, and the reference to "6 images." It simply provides the instruction, the output format constraint, the list of classes, and the word "Select all that apply." This is the control condition — the model sees only the true-color RGB image with a minimal prompt, testing whether the multi-spectral images and descriptive prompt actually provide additional value.

For EuroSat: The prompt structure is similar but the task changes from multi-label to single-label classification. The instruction becomes "To which of the following class does the given image belong to?" (singular "class"), the class list has 10 rather than 43 items, and the example formatting shows a single answer like "(1)" rather than "(1),(3)". The output format instruction is accordingly simplified.

Why this adaptation matters: The prompt is part of the "method" — the model's behavior is jointly determined by the images and the text that accompanies them. Changing the prompt changes the model's behavior. The paper's comparison between RGB-only and multi-spectral settings modifies both the input images (adding five pseudo-images) and the prompt (adding sensor descriptions and image explanations). This is appropriate because the additional images are meaningless without the explanatory text, but it means the performance gain cannot be attributed purely to the visual information — part of it may come from the additional textual context provided by the prompt (e.g., the model knowing it is looking at Sentinel-2 imagery rather than an arbitrary photograph may affect its classification behavior).

Implementation Details: Normalization and Image Generation

The paper provides specific implementation details for image generation, though some parameters are left implicit:

Normalization for true-color and false-color images: The paper states: "When generating the true color image and false color composite image (i.e. the first and the second images), we perform normalization of each band to bring the input value in the range [0, 1]. We then scale these values to fall in the range [0, 255] before stacking the bands together to be saved as RGB image."

This description implies per-band min-max normalization, where for each band independently, the minimum observed value across the image is mapped to 0 and the maximum to 1. This is the standard approach for visualizing satellite imagery because raw digital number values or top-of-atmosphere reflectance values can span very different ranges for different bands (NIR reflectance over vegetation is typically much higher than blue reflectance over the same scene). Without per-band normalization, the NIR channel would dominate the composite, washing out the red and green channels.

The alternative — using a global normalization based on sensor calibration — would produce more physically meaningful values but would often result in visually dull or poorly contrasted images because the dynamic range of natural scenes is much narrower than the sensor's full measurement range. The per-band min-max normalization is a visualization choice, not a radiometric one, and it means the absolute reflectance values are lost — only the relative brightness within each band is preserved.

Resolution handling: The Sentinel-2 bands have three different native resolutions: 10 m (B02, B03, B04, B08), 20 m (B05, B06, B07, B8A, B11, B12), and 60 m (B01, B09). The paper does not explicitly describe how these resolution differences are handled when bands of different resolutions are combined into a single RGB image. The standard approach in remote sensing would be to resample all bands to a common grid — typically the 10 m resolution — using nearest-neighbor or bilinear interpolation. This means that the 20 m and 60 m bands are upsampled, filling in sub-pixel values with interpolated estimates. The resulting pseudo-images thus contain a mix of genuine 10 m detail (from the native 10 m bands) and interpolated detail (from coarser-resolution bands), which could introduce subtle artifacts in the derived spectral indices.

Colormap application for spectral indices: The paper specifies colormaps for the NDVI, NDWI, and NDMI images in terms of linear interpolation between control points in RGB color space. For NDMI, the mapping is explicitly "[(1, 0, 0), (0, 1, 0), (0, 0, 1)]" — this is a linear ramp from red at the low end of the valid range, through green at the midpoint, to blue at the high end. Notably, this is not a perceptually uniform colormap — the transition from red to green passes through yellow (a perceptually salient color), and the transition from green to blue passes through cyan. The specific color interpolation is linear in RGB space, which is standard for programmatically generated scientific visualizations.

The choice of six images vs. more or fewer: The paper explicitly notes that "out of 12 multi-spectral bands we only created five example image combinations, whereas many other useful combinations can be included." This implies that the specific set of six images (RGB + 5 derived) is not necessarily optimal — it is a reasonable starting point that captures major spectral features (vegetation via NDVI, water via NDWI, moisture via two NDMI variants, general spectral structure via false-color NIR composite) without requiring an exhaustive search over all possible three-band combinations from 12 bands, which would produce $\binom{12}{3} = 220$ possible false-color composites plus numerous derived indices, far exceeding what could be practically included in a single prompt given context window or image token budget constraints.

What This Approach Is NOT Doing

Understanding the negative space — what the paper deliberately avoids — clarifies the contribution:

No fine-tuning or model adaptation: Gemini 2.5 is used exactly as provided. No weights are modified, no LoRA adapters are attached, no prompt tuning or prefix learning is performed. The model's parameters are frozen.

No in-context examples: The prompt includes only the task instruction, the sensor description, the image explanations, and the class list. Unlike few-shot prompting (where examples of correct input-output pairs are included in the prompt), this is strictly Zero-Shot — the model must infer the mapping from class names to visual features based solely on its pre-existing knowledge of what terms like "Broad-leaved forest," "Pastures," or "Industrial or commercial units" mean visually, combined with the information in the multi-spectral images.

No embedding extraction or nearest-neighbor retrieval: The approach does not use the model to generate feature vectors that are then compared to labeled examples, as in CLIP-style zero-shot classification (which computes cosine similarity between image embeddings and text embeddings of class names). The model directly generates a classification decision in a single forward pass, with all reasoning internal to the model.

No inductive inference from unlabeled data: The paper compares against ZLaP (Stojnic et al., 2024), which performs "Inductive Zero-Shot inference, meaning it is allowed to use the dataset examples, but without labels." This is a transductive or semi-supervised setting where the model can observe the distribution of unlabeled test examples and use that distributional information to improve its predictions. The proposed approach uses no information from the test set beyond the prompt and images for each individual example.

No dataset-specific tuning: The same Gemini 2.5 model, the same image generation procedure, and the same prompt structure (modulo class list and task format) are applied to both BigEarthNet and EuroSat. There is no per-dataset optimization of the pseudo-image combinations, the colormap ranges, or the prompt wording.

This is important because it means the reported results represent a lower bound on what the approach could achieve with tuning — the paper's goal is to demonstrate that the method works out of the box, not to squeeze every last point of accuracy through careful optimization. Future work could improve results by adding more spectral indices, optimizing colormap ranges, engineering more informative prompt language, or using in-context examples.

Design Rationale: Why Pseudo-Color Images Instead of Raw Band Values?

A natural question is: why go through the trouble of creating pseudo-color RGB images with specific color mappings, when the model (as a multimodal LLM) could potentially accept numerical data in a table or as a structured text description? The paper does not address this explicitly, but the design choice is grounded in the model's architecture and training:

The model's visual encoder is designed for photographic imagery. Gemini 2.5's vision component was trained on billions of natural images — photographs, screenshots, diagrams, charts, and other visual content with spatial structure. The visual encoder's representations are optimized for this distribution. Feeding an image of a false-color composite or a pseudo-colored NDVI map means the visual encoder processes it through the same convolutional or transformer layers that have learned to extract edges, textures, shapes, and spatial patterns from photographs. A tabular representation of the same spectral data (e.g., a list of 12 numerical values per pixel) would bypass the entire visual processing pipeline that gives the model its power — the model would have to reason about raw numbers rather than spatial patterns.

Color-coded spatial information is more information-dense for visual transformers. A 120×120 NDVI colormap image contains 14,400 pixels, each encoding a specific NDVI value as a color. The visual encoder can process this as a single image, extracting spatial patterns (where is vegetation concentrated? what is the shape of vegetated areas? how does vegetation relate to visible features in the RGB image?) in a way that is impossible with a pixel-wise numeric table in text form. The spatial coherence of spectral indices — the fact that adjacent pixels tend to have similar spectral properties — is preserved in the pseudo-image representation but would be scrambled in a textual representation.

The prompt provides the missing semantic grounding. The pseudo-color image alone would be ambiguous — the model sees colors but does not know what they mean. The prompt resolves this by explicitly stating: "Red means dry, green means intermediate moisture, blue means wet" for NDMI. This is analogous to how a human reads a scientific figure: the figure provides the spatial pattern, the caption and legend provide the semantic interpretation. The model is doing the same thing — joint reasoning over visual spatial information and textual semantic information.

The approach generalizes across models without architecture changes. Any multimodal model that accepts RGB images and text can use this approach. There is no need to modify the model's input layer to accept additional channels beyond RGB, no need to extend the tokenizer to handle spectral band values, and no need to redesign the positional encoding for multi-band inputs. The approach is purely an input preprocessing strategy, making it immediately applicable to any RGB-trained multimodal model without requiring access to model internals.

Summary of Design Choices and Their Justifications

Five derived pseudo-images (not more, not fewer): Covers the major spectral dimensions relevant for land cover classification (vegetation, water, moisture) without overwhelming the model with redundant or highly correlated inputs. Adding more band combinations is straightforward and expected to improve performance further.
Standard spectral indices (NDVI, NDWI, NDMI) rather than raw band visualizations: These indices are well-established in remote sensing with known physical interpretations, and they compress multi-band information into single-channel quantities that can be displayed with intuitive color scales. The model may have also encountered similar visualizations in web training data, though the paper does not test this.
Per-band min-max normalization for RGB composites: Produces visually interpretable images with good contrast. The tradeoff is loss of absolute radiometric information — two images with the same visual appearance after normalization might have very different absolute reflectance values, which could matter for some classification decisions.
Explicit color mapping descriptions in the prompt: Without these, the pseudo-color images would be meaningless to the model. The prompt supplies the legend. This is a critical design principle: any visual encoding must be accompanied by a textual key that explains the encoding.
Including the RGB image alongside the derived products: This anchors the model's analysis in the familiar visual domain, allowing it to correlate spectral index patterns with recognizable visual features. A forest's NDVI image shows green across the forested area; the model can see in the RGB image that this area is indeed a forest, reinforcing the connection between the NDVI signal and the land cover type.
Strict output format constraints: Ensures mechanical evaluation is possible and discourages the model from producing verbose, ambiguous, or hallucinated outputs that would be difficult to compare against the ground-truth label set.
No in-context examples or dataset-specific tuning: Keeps the approach genuinely Zero-Shot and demonstrates that the core mechanism (pseudo-image representation + descriptive prompting) works without dataset-specific optimization, establishing a lower bound on achievable performance.

4. Key Insights and Innovations

Innovation 1: Inverting the Adaptation Problem — Adapt Data to Models, Not Models to Data

The central conceptual move in this paper is deceptively simple but represents a genuine inversion of the dominant paradigm in remote sensing machine learning. The field has invested enormous effort in building models that can natively ingest specialized sensing modalities: modifying architectures to accept multi-band inputs (SatMAE, Prithvi), designing 3D transformers for hyperspectral data (SpectralGPT, HyperSIGMA), developing contrastive learning objectives that align radar and optical modalities (CROMA), and pre-training on massive multi-sensor collections (SkySense, OmniSat, MMEarth). Every one of these approaches shares a common, largely unquestioned assumption: the model must be trained or re-trained to accommodate the data format.

This paper asks a different question: what if the data is re-formatted to accommodate the model's existing capabilities? The shift from "how do we build a model that understands 12-channel Sentinel-2 imagery?" to "how do we represent 12-channel Sentinel-2 imagery so that an RGB-trained model can reason about it?" is not merely a pragmatic workaround — it is a conceptual reframing of the relationship between specialized sensing modalities and generalist AI systems. The model is treated as a fixed, powerful reasoning engine whose input interface is RGB + text, and the adaptation problem becomes one of encoding domain knowledge into that interface rather than retraining the engine.

This inversion has implications that go beyond the specific remote sensing application. It suggests a general design principle for deploying generalist multimodal models in specialized domains: translate the domain's information into the model's native representational space, and use language to supply the semantic grounding that the visual encoding lacks. The pseudo-color NDVI image is a visual encoding of spectral information; the prompt description is the semantic grounding that maps colors to physical meaning. Together they form a complete input that the model can process without any modification to its parameters or architecture. This principle — visual encoding + linguistic grounding = domain adaptation without training — could apply to any specialized sensing modality that can be meaningfully visualized: thermal infrared as heatmap overlays, LiDAR point clouds as depth-map renderings, SAR backscatter as grayscale intensity images, hyperspectral data as three-band false-color composites with text explaining the selected wavelengths.

The significance of this inversion is not measured by a single accuracy number in a table — it is measured by the flexibility it unlocks. When a new satellite is launched with a different spectral configuration, a model trained under the old paradigm needs to be retrained or at least fine-tuned. Under the proposed paradigm, only the prompt and the image generation script need to change. This is a qualitative difference in adaptability, not an incremental improvement. The paper does not frame it as an alternative that replaces specialized multi-spectral models for all purposes — rather, it frames it as a complementary capability that fills a gap no specialized model can address: the ability to incorporate new sensing modalities at inference time, without training, by anyone with access to a generalist multimodal model.

The comparison to prior work is stark. RemoteCLIP, SkyCLIP, and RS-CLIP adapt CLIP-style models to overhead imagery but remain firmly RGB-only — they adapt the model to the domain's visual statistics, not to its sensor physics. Multi-spectral foundation models like SatMAE and Prithvi adapt the model to the sensor physics but require training from scratch or extensive fine-tuning, and they are locked to the sensor configuration they were trained on. Embedding approaches like MOSAIKS and AlphaEarth can ingest multi-sensor data but require at least some labeled examples or retrieval-based inference, lacking true Zero-Shot flexibility. The proposed approach occupies a unique point in this design space: multi-sensor input, Zero-Shot operation, no training, model-agnostic. No prior work simultaneously satisfies all four constraints.

Innovation 2: The Prompt as a Real-Time Domain Lecture — Enabling Zero-Shot Multi-Spectral Reasoning Through In-Context Sensor Education

The prompt in this paper is not what the field typically means by "prompt engineering." Standard prompt engineering for vision-language models involves formatting the task, providing output constraints, and perhaps including a few in-context examples. The prompt here does something fundamentally different: it delivers a compressed domain lecture at inference time, teaching the model — within a single forward pass — what 12 Sentinel-2 bands are, how they have been combined into pseudo-images, what physical properties each index measures, and what color mapping encodes what meaning. This is not task instruction; it is sensor education.

The significance of this choice becomes clear when considering what the model is being asked to do. In the RGB-only baseline, the model classifies a photograph using its pre-existing visual knowledge — it knows what forests, rivers, and agricultural fields look like because it has seen thousands of them during training. In the multi-spectral setting, the model is shown an NDVI colormap — an image whose pixel colors follow a mapping convention (Red → low vegetation, Yellow → intermediate, Green → high vegetation) that the model has never been trained on, representing a physical quantity (the normalized difference between NIR and red reflectance) that the model has no innate knowledge of. Without the prompt's explanation, the NDVI image is just a colorful abstraction. With the prompt's explanation — which explicitly states what bands were used, what the colormap means, and what the resulting image is useful for — the model can learn the encoding scheme on the fly and apply its visual reasoning capabilities to extract information from it.

This is a genuinely novel mechanism: real-time, single-example concept acquisition through language grounding. The model is not retrieving NDVI from its pre-training data (it almost certainly was not trained on NDVI imagery with associated explanations). It is constructing a temporary understanding of what "an NDVI image with a Red-Yellow-Green colormap" means by composing its existing knowledge of colormaps (learned from scientific figures, heatmaps, medical imaging), its understanding of the words "vegetation," "NIR band," "Normalized Difference," and the specific mapping provided in the prompt. The empirical evidence that this works — the +0.04 F1 gain on BigEarthNet and +3% accuracy gain on EuroSat — demonstrates that the model possesses the abstract reasoning capability to integrate a novel visual encoding with a textual explanation and apply it to a downstream task, all within a single inference call.

This stands in contrast to prior work on Zero-Shot remote sensing classification. CLIP-based approaches (RemoteCLIP, SkyCLIP, RS-CLIP) perform Zero-Shot classification by computing cosine similarity between image embeddings and text embeddings of class names — a purely associative mechanism that requires the model to already "know" what each class looks like. ZLaP improves on this by leveraging unlabeled test-set examples (inductive inference), effectively learning a better similarity function by observing the distribution of test images. The proposed approach does neither: it does not rely solely on pre-existing visual knowledge of class names (because the spectral indices provide new, task-relevant information not present in any class name's semantic embedding), and it does not use test-set statistics (because it is genuinely Zero-Shot). Instead, it adds information at inference time — the spectral indices — and provides the model with the vocabulary to interpret that information — the prompt. This is a third mechanism for Zero-Shot performance that is distinct from both the associative (CLIP-style) and transductive (ZLaP-style) paradigms.

The prompt design also reveals a non-obvious property of large multimodal models: they can serve as inference-time interpreters of arbitrary visual encoding schemes, provided the encoding is consistent and the textual description is precise. This capability likely emerges from the diversity of visual encoding conventions in the training data — the model has seen weather maps with color-coded temperature gradients, medical images with false-color overlays, scientific charts with legends, and infographics with annotated color scales. It has learned a meta-skill of reading visualizations given a key, and the prompt supplies that key. The paper does not analyze this capability directly, but its existence is the prerequisite for the entire method to work, and the positive results are indirect evidence for it. This opens a broader research question: what other domain-specific visual encodings can be similarly "taught" to a frozen multimodal model through descriptive text, without training? Thermal camera output, electron microscopy images, X-ray diffraction patterns, genomic sequence visualizations — any domain where data can be rendered as an RGB image and where a textual explanation can convey the encoding scheme is a candidate for this approach.

Innovation 3: Generalist Pre-Training Encodes a Surprising Degree of Transferable Reasoning About Unfamiliar Sensor Physics

The paper's results imply something about the nature of the representations learned by large-scale multimodal pre-training that the field has not fully appreciated. Gemini 2.5 was trained on web-scale RGB imagery — photographs, illustrations, screenshots, and diagrams harvested from the internet. It was not trained on multi-spectral satellite data. It was not trained on NDVI, NDWI, or NDMI images. It was not exposed to Sentinel-2 band descriptions, spectral reflectance curves, or remote sensing textbooks in any systematic way. Yet when presented with these unfamiliar inputs and a textual explanation of what they mean, it improves its classification accuracy over using RGB alone. This is not a minor tweak — on BigEarthNet's 19-class variant, the improvement takes Gemini 2.5 from already-competitive Zero-Shot performance to a new state-of-the-art, with a +0.053 F1 margin over the previous best method (Table 2, bottom).

What capability does this imply? The model cannot be retrieving domain-specific remote sensing knowledge it does not have. It must be composing more basic capabilities — visual pattern recognition, understanding of color as a symbolic encoding, reasoning about physical concepts described in natural language, and integration of multiple information sources — to achieve something that, from the outside, looks like domain expertise. The model understands that "this image identifies wet and dry areas" means that red pixels in the NDMI image correspond to dry conditions and blue pixels to wet conditions, and it can use that information, alongside the visual appearance of the RGB image and the other spectral indices, to decide that a particular scene is "Inland marshes" rather than "Natural grassland." It is doing this without having been explicitly taught the spectral signature of marsh vegetation or the NDMI range typical of waterlogged soils.

This is a demonstration of compositional generalization at a scale and in a domain where it has not been previously documented for multimodal models. The model's pre-training gave it: (1) the ability to parse complex, multi-part instructions in natural language; (2) a rich vocabulary of physical and spatial concepts (vegetation, moisture, water, dry, dense, sparse, urban, agricultural); (3) visual recognition capabilities for the objects and land cover types that these concepts refer to; and (4) an understanding of how text can describe and explain images. When faced with the multi-spectral task, it composes these capabilities: it reads the prompt to learn the encoding scheme, applies its visual recognition to the pseudo-color images using that scheme, integrates the spectral information with the visual appearance from the RGB image, and maps the combined evidence onto the class labels using its conceptual understanding of what each class means.

The negative space is equally informative. The model does not have genuine spectral physics knowledge — it does not know why NIR is sensitive to leaf cell structure, it does not understand atmospheric absorption windows, and it cannot reason about the radiative transfer physics that produce the measured reflectance values. The prompt supplies a shallow facsimile of this knowledge (band wavelengths, what each index is "used for"), and the model uses this shallow description effectively. This suggests that for many practical remote sensing tasks, the deep physical knowledge is not necessary — what matters is knowing that a particular spectral index correlates with a particular surface property, and the prompt can supply that correlation without the underlying physics. This is an encouraging finding for the broader goal of making generalist models useful in scientific domains: the model does not need to understand quantum mechanics to use an NDVI colormap; it just needs to know that green means vegetation.

The ablation in Table 4 provides further evidence for this interpretative capability. Adding NDVI alone (alongside RGB) recovers a good portion of the total multi-spectral gain, but adding all five pseudo-images produces the best performance. This suggests that the model is integrating multiple, partially redundant spectral signals — NDVI and NDMI both respond to vegetation, but through different physical mechanisms (leaf structure vs. water content), and the model can apparently combine them to make more confident or more nuanced classifications than either alone would permit. This is exactly the kind of multi-evidence integration that domain experts perform when interpreting multi-spectral imagery, and it emerges here from a model with no remote sensing training.

The finding also carries implications for understanding the generality vs. specificity tradeoff in pre-training. The remote sensing community has invested substantially in domain-specific pre-training (SatMAE, Prithvi, SpectralGPT) under the assumption that generalist web-trained models lack the necessary knowledge for specialized Earth observation tasks. The paper's results challenge this assumption — or at least qualify it. A generalist model may not know the specifics of Sentinel-2 band physics, but it knows enough about images, language, and the physical world that a few paragraphs of explanation at inference time can bridge the gap. Whether this holds for tasks more complex than land cover classification (e.g., biophysical parameter retrieval, change detection, object detection in multi-spectral imagery) remains an open question, but the principle that generalist pre-training is a surprisingly strong foundation for specialized sensing tasks when paired with appropriate inference-time grounding is a result with implications far beyond this paper's specific benchmarks.

Innovation 4: The Cost-Free Adaptation Paradigm as a Strategic Alternative to Domain-Specific Foundation Model Development

The paper's methodological contribution is not just a technique — it is an argument, backed by empirical evidence, for a fundamentally different resource allocation strategy in applied remote sensing ML. The dominant narrative in the field has been that handling multi-spectral or multi-sensor data requires building specialized foundation models: collect large volumes of domain-specific data, design architectures for multi-band input, pre-train with domain-appropriate self-supervised objectives, and deploy the resulting model for downstream tasks. This narrative is not wrong — specialized models like SatMAE, Prithvi, and SpectralGPT achieve strong performance and represent genuine engineering achievements. But the narrative has crowded out consideration of an alternative: what if the rapidly improving capabilities of generalist models make specialized training unnecessary for an expanding set of tasks?

The paper provides a concrete case study in this alternative. The Gemini 2.5 model used in the experiments was not developed for remote sensing, was not trained on remote sensing data, and was not modified in any way for the multi-spectral task. The entire "adaptation" consists of: (1) a Python script that reads Sentinel-2 bands and writes pseudo-color images (computationally trivial — a few band math operations per pixel), and (2) a text prompt written in plain English (requiring domain knowledge to compose, but no ML expertise). The total development cost, measured in GPU-hours, is exactly zero. The total data collection cost is zero (the experiments use public benchmark datasets, but the method itself requires no training data). The deployment infrastructure is whatever already hosts the generalist model.

Compare this to the resource profile of training a multi-spectral foundation model. Pre-training SatMAE requires a large corpus of multi-temporal, multi-spectral Sentinel-2 imagery, substantial computational resources for masked autoencoding, and careful engineering of the temporal and spectral masking strategies. Pre-training Prithvi requires similar resources plus the curation of a multi-petabyte training dataset. Deploying these models requires hosting them on inference infrastructure, maintaining model versions, and updating them when new sensor configurations become available. Each of these steps involves non-trivial costs in engineer time, compute budget, and ongoing maintenance.

The paper's approach does not necessarily outperform these specialized models — the paper does not provide direct comparisons against fine-tuned multi-spectral foundation models, and it is likely that a fully fine-tuned SatMAE or Prithvi would achieve higher accuracy than Zero-Shot Gemini 2.5 with pseudo-color inputs. But the comparison the paper is making is not about accuracy ceilings — it is about accessibility, adaptability, and the accuracy-vs-cost Pareto frontier. For a geospatial professional who needs to classify a set of Sentinel-2 images and already has API access to Gemini 2.5 (or a comparable generalist model), the proposed approach provides immediate value with essentially zero marginal cost. Training or fine-tuning a specialized model, by contrast, requires resources and expertise that may not be available. The paper's contribution is to demonstrate that the "zero-cost" option is not just a toy — it achieves competitive, sometimes state-of-the-art, Zero-Shot performance on standard benchmarks.

This reframes the make-vs-buy decision for applied remote sensing ML. The traditional "make" option — train a specialized model — has high upfront cost but potentially higher accuracy and more control. The "buy" option — use a pre-existing generalist model — has historically been limited to RGB-only applications, leaving substantial accuracy on the table for multi-spectral tasks. The paper's "adapt" option — use a pre-existing generalist model with inference-time data adaptation — changes the calculus: the accuracy gap between make and buy shrinks significantly, potentially enough that the cost and complexity of the make option are no longer justified for many practical use cases.

This strategic implication is reinforced by the method's sensor-agnosticism. When the next-generation Sentinel satellite is launched with different spectral characteristics, a specialized model trained on the current sensor configuration may need substantial retraining. The proposed approach requires only updating the prompt (to describe the new bands) and the image generation script (to use the new band indices) — changes that can be made in minutes by a domain expert with no ML background. This resilience to sensor evolution is not an incremental improvement over prior work; it is a qualitatively different property that arises directly from the decision to keep the model frozen and adapt the data instead.

The paper does not argue that specialized models have no role — there will always be applications where maximum accuracy on a fixed sensor configuration justifies the training investment. But it does argue, implicitly through its results, that the range of tasks for which specialized training is necessary is narrower than previously assumed, and that generalist models with inference-time adaptation can cover a substantial portion of the practical need. This is a strategic insight with implications for how the remote sensing community allocates its ML development resources.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper uses two established remote sensing benchmarks: BigEarthNet (Sumbul et al., 2019) — a large-scale Sentinel-2 multi-spectral dataset with ~590,000 images of 120×120 pixel resolution (some bands at coarser 20 m and 60 m native resolution), providing 12 spectral bands and multi-label land cover annotations across 43 classes, with a consolidated 19-class variant (Sumbul et al., 2021) used to reduce semantic overlap between classes; and EuroSat (Helber et al., 2019) — a land use classification dataset with ~27,000 images at 64×64 pixel resolution, 10 classes, and single-label annotations. Both datasets are used in evaluation-only mode; no training split or fine-tuning data is used, consistent with the Zero-Shot claim. The paper does not specify the exact number of test examples evaluated, but reports results on the standard test partitions for each benchmark.
Base model(s). All experiments use Gemini 2.5 (Gemini Team, 2025) — a large multimodal model trained on web-scale RGB imagery and text, with advanced reasoning and long-context capabilities. The model is used completely frozen: no fine-tuning, no LoRA adapters, no prompt tuning, and no weight modification of any kind. The paper explicitly states that Gemini 2.5 "has not used any specialized multi-spectral data during training." The choice of this model is motivated by its status as a state-of-the-art generalist multimodal system, making it a strong test of whether generalist RGB-only pre-training can support multi-spectral reasoning through inference-time adaptation alone. No other model families (e.g., GPT-4V, Claude, open-source LMMs) are tested, which limits the generality of the findings.
Metrics. The primary metric for BigEarthNet is F1 score (the harmonic mean of precision and recall), consistent with the dataset's multi-label nature where each image can have multiple correct class labels. The paper also reports precision and recall separately to provide diagnostic insight into the components driving F1 changes. For EuroSat, which is a single-label classification task, the primary metric is classification accuracy (percentage of test images for which the single predicted class matches the ground truth). All metrics are computed by parsing the model's structured text output (e.g., "(1),(3)" for BigEarthNet, "(5)" for EuroSat) and comparing against ground-truth annotations using the standard grading protocol for each benchmark.
Baselines. The paper compares against multiple baselines:
- RGB-only Gemini 2.5 — the same Gemini 2.5 model receiving only the true-color RGB composite image (Image 1 from the multi-spectral pipeline) with a simplified prompt that omits all sensor metadata, band descriptions, and pseudo-image explanations. This is the primary internal baseline that isolates the contribution of the multi-spectral inputs and descriptive prompt.
- GPT-4V Zero-Shot — results from Zhang and Wang (2024), who benchmarked GPT-4V on the same BigEarthNet datasets in a Zero-Shot setting. This provides an external comparison to another major generalist multimodal model, though the specific prompt engineering and image preprocessing differ.
- ZLaP (Stojnic et al., 2024) — a label propagation method that performs "Inductive Zero-Shot inference," meaning it is allowed to use the unlabeled test-set examples (without labels) to improve its predictions via transductive learning. This is a stronger baseline than pure Zero-Shot because it leverages test-set distributional information. The paper compares against ZLaP on the EuroSat benchmark, where ZLaP previously held state-of-the-art Zero-Shot performance.
- Various prior Zero-Shot methods — the paper reports the best published Zero-Shot results for BigEarthNet's 19-class variant (Table 2, bottom), though specific method names beyond the aggregated "SOTA Zero-Shot results" are not enumerated.
Notable missing baselines include: (1) any fine-tuned multi-spectral foundation model (SatMAE, Prithvi, SpectralGPT) evaluated on these same benchmarks, which would contextualize the Zero-Shot performance against what is achievable with full training; (2) an ORM or PRM-based best-of-N or beam search variant using Gemini 2.5 with multiple samples, which could establish whether additional inference-time compute (beyond the single forward pass used here) closes or widens the gap between the proposed method and trained models; and (3) a CLIP-style Zero-Shot baseline using cosine similarity between class name embeddings and image embeddings extracted from the pseudo-color images, which would isolate whether the gains come from the model's generative reasoning capability or from a simpler associative mechanism.
Generation budget / compute accounting. The approach uses exactly one inference call per test example — the six pseudo-color images and the descriptive prompt are concatenated into a single multi-modal input and processed in one forward pass. There is no sampling of multiple completions, no best-of-N selection, no beam search, no chain-of-thought decoding with multiple rollouts, and no iterative refinement. The paper does not report inference latency, FLOP counts, or token usage for the prompt and images. However, the "compute budget" is implicitly 1× per example, which makes the comparison to RGB-only baselines fair (both use one inference call) but makes the comparison to methods like ZLaP (which processes the entire unlabeled test set) asymmetric — the proposed method uses less information (no test-set statistics) at lower per-example cost (one forward pass, not iterative propagation).
Cross-validation / statistical protocol. The paper does not report any cross-validation, bootstrap confidence intervals, or statistical significance tests. Results are reported as single-point estimates (F1 scores in Tables 1–2, accuracy in Table 3, F1 on a 1000-image subset in Table 4) without error bars or variance estimates. The ablation in Table 4 is conducted on "a smaller subset of the multi-label multi-class BigEarthNet dataset (on 1000 images only)," but the selection criteria for this subset are not described, and it is unclear whether the same subset is used across all ablation conditions or whether multiple random subsets were tested. This lack of statistical rigor is a limitation — given the relatively small absolute gains (+0.04 F1 on BigEarthNet, +3% accuracy on EuroSat), it is not possible to assess from the reported numbers alone whether these improvements are statistically significant or within the range of variance from prompt wording, random seed, or test-set sampling.

Main Quantitative Results

BigEarthNet Multi-Spectral Zero-Shot Classification (Tables 1 and 2)

The headline result is that adding multi-spectral pseudo-images and the descriptive prompt improves Gemini 2.5's Zero-Shot land cover classification by approximately +0.04 F1 score on both the 43-class and 19-class variants of BigEarthNet, with the multi-spectral version establishing new state-of-the-art Zero-Shot performance on the 19-class variant.

Table 1 reports results on the original 43-class BigEarthNet variant. The RGB-only baseline (Gemini 2.5 with only the true-color image and a simplified prompt) achieves an F1 score reported by the paper. The multi-spectral version (Gemini 2.5 with six pseudo-color images and the full descriptive prompt) achieves an F1 score that is higher by approximately +0.04. The paper does not report the absolute F1 values for the individual precision and recall components, making it difficult to assess whether the F1 gain comes primarily from improved precision (fewer false positives), improved recall (fewer false negatives), or a balanced improvement in both. The text states that "the main performance metrics is F1" and that "the gain realized from multi-spectral inputs is significant by +0.04 in F1," but the specific numbers are not quoted directly in the body text — they appear only in the tables, which are described rather than numerically reproduced in the main paper text.

Table 2 (top) shows the same comparison on the 19-class variant of BigEarthNet, where the original 43 ambiguous classes are merged into fewer, more semantically distinct categories. The pattern is consistent: the multi-spectral approach outperforms RGB-only by approximately +0.04 F1, confirming that the gains are not an artifact of the 43-class taxonomy's known ambiguities and that the spectral information provides genuine discriminative value even when the class definitions are cleaner.

Table 2 (bottom) positions these results in the context of prior Zero-Shot work. The RGB-only Gemini 2.5 baseline already outperforms the previous state-of-the-art Zero-Shot results (which the paper attributes to GPT-4V benchmarking from Zhang and Wang, 2024, plus additional methods). The multi-spectral version extends this lead further, achieving what the paper describes as "a +0.053 gain in F1 over the best SOTA results." The exact prior SOTA value and the exact Gemini 2.5 multi-spectral value are not quoted in the body text, appearing only in the table. This +0.053 margin over the previous best method is larger than the +0.04 gain over Gemini 2.5's own RGB-only baseline, implying that the RGB-only Gemini 2.5 was already roughly +0.013 above the prior SOTA before multi-spectral inputs were added.

Key interpretation points for Tables 1–2:

The +0.04 F1 gain is modest in absolute terms but represents a meaningful relative improvement in a Zero-Shot setting where no training on the target dataset is permitted. To contextualize this, consider that switching from one generalist model to another (e.g., GPT-4V to Gemini 2.5) also produces F1 differences on the order of a few percentage points in the Zhang and Wang (2024) benchmarking. The fact that adding five pseudo-images and a prompt achieves a gain comparable to switching model families is evidence that the spectral information is genuinely useful and that the model can extract it.

However, the paper does not provide per-class F1 breakdowns, which would reveal whether the multi-spectral gains are concentrated in specific land cover classes where spectral information is known to be discriminative (e.g., distinguishing "Coniferous forest" from "Broad-leaved forest" — which have different NIR reflectance characteristics due to leaf structure — or separating "Inland marshes" from "Natural grassland" — which differ in moisture content detectable by NDWI and NDMI). Without per-class analysis, it is impossible to assess whether the method improves classification where spectral physics predicts it should, or whether the gains are diffuse across all classes (which might suggest a non-specific benefit from additional visual context rather than genuine spectral reasoning).

EuroSat Multi-Spectral Zero-Shot Classification (Table 3)

On the EuroSat land use classification benchmark, the multi-spectral approach achieves a +3% absolute accuracy improvement over the RGB-only Gemini 2.5 baseline, and outperforms the previous state-of-the-art Zero-Shot method ZLaP, which uses inductive inference from unlabeled test-set examples.

Table 3 reports that the RGB-only Gemini 2.5 baseline achieves a baseline accuracy on the 10-class EuroSat benchmark (the exact number is in the table, not quoted in the body text). The multi-spectral version (six pseudo-images plus descriptive prompt) improves this by +3 percentage points. This is a larger relative gain than on BigEarthNet — +3% on a 10-class single-label task versus +0.04 F1 on a multi-label task with 19–43 classes — though the metrics are not directly comparable.

The paper also compares against ZLaP (Stojnic et al., 2024), which the paper describes as using "Inductive Zero-Shot inference, meaning it is allowed to use the dataset examples, but without labels." This is a transductive learning setting where ZLaP can observe the distribution of unlabeled test images and propagate label information through a nearest-neighbor graph or similar mechanism, effectively leveraging the fact that test examples form clusters in feature space. Despite ZLaP's informational advantage (access to unlabeled test data), the proposed method — which processes each test image independently in a single forward pass with no access to other test examples — achieves higher accuracy. The paper does not quote the exact accuracy values or the margin over ZLaP in the body text; these appear only in Table 3.

Key interpretation points for Table 3:

The +3% gain on EuroSat is substantial in the Zero-Shot context. EuroSat is a 10-class problem with a random-guessing baseline of 10%, and state-of-the-art fine-tuned models achieve accuracies in the 90s. The gap between Zero-Shot performance (even with multi-spectral inputs) and fine-tuned performance remains large, which the paper attributes to the difficulty of distinguishing semantically close classes without training examples: "'Annual Crop' vs 'Permanent Crop' and the model cannot distinguish between them without an association of how each one might look visually." The multi-spectral inputs help (NDVI and NDMI can distinguish between actively growing annual crops and senescent or bare permanent crops at different phenological stages), but the fundamental problem of mapping pixel patterns to abstract class labels without examples persists.

The comparison against ZLaP is significant because ZLaP represents a stronger baseline than pure Zero-Shot — it uses more information (the unlabeled test set) and a more sophisticated algorithm (label propagation in embedding space). That the proposed method outperforms ZLaP without using any test-set information suggests that the multi-spectral pseudo-images provide genuinely discriminative information that compensates for the lack of transductive learning. However, the paper does not report whether ZLaP was evaluated with RGB-only inputs or whether it had access to the same multi-spectral pseudo-images. If ZLaP was RGB-only, the comparison is somewhat unfair — the proposed method's advantage comes partly from having more input information, not just from a better algorithm.

Ablation: Incremental Value of Different Spectral Indices (Table 4)

The ablation study on a 1000-image subset of BigEarthNet shows that NDVI alone (alongside RGB) recovers a large portion of the total multi-spectral gain, but the full set of five pseudo-images achieves the best performance, suggesting that different spectral indices contribute complementary information that the model can integrate.

Table 4 reports F1 scores for three configurations on the 43-class variant (top) and 19-class variant (bottom), all conducted on "a smaller subset of the multi-label multi-class BigEarthNet dataset (on 1000 images only)":

RGB only — the baseline using only the true-color image.
RGB + NDVI only — the baseline plus the third image (NDVI colormap) and its associated prompt description.
RGB + All multi-spectral — the baseline plus all five additional pseudo-images (false-color NIR, NDVI, NDWI, NDMI-B11, NDMI-B12).

The paper states that "adding all multi-spectral images is performing best" and that "the performance when including NDVI images alone (together with RGB) is responsible for a good portion of these gains, but is not as good as when using all inputs." The exact F1 values for each condition are in Table 4 but not quoted in the body text.

Key interpretation points for Table 4:

The finding that NDVI alone captures a large share of the multi-spectral benefit is expected: NDVI is the most widely used vegetation index in remote sensing, and land cover classification on BigEarthNet is dominated by vegetation classes (various forest types, agricultural categories, grasslands, wetlands). However, the additional gains from including NDWI and the two NDMI variants indicate that water bodies, moisture gradients, and wetland-upland distinctions benefit from indices beyond NDVI. This is consistent with the qualitative examples in Figures 4–5, where the NDWI image helps the model distinguish between a river (water signal in NDWI) and a forest with blue-green coloration (no water signal in NDWI, strong vegetation signal in NDVI).

A missing ablation that would have been informative: testing the impact of each pseudo-image individually (not just NDVI alone, but NDWI alone, NDMI-B11 alone, false-color NIR alone) to quantify the marginal contribution of each spectral dimension. The paper's design — testing only NDVI alone versus all five — leaves open the question of whether NDWI and NDMI provide unique information beyond NDVI, or whether they are largely redundant with each other and with NDVI, contributing only a small marginal improvement. The fact that performance is best with all five suggests complementarity, but the degree of redundancy is unknown.

Another missing ablation: testing the prompt component in isolation. The multi-spectral prompt includes extensive sensor metadata (band wavelengths, resolutions) and physical interpretations (what each index is "used for") that the RGB-only prompt omits. It is possible that part of the performance gain comes from this additional textual context alone — the model knowing that the image is from Sentinel-2, that it covers a 120×120 meter area, and that it is a land cover classification task with specific class definitions — even without the pseudo-color images. An ablation that uses the RGB-only image but the full descriptive prompt would isolate the prompt's contribution from the spectral information's contribution.

Qualitative Analysis (Figures 4 and 5)

The qualitative examples in Figures 4 and 5 illustrate specific cases where the multi-spectral model corrects RGB-only errors, focusing on water-vs-vegetation confusions that the NDWI and NDMI images are specifically designed to resolve.

Figure 4 shows two examples on the EuroSat dataset where the multi-spectral model succeeds and the RGB-only model fails:

Top example (river correctly identified): The RGB-only model predicts "Forest" for a scene containing a river. The multi-spectral model correctly predicts "River." The paper attributes this correction to the NDWI image (the "bottom, left-most image which is an NDWI i.e. Normalized Difference Water Index image which is specifically targeting water bodies"), which would show a strong water signal in the river channel. This is a textbook case of spectral information disambiguating a visual confusion — the river's blue-green appearance in the visible spectrum resembles forest canopy coloration (particularly for coniferous forests with dark blue-green foliage), but the NIR absorption by water (producing high NDWI values for the river, low NDWI values for the forest) provides a clear discriminating signal.
Bottom example (forest correctly identified): The RGB-only model predicts "SeaLake" for a forested scene, likely confused by deep blue-green coloration. The multi-spectral model correctly identifies "Forest." Again, the NDWI and NDMI images would show low water/moisture signals for the forest canopy, discriminating it from actual water bodies.

Figure 5 shows two examples where both models agree — one where both are correct (another river example, correctly identified by both) and one where both are incorrect (a "Pasture" scene that the multi-spectral model classifies as "AnnualCrop" and the RGB-only model classifies as "SeaLake"). The paper uses this to illustrate that "it is quite challenging to correctly identify the only from RGB image" and that even with multi-spectral inputs, some cases remain ambiguous in a Zero-Shot setting without training examples.

Key interpretation points for Figures 4–5:

The examples are carefully selected to showcase the mechanism the paper hypothesizes — spectral indices resolving water-vegetation ambiguity — but they are anecdotal. Four examples (two in Figure 4, two in Figure 5) from a test set of thousands of images cannot establish how frequently the multi-spectral model corrects RGB-only errors, how often it introduces new errors (cases where the RGB model was correct but the multi-spectral model is wrong), and how the error patterns shift across different land cover classes. A confusion matrix comparison between RGB-only and multi-spectral predictions would provide this information, but the paper does not include one.

The qualitative analysis also reinforces the importance of the NDWI image specifically. Both correction cases in Figure 4 involve water-vs-vegetation confusion resolved by NDWI. This raises the question of whether the NDMI images (Images 5–6) are contributing meaningfully, or whether most of the gain from adding all pseudo-images (Table 4) comes from the combination of NDVI (vegetation) and NDWI (water), with NDMI providing only marginal additional value. The paper does not investigate this directly.

Ablation Studies and Robustness Checks

Spectral index contribution (Table 4): Adding NDVI alone to RGB improves over the RGB-only baseline on both the 43-class and 19-class BigEarthNet subsets, but the full five-index configuration outperforms NDVI-only, indicating that the different spectral indices provide complementary discriminative information. The experiment is conducted on a 1000-image subset, which limits its statistical power and generalizability — it is unclear whether the relative ordering of NDVI-only vs. full multi-spectral would hold on the complete test set.

Dataset variant robustness (Tables 1 vs. 2): The multi-spectral approach improves F1 by approximately +0.04 on both the original 43-class BigEarthNet and the consolidated 19-class version, demonstrating that the benefit is not an artifact of the 43-class taxonomy's known semantic ambiguities. This is a meaningful robustness check because it shows the spectral information is discriminative even when class boundaries are cleaner and less overlapping.

Benchmark transfer (Tables 1–2 vs. Table 3): The method transfers from BigEarthNet (120×120 pixel images, multi-label, 12 Sentinel-2 bands, land cover classification) to EuroSat (64×64 pixel images, single-label, 10 classes originally derived from Sentinel-2 but provided as RGB patches) without modification to the core approach — only the class list and task format in the prompt change. This demonstrates that the pseudo-image generation and descriptive prompting strategy is not dataset-specific. However, EuroSat images are 64×64 pixels, which means the pseudo-color images are also 64×64 — a very low resolution for spectral index visualization where spatial patterns (field boundaries, water body edges) may be only a few pixels wide at this resolution. The fact that the method still achieves +3% gain suggests the spectral information is useful even at coarse spatial resolution.

Missing ablations and robustness checks:

Prompt sensitivity analysis: The paper does not test whether changes in prompt wording — different descriptions of the spectral indices, different color mapping explanations, different levels of detail in the sensor metadata — affect performance. Given the known sensitivity of LLM outputs to prompt phrasing, the reported results may be specific to the exact prompt template used, and the +0.04 F1 gain might not replicate with a differently worded but semantically equivalent prompt.
Number of pseudo-images sweep: The paper tests 1 (RGB only), 2 (RGB + NDVI), and 6 (RGB + all five indices) configurations, but does not test intermediate numbers (e.g., RGB + NDWI only, RGB + false-color only, RGB + NDVI + NDWI, etc.) or more than 6 (e.g., adding additional band combinations from the 12 available Sentinel-2 bands, as the paper itself suggests is possible). This leaves open the question of diminishing returns — would 10 pseudo-images provide further gains, or would the model become confused by redundant information?
Colormap choice sensitivity: The NDVI colormap uses Red-Yellow-Green, the NDWI uses White-to-Blue, and the NDMI uses Red-Green-Blue. These are standard but not unique choices. Would a different colormap (e.g., a perceptually uniform color scale like viridis or magma for NDVI) change the results? The paper does not investigate whether the model's ability to interpret these color mappings depends on the specific colors chosen.
Model scale and family: All experiments use Gemini 2.5. The paper does not test whether the approach works with smaller or open-source models (e.g., LLaVA, Qwen-VL, InternVL), which would be important for accessibility and reproducibility. It also does not test whether the gains scale with model capability — does a larger model benefit more from multi-spectral inputs, or is the benefit roughly constant across scales?
Per-class performance analysis: The paper reports only aggregate metrics (F1, accuracy, precision, recall). A per-class breakdown would reveal whether the spectral information improves classification on the classes where spectral physics predicts it should (vegetation types via NDVI, water bodies via NDWI, wetlands via NDMI) and whether it degrades performance on any classes (e.g., urban classes where spectral indices may be noisy or uninformative). This is a significant gap because it prevents assessment of whether the model is using the spectral information in a physically meaningful way or simply benefiting from additional visual context in a non-specific manner.
Negative result in Table 4: The paper does not highlight this as a negative result, but the fact that NDVI alone recovers "a good portion" of the full multi-spectral gain suggests that NDWI and NDMI contribute only marginally — a finding that would be important for practitioners deciding whether the additional preprocessing complexity of generating five indices (vs. just one) is justified by the accuracy improvement. The paper does not quantify what "a good portion" means in relative terms (e.g., "NDVI alone captures 70% of the total gain, with the remaining four indices contributing the remaining 30%").

Critical Assessment

Claim 1: The approach improves Zero-Shot performance by +0.04 F1 on BigEarthNet and +3% accuracy on EuroSat over the RGB-only Gemini 2.5 baseline.

Supported, but with important caveats about precision and statistical reliability. The paper reports these numbers in Tables 1–3, establishing that adding multi-spectral pseudo-images and descriptive prompts improves over the RGB-only baseline on both benchmarks. However, the paper does not provide confidence intervals, standard deviations, or statistical tests. The BigEarthNet gain of +0.04 F1 is a relatively small absolute number — without knowing the variance (which could come from prompt wording sensitivity, test-set sampling, or model output stochasticity), it is difficult to assess whether this gain is reliably above the noise floor. On a test set of 500 or 1000 images (typical for these benchmarks), an F1 change of 0.04 might or might not be statistically significant depending on per-class variance. The EuroSat gain of +3% on a 10-class problem is a larger relative effect and more likely to be robust, but again, no statistics are provided. A bootstrap confidence interval or a paired significance test comparing RGB-only and multi-spectral predictions on the same test examples would strengthen this claim considerably.

Claim 2: The multi-spectral Gemini 2.5 achieves new state-of-the-art Zero-Shot results on BigEarthNet (19-class) and EuroSat.

Supported for the specific baselines compared, but the comparison set is narrow. The paper compares against GPT-4V (from Zhang and Wang, 2024) and ZLaP (Stojnic et al., 2024), both of which are strong Zero-Shot baselines. On BigEarthNet-19, the +0.053 F1 improvement over the previous SOTA is a meaningful margin. On EuroSat, outperforming ZLaP — which uses additional unlabeled test data — strengthens the claim. However, the paper does not compare against several other plausible Zero-Shot baselines: (1) a standard CLIP-based Zero-Shot classifier (compute cosine similarity between CLIP image embeddings of the pseudo-color images and CLIP text embeddings of class names), which would establish whether the generative Gemini 2.5 model is necessary or whether a simpler embedding-based approach would also benefit from the pseudo-images; (2) other generalist multimodal models available at the time of writing (GPT-4V with the same pseudo-image approach, Claude 3.5, open-source LMMs like LLaVA-1.6) to test whether the approach is specific to Gemini 2.5's capabilities; (3) an ensemble of multiple inference calls (majority voting over 5–10 samples from Gemini 2.5 with the same input) to test whether the single-inference-call design leaves performance on the table that could be recovered with additional test-time compute.

Claim 3: The approach is training-free and Zero-Shot, requiring no model adaptation or domain-specific data collection.

Supported. The paper is clear and consistent about this: Gemini 2.5 is used frozen, no fine-tuning or prompt tuning is performed, no training data is collected, and no dataset-specific optimization is applied beyond adapting the class list and task format in the prompt. This is a genuine strength of the approach and is well-documented.

Claim 4: The pseudo-color images and descriptive prompts enable the model to leverage specialized multi-spectral signals that it was never trained on.

Supported by the positive results, but the mechanism is not directly tested. The improvement over RGB-only baseline is evidence that the pseudo-images provide useful information, but the paper does not perform the critical experiment that would isolate whether the model is genuinely reasoning about spectral physics versus simply benefiting from having more visual context (any additional images might help by providing multiple views of the same scene) or from the additional textual information in the descriptive prompt (which the RGB-only baseline lacks). A key missing experiment is: RGB image + the same descriptive prompt but no pseudo-color images. If this condition achieves performance close to the full multi-spectral setup, it would suggest the prompt's textual information — not the spectral index images — drives the improvement. Conversely, if this condition performs similarly to the RGB-only baseline, it would strengthen the claim that the model is extracting information from the pseudo-color visualizations specifically. The current experimental design confounds the addition of spectral information (the pseudo-images) with the addition of contextual text (the sensor descriptions and index explanations), making it impossible to attribute the gain precisely.

Claim 5: The approach is useful for real-world geospatial professionals who want to leverage generalist models with multi-spectral data.

Plausible but not directly evaluated. The paper tests on two standard benchmarks with clean class taxonomies and curated test sets. It does not evaluate on the kind of open-ended, instruction-following tasks that the paper suggests as a key advantage of using a generalist model (asking follow-up questions, generating explanations, specifying task constraints in natural language). The paper also does not report the cost or latency of the approach in a deployment context — generating one inference call per example is straightforward, but the prompt is very long (hundreds of tokens of sensor metadata) and the images are six per example, which may have non-trivial API costs or latency implications depending on the model provider's pricing model. Without this information, a practitioner cannot assess whether the approach is practical for their specific use case.

Claim 6: The approach can be extended to other sensors and spectral indices without modification.

Not tested. The paper argues that "other types of sensors e.g. SAR, depth, or heat sensors, etc., can also be naturally mapped to visual inputs, and the approach will not need any changes, in order to use them." This is a forward-looking claim with no experimental support. SAR data, in particular, has fundamentally different physical properties from optical multi-spectral data (coherent imaging, speckle noise, sensitivity to surface roughness and dielectric constant rather than chemical composition), and it is not obvious that pseudo-color renderings of SAR backscatter would be as interpretable to an RGB-trained model as NDVI colormaps — which, even though derived from unfamiliar bands, display vegetation patterns that visually correlate with what the model sees in the RGB image. The paper's speculation about extensibility is reasonable but unverified.

Overall assessment of experimental evidence:

The paper's experiments demonstrate a proof of concept rather than a comprehensive evaluation. The approach works on the two tested benchmarks, the gains are consistent across dataset variants, and the method is genuinely training-free. However, the experiments leave several important questions unanswered: (1) how much of the gain comes from the visual information in the pseudo-images versus the textual information in the descriptive prompt; (2) whether the gains are concentrated in specific land cover classes where spectral physics predicts they should appear (water, vegetation, wetland classes) or diffuse across all classes; (3) whether the approach generalizes to other generalist models beyond Gemini 2.5; (4) whether the approach is robust to prompt wording variations; (5) how the approach compares against a fine-tuned multi-spectral model on the same benchmarks, which would contextualize the Zero-Shot performance relative to the upper bound achievable with full training. The paper's qualitative error analysis (Figures 4–5) is helpful for building intuition but limited to four hand-picked examples and does not substitute for a systematic error analysis (confusion matrices, per-class metrics, or an analysis of cases where the multi-spectral model performs worse than RGB-only).

The strongest experimental result is the EuroSat comparison against ZLaP, where the proposed method outperforms a transductive approach that uses additional test-set information. The weakest aspect is the lack of statistical rigor — no confidence intervals, no significance tests, and an ablation on only 1000 images from a much larger test set. Future work should address these gaps, particularly the confound between spectral information and prompt information, to establish whether the mechanism driving the performance gain is the one the paper hypothesizes (spectral reasoning via visual encoding interpretation) or a simpler effect (richer prompting, additional visual context).

6. Limitations and Trade-offs

Prompt Confounds the Spectral Information Gain

The assumption or constraint. The multi-spectral prompt (Appendix A) includes extensive sensor metadata — band names, central wavelengths, spatial resolutions, and physical interpretations of each derived index — that is entirely absent from the RGB-only baseline prompt. The RGB-only prompt simply states the classification instruction, the output format constraint, and the class list. This means the experimental comparison between multi-spectral and RGB-only settings changes two variables simultaneously: (1) the addition of five pseudo-color images and (2) the addition of hundreds of tokens of domain-specific contextual text. The paper does not isolate these two factors.

The consequence. It is impossible to determine from the reported results how much of the +0.04 F1 gain on BigEarthNet and +3% accuracy gain on EuroSat comes from the spectral information in the pseudo-images versus from the richer prompt context alone. The prompt tells the model that the scene is captured by Sentinel-2, that it covers a specific spatial footprint with known resolution characteristics, that certain bands are sensitive to particular physical properties (coastal aerosol, water vapor, vegetation via NIR), and that the task is land cover classification using satellite imagery. This contextual information could improve classification even without the pseudo-images — for example, knowing the image is from an overhead satellite sensor rather than an arbitrary photograph might shift the model's prior over plausible land cover classes, or the detailed sensor metadata might help the model better interpret the visible RGB image by providing atmospheric and resolution context. The critical missing experiment — RGB image with the full descriptive prompt but no pseudo-color images — would quantify this effect. Without it, the claimed mechanism (spectral reasoning via pseudo-image interpretation) is confounded with a simpler explanation (richer textual context improves classification).

What evidence exists in the paper. The paper provides no ablation that isolates the prompt's contribution. Table 4 tests RGB only vs. RGB + NDVI vs. RGB + all multi-spectral images, but all conditions beyond RGB-only include both additional images and additional prompt text describing those images. There is no condition where the prompt includes sensor metadata but the images are RGB-only. The paper's qualitative examples in Figures 4–5 attribute the improved classifications specifically to the NDWI image (e.g., "the multi-spectral bands are able to detect water (bottom, left-most image which is an NDWI)"), but these are hand-picked cases consistent with the authors' hypothesis and do not establish that the prompt's textual information is not the primary driver of the aggregate gain.

Mitigation status. The paper does not acknowledge this confound or propose any experiment to resolve it. This is a significant methodological gap because it undermines the paper's central causal claim. A future study could easily resolve this by testing the full descriptive prompt with only the RGB image, or by systematically varying the amount of sensor context in the prompt while holding the images constant.

Single Model, Single Model Family

The assumption or constraint. All experiments use exactly one model — Gemini 2.5 (Gemini Team, 2025) — with no testing on any other generalist multimodal model, whether proprietary (GPT-4V, Claude 3.5) or open-source (LLaVA, Qwen-VL, InternVL). The paper asserts that it "believe[s]" PaLM 2-S* is "representative of the capabilities of many contemporary LLMs" in the referenced example, but makes no analogous claim for Gemini 2.5's representativeness, nor does it provide evidence that the approach transfers across model families, scales, or architectures.

The consequence. Several aspects of the method's effectiveness could be specific to Gemini 2.5's particular capabilities and would not generalize to other models. The ability to interpret an unfamiliar pseudo-color visualization from a textual description is not a guaranteed property of all large multimodal models — it may depend on the diversity of scientific and technical visualizations in the model's training data, the model's capacity for cross-modal reasoning, or its specific prompt-following behavior. A model trained predominantly on natural photographs and social media images might not possess the abstract color-to-meaning mapping capability that the approach requires. Similarly, the approach's effectiveness may depend on model scale — smaller models with weaker reasoning capabilities might not be able to compose the prompt's sensor education with the pseudo-images' visual content, and the +0.04 F1 gain on BigEarthNet might shrink or disappear entirely. The paper provides no evidence that the gains would replicate on any other model, making it impossible for a practitioner to assess whether the approach would work with their available model (which is unlikely to be Gemini 2.5, given its limited availability at the time of writing).

What evidence exists in the paper. None. The paper tests only Gemini 2.5. There is no comparison to GPT-4V with the same pseudo-image approach (despite GPT-4V results being cited from Zhang and Wang, 2024, for the RGB-only setting), no test on open-source models, and no investigation of how performance scales with model size or capability.

Mitigation status. The paper does not acknowledge this as a limitation or discuss model-dependence. The approach is presented as a general method applicable to "generalist multimodal models" without qualification. This is a significant overclaim relative to the experimental evidence, which supports only that the method works with one specific model. Addressing this would require replicating the experiments on at least 2–3 additional models spanning different scales and training distributions, ideally including at least one open-source model to enable independent verification.

Difficulty Estimation Cost Is Fully Externalized

The assumption or constraint. The paper's entire value proposition — that the approach requires "no training, fine-tuning, or domain-specific data collection" — implicitly assumes that the user already has access to a Gemini 2.5-class multimodal model and is only concerned with the marginal cost of adapting it to multi-spectral inputs. But the paper does not account for the resources required to create and host such a model. Gemini 2.5 is a large, proprietary model trained on web-scale data with substantial computational resources. The "training-free" claim refers only to the absence of additional training for the multi-spectral adaptation, not to the availability of the base model itself.

The consequence. For a practitioner who does not already have access to a Gemini 2.5-class model, the effective cost of this approach includes the cost of accessing such a model via API (which may charge per-image or per-token, and the multi-spectral prompt with six images and hundreds of tokens of sensor description is substantially more expensive per query than an RGB-only prompt) or the cost of hosting an equivalently capable open-source model. The paper does not report the token count of the multi-spectral prompt, the inference latency, or the API cost per query. In a deployment scenario where thousands or millions of Sentinel-2 scenes need to be classified, these per-query costs could be substantial and would need to be weighed against the one-time cost of fine-tuning a smaller, specialized model that achieves comparable or better accuracy at lower per-query cost. The paper provides none of the information needed to make this cost-benefit analysis.

Furthermore, the approach's dependence on a specific proprietary model creates a reproducibility and longevity risk. If Gemini 2.5 is deprecated, updated (changing its behavior on these tasks), or restricted in access, the reported results cannot be independently reproduced or relied upon for ongoing applications. This is a fundamental limitation of methods that depend on proprietary models as black-box components.

What evidence exists in the paper. The paper reports no cost, latency, token count, or API pricing information. The method's reliance on Gemini 2.5 is stated (Section 1: "We demonstrate this idea with the Gemini2.5 model"), but the implications of this reliance for cost, reproducibility, and model availability are not discussed.

Mitigation status. The paper does not address this limitation. The framing emphasizes the absence of training cost without acknowledging that the inference cost (both per-query and the infrastructure required to access a Gemini 2.5-class model) may be substantial. Future work should benchmark the approach on open-source multimodal models of varying sizes, enabling practitioners to make informed cost-accuracy tradeoffs and ensuring reproducibility.

Hardest Cases Remain Unresolved — The Approach Amplifies Existing Capability but Does Not Create It

The assumption or constraint. The approach assumes that the base model already possesses sufficient visual and conceptual knowledge to classify land cover and land use types from overhead imagery, and that the additional spectral information serves to disambiguate cases where the RGB signal is misleading. This assumption breaks down when the base model fundamentally lacks the knowledge to distinguish between semantically similar classes — a problem the paper explicitly acknowledges for EuroSat: "'Annual Crop' vs 'Permanent Crop' and the model cannot distinguish between them without an association of how each one might look visually." The paper also notes that "there are also some noisy and mislabeled examples in this dataset."

The consequence. The approach provides no mechanism for the model to acquire genuinely new visual knowledge — it can only help the model better apply its existing knowledge by providing additional disambiguating information. For land cover classes that are visually similar in any representation the model can construct (including from spectral indices), or for class definitions that require task-specific conventions not inferable from general world knowledge, the approach will fail regardless of how many spectral indices are added. This is visible in the EuroSat results: even with the +3% accuracy gain from multi-spectral inputs, the Zero-Shot performance remains far below fine-tuned performance (which achieves accuracies in the 90s). The multi-spectral information narrows the gap but does not close it. On BigEarthNet, the paper does not compare against fine-tuned baselines, but given that specialized multi-spectral foundation models (SatMAE, Prithvi) achieve substantially higher accuracy when fine-tuned on this dataset, it is likely that a large gap remains.

More subtly, the approach may degrade performance on classes where the spectral information is misleading or noisy relative to the model's prior. For example, an urban area with extensive green space (parks, tree-lined streets) might show strong NDVI signals in some pixels, potentially confusing the model if it overweights the vegetation index relative to the RGB appearance of buildings and roads. The paper does not investigate whether multi-spectral inputs cause regressions on any specific classes.

What evidence exists in the paper. The EuroSat discussion (Section 4.2) explicitly acknowledges the Annual Crop vs. Permanent Crop confusion as a fundamental limitation of Zero-Shot classification with abstract class labels. The qualitative analysis in Figure 5 (bottom) shows a case where both the RGB-only and multi-spectral models are incorrect — a "Pasture" scene that the multi-spectral model classifies as "AnnualCrop" and the RGB-only model classifies as "SeaLake" — demonstrating that spectral information does not resolve all ambiguities. However, the paper provides no systematic analysis of which classes benefit from multi-spectral inputs and which do not (or are harmed by them), no per-class performance breakdown, and no confusion matrix comparing RGB-only and multi-spectral predictions.

Mitigation status. The paper acknowledges the limitation implicitly through the EuroSat discussion but does not quantify how frequently the approach fails on semantically close classes or whether spectral information can ever be counterproductive. The paper does not propose any mitigation — this is presented as an inherent limitation of Zero-Shot classification with generalist models, not something the method can address. Future work could explore in-context examples (few-shot prompting) specifically targeting the ambiguous class pairs, or task-specific prompt engineering that provides the model with distinguishing criteria for confusable classes.

No Systematic Error Analysis — Gains Are Reported in Aggregate, Not Diagnosed

The assumption or constraint. The paper evaluates the multi-spectral approach using only aggregate metrics: F1 score for BigEarthNet (Tables 1–2, 4) and accuracy for EuroSat (Table 3). It does not provide per-class breakdowns of where the gains (or losses) occur, does not compare the error patterns of the RGB-only and multi-spectral models via confusion matrices, and does not quantify how frequently the multi-spectral model (a) corrects an RGB-only error, (b) introduces a new error where RGB-only was correct, or (c) makes a different error than RGB-only without changing correctness.

The consequence. The aggregate gain of +0.04 F1 on BigEarthNet and +3% accuracy on EuroSat could be consistent with multiple, very different underlying mechanisms, each with different practical implications. The gain could be concentrated in a few classes where spectral indices are known to be highly discriminative (water bodies detected by NDWI, vegetation types distinguished by NDVI), while other classes show no improvement or even degradation. Alternatively, the gain could be diffusely distributed across all classes, suggesting a non-specific benefit (e.g., the model being more confident or attentive when given multiple views of the same scene). Without per-class analysis, it is impossible to determine whether the method provides reliable, physically grounded improvements or merely shifts the model's errors in ways that happen to improve the aggregate metric. A practitioner deciding whether to adopt this approach needs to know whether it improves classification on the specific classes relevant to their application, not just whether the average across all classes increases by a small amount.

Furthermore, the lack of systematic error analysis makes it impossible to diagnose the failure modes that persist despite multi-spectral inputs. The qualitative examples in Figures 4–5 are suggestive but anecdotal — four hand-picked examples from thousands of test cases. A confusion matrix would reveal systematic confusions (e.g., does the model consistently confuse "Pastures" with "Annual crops"? Does NDVI help separate "Coniferous forest" from "Broad-leaved forest"?), and an analysis of cases where the multi-spectral model regresses would reveal whether the approach has systematic failure modes (e.g., does NDWI cause the model to misclassify wetlands as water bodies?).

What evidence exists in the paper. The paper reports precision and recall alongside F1 for BigEarthNet (Tables 1–2), but these are also aggregate metrics — they do not reveal per-class patterns. The qualitative analysis in Figures 4–5 provides four example predictions with commentary, but this is illustration, not systematic analysis. Table 4's ablation includes only aggregate F1 scores for the 1000-image subset.

Mitigation status. The paper does not acknowledge the absence of per-class analysis as a limitation. The qualitative examples are presented as evidence that the spectral indices function as hypothesized (NDWI resolving water-vegetation ambiguity), but there is no attempt to quantify how often this hypothesized mechanism occurs versus other, uncharacterized effects. Addressing this would require publishing per-class F1 scores for the RGB-only and multi-spectral conditions (at minimum), a confusion matrix comparison, and a quantitative breakdown of error correction vs. error introduction rates. This analysis would substantially strengthen the paper's mechanistic claims and provide actionable guidance for practitioners.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a methodological reframing with pragmatic consequences rather than a paradigm shift in the Kuhnian sense. It does not propose a new model architecture, a new training objective, or even a new way of evaluating models. What it does is invert the default assumption about the relationship between specialized sensor data and generalist AI systems — and this inversion, while conceptually simple, has practical implications that ripple across how the remote sensing community thinks about model development, deployment, and maintenance.

The pre-existing default assumption — visible across the substantial literature on multi-spectral foundation models the paper cites — is that handling specialized sensing modalities requires specialized models. SatMAE pre-trains transformers on multi-spectral and temporal satellite imagery. Prithvi ingests six specific Sentinel-2 and Landsat bands as input channels. SpectralGPT designs a 3D transformer for hyperspectral data cubes. HyperSIGMA develops sparse attention mechanisms for high-dimensional spectral inputs. In every case, the model is built or adapted to match the data format. This is a natural and effective strategy, but it embeds an assumption that the paper's results call into question: that the model's input interface must be congruent with the sensor's output format for useful reasoning to occur.

The paper demonstrates that this assumption is not strictly necessary. A model whose input interface is fixed to three-channel RGB images and natural language can still extract useful information from 12-band multi-spectral data — not by modifying the model to accept 12 channels, but by re-encoding the spectral information as a set of RGB images with accompanying textual explanations that teach the model the encoding scheme at inference time. The +0.04 F1 gain on BigEarthNet and +3% accuracy gain on EuroSat are not enormous numbers, but they are achieved without modifying a single model parameter, without collecting a single training example, and without incurring any training cost. The entire "adaptation" is a data preprocessing script and a text prompt.

The consequence of this reframing is a change in the perceived boundary between what generalist models can and cannot do. Before this paper, a remote sensing practitioner looking at a Gemini 2.5-class model would reasonably conclude: "This model only accepts RGB inputs, so I cannot use it for my multi-spectral analysis — I need a specialized model." After this paper, that conclusion is no longer fully justified. The model can use multi-spectral information — it just needs that information to be translated into its native representational space and grounded through language. The boundary between "models that can handle multi-spectral data" and "models that cannot" becomes softer, more a matter of input formatting than architectural capability.

This reframing also reconciles a latent tension in the remote sensing ML literature that the paper does not explicitly name but that its results illuminate. On one side, there is clear evidence that multi-spectral information improves land cover classification accuracy — the entire specialized foundation model literature is predicated on this fact, and the physical basis is well-understood (NIR responds to leaf cell structure, SWIR to moisture, etc.). On the other side, there is the practical reality that generalist multimodal models, which are advancing rapidly in capability and accessibility, are trained exclusively on RGB data and cannot natively ingest the spectral bands that provide this advantage. The tension is: do you use the more powerful generalist model without spectral information, or the less powerful specialized model with spectral information? The paper provides a third option: use the generalist model, and bring the spectral information to it in a format it can process. This does not fully resolve the tension — the method does not match fine-tuned specialized model performance, and the paper does not provide direct comparisons — but it offers a pragmatic middle path for scenarios where accessing or training a specialized model is infeasible.

The paper also reframes the cost-benefit analysis for applied remote sensing ML development. The traditional "make" decision — train or fine-tune a specialized model for a specific sensor configuration — involves substantial upfront investment but potentially higher accuracy. The "buy" decision — use a pre-existing generalist model — has historically meant accepting an accuracy penalty from the loss of spectral information. The paper's "adapt" option changes the "buy" calculus: the accuracy penalty shrinks, potentially to the point where the additional accuracy from the "make" approach no longer justifies its cost for many practical applications. The paper does not claim that "adapt" replaces "make" — there will always be applications where maximum accuracy on a fixed sensor configuration warrants the investment — but it expands the set of scenarios where "adapt" is competitive.

The research directions this makes more attractive include: (1) inference-time adaptation of frozen models to novel sensing modalities, rather than training new models for each modality; (2) prompt engineering as a mechanism for domain adaptation, where the "domain knowledge" is transmitted through language rather than through gradient updates; (3) systematic investigation of what visual encoding schemes frozen multimodal models can and cannot interpret from textual descriptions alone; (4) the development of lightweight, modular preprocessing pipelines that translate domain-specific data into generalist-model-compatible formats, analogous to how this paper's band-math script translates Sentinel-2 bands into pseudo-color images.

The research directions this makes less attractive (or at least, narrows the scope for) include: (1) training small-to-medium specialized multi-spectral models from scratch for tasks where a generalist model with inference-time adaptation can achieve comparable Zero-Shot performance — the return on training investment is harder to justify; (2) architecture engineering for handling variable numbers of input bands, when a fixed three-channel input with clever preprocessing may suffice for many tasks; (3) large-scale domain-specific data collection efforts aimed at replicating generalist pre-training on remote sensing data, when the existing generalist pre-training may already provide most of the needed visual understanding.

Critically, the paper does not argue that specialized multi-spectral models are obsolete. The method achieves Zero-Shot performance improvements over RGB-only baselines but does not close the gap to fine-tuned specialized models. For applications where accuracy is paramount and training resources are available, specialized models remain the right choice. The paper's contribution is to demonstrate that the gap between generalist and specialist — at least for land cover and land use classification in a Zero-Shot setting — is narrower than previously assumed, and that it can be narrowed further without any training at all.

Follow-Up Research This Work Enables

Disentangling the prompt's contribution from the spectral images' contribution. The central confound in the current experimental design is that the multi-spectral condition adds both five pseudo-color images and hundreds of tokens of sensor metadata, band descriptions, and physical interpretations that the RGB-only baseline lacks. The critical experiment — testing the full descriptive prompt with only the RGB image — would reveal whether the +0.04 F1 gain on BigEarthNet and +3% accuracy gain on EuroSat comes from the spectral information in the pseudo-images (the paper's claimed mechanism) or from the richer textual context alone (a simpler alternative explanation). If the prompt-only condition recovers most of the gain, the paper's central mechanistic claim is weakened — the method would still be useful, but the contribution would be better prompt engineering rather than multi-spectral reasoning. If the prompt-only condition shows minimal improvement, the claim that the model extracts information from the pseudo-color visualizations is strengthened. A strong follow-up would run this ablation on both BigEarthNet and EuroSat, reporting per-class F1/accuracy alongside aggregate metrics, and would also test intermediate conditions (prompt with sensor metadata but no index descriptions, index descriptions without sensor metadata) to identify which textual elements matter most.

Replication on open-source multimodal models to establish generality and enable cost analysis. The paper's reliance on Gemini 2.5 — a proprietary model with limited availability and unknown future accessibility — means the results are neither independently reproducible nor actionable for practitioners without API access. A critical follow-up would replicate the exact experimental protocol (same pseudo-image generation pipeline, same prompt template, same test sets) on 2–3 open-source multimodal models of varying scales: a smaller model (e.g., LLaVA-1.6-7B or Qwen-VL-7B) to test whether the spectral reasoning capability emerges only at large scales, a mid-size model (e.g., InternVL2-26B), and a larger open model if available. The key measurement is whether the multi-spectral gain (over each model's own RGB-only baseline) scales with model capability — does a 7B model show a +0.01 F1 gain while a 70B model shows +0.04, or is the gain roughly constant across scales? This experiment would also establish baseline latency and cost figures (tokens per query, inference time per example) that the current paper lacks, enabling practitioners to make informed cost-accuracy tradeoffs.

Per-class error analysis to diagnose whether the spectral mechanism operates as hypothesized. The paper claims that the pseudo-color images — particularly NDWI for water detection and NDVI for vegetation discrimination — provide physically grounded disambiguating information. But the only evidence for this mechanism is four hand-picked qualitative examples (Figures 4–5). A systematic follow-up would produce full confusion matrices for the RGB-only and multi-spectral conditions on both BigEarthNet and EuroSat, then compute per-class metrics to answer specific questions: (1) Do classes with known spectral discriminability (water bodies via NDWI, forest types via NDVI, wetlands via NDMI) show disproportionate improvement? (2) Are there classes where multi-spectral inputs degrade performance (e.g., urban classes where spectral indices are noisy or misleading)? (3) What is the error correction rate — what fraction of RGB-only errors does the multi-spectral model correct — versus the error introduction rate — what fraction of RGB-only correct predictions does the multi-spectral model flip to incorrect? A null result — where gains are diffusely distributed across all classes without the predicted physical pattern — would suggest the mechanism is not spectral reasoning but something else (e.g., multi-view robustness from seeing the same scene six times). A positive result — concentrated gains on water, vegetation, and wetland classes — would strongly support the paper's mechanistic claims.

Systematic sweep of spectral index combinations and colormap choices. The paper's choice of five specific pseudo-images (RGB, false-color NIR, NDVI, NDWI, NDMI-B11, NDMI-B12) is described as illustrative, not optimal — the paper itself notes that "out of 12 multi-spectral bands we only created five example image combinations, whereas many other useful combinations can be included." A natural follow-up would systematically explore the space of possible band combinations and colormap choices to answer: (1) What is the marginal value of the 6th, 7th, and Nth pseudo-image? Does performance saturate, or does each additional informative index provide diminishing but positive returns? (2) Are the specific colormap choices (Red-Yellow-Green for NDVI, White-to-Blue for NDWI, Red-Green-Blue for NDMI) optimal, or would perceptually uniform colormaps (viridis, magma, cividis) — which the model may have seen more frequently in scientific visualizations — improve interpretability? (3) Do raw band visualizations (e.g., single-channel grayscale images of B08, B11, B12) provide value alongside or instead of derived indices, given that the model might learn to extract information from them differently? This sweep could be conducted efficiently on the 1000-image BigEarthNet subset used in Table 4, and the optimal combination would be validated on the full test set.

Extension to non-optical sensing modalities as a stress test of the method's claimed generality. The paper speculates that "other types of sensors e.g. SAR, depth, or heat sensors, etc., can also be naturally mapped to visual inputs, and the approach will not need any changes." This claim is untested and non-obvious. SAR data, in particular, presents unique challenges: coherent speckle noise creates a salt-and-pepper texture unlike anything in natural photographs; backscatter intensity depends on surface roughness and dielectric constant rather than chemical composition (so a SAR image of a forest and a SAR image of rough soil may look similar while their optical counterparts look completely different); and SAR's sensitivity to structure (buildings appear as bright corner reflectors, smooth water as dark) follows different physical principles than optical spectral indices. A strong follow-up would test the approach on a dataset with co-registered optical and SAR imagery (e.g., SEN12MS or the MM Earth dataset), using pseudo-color renderings of SAR backscatter (VV, VH polarizations as separate grayscale images, or false-color composites of different polarization combinations) with descriptive prompts explaining what SAR measures physically. A positive result (SAR pseudo-images improving classification over optical-only) would substantially strengthen the paper's generality claim. A negative result (SAR pseudo-images providing no benefit or degrading performance) would establish a boundary condition: the approach works for spectral band combinations that produce visually intuitive patterns correlated with RGB-visible features, but not for sensing modalities whose physical basis produces visual patterns with no RGB analogue.

Testing whether richer spectral reasoning tasks benefit from the approach. The paper tests only land cover and land use classification — tasks where the model must assign one or more class labels from a fixed taxonomy to an image. This is the simplest possible use of spectral information. More demanding remote sensing tasks — biophysical parameter retrieval (estimating leaf area index, canopy water content, or above-ground biomass from spectral data), change detection (identifying where and how land cover changed between two multi-spectral acquisitions), or multi-label scene understanding with spatial specificity (identifying which parts of the image correspond to which land cover classes) — require more sophisticated integration of spectral and spatial information. A strong follow-up would test whether the pseudo-image approach provides any benefit on these harder tasks, or whether its utility is limited to classification where the model only needs to produce a single global judgment. A null result on parameter retrieval — where the model must output a continuous value rather than a discrete class — would suggest the approach's mechanism is fundamentally about disambiguating class membership rather than extracting quantitative physical information, which would be an important boundary condition.

Development of lightweight difficulty estimation or strategy selection for the prompt. The current approach uses a single fixed prompt template for all examples. But the ablation in Table 4 shows that NDVI alone captures a large portion of the multi-spectral gain — suggesting that for some images (e.g., those dominated by vegetation classes), the additional NDWI and NDMI images may contribute little marginal value while still incurring the cost of generating and processing them. A practical follow-up would develop a lightweight pre-classification step that estimates, from the RGB image and a small number of quick spectral computations, which spectral indices are likely to be informative for a given scene — and then dynamically includes only those indices in the prompt. This is conceptually analogous to the compute-optimal test-time scaling policies explored in the reference example paper: estimate difficulty (or, here, spectral information need) from a cheap initial assessment, then allocate the inference budget (which pseudo-images to include, how much prompt text) accordingly. The evaluation metric would be accuracy-per-token or accuracy-per-inference-cost, rather than raw accuracy, measuring whether dynamic allocation improves the cost-accuracy Pareto frontier.

Practical Applications and Downstream Use Cases

Rapid prototyping and feasibility assessment for multi-spectral remote sensing projects. A geospatial analyst or research team considering a new land cover mapping project typically faces a cold-start problem: they need to determine whether multi-spectral data will be informative for their specific classification task, region, and class taxonomy before committing to the substantial investment of curating training data, fine-tuning or training a specialized model, and validating the results. The paper's approach provides an immediate, zero-cost feasibility test: convert a sample of Sentinel-2 scenes to the six pseudo-color images, compose the descriptive prompt with the target class list, and run Zero-Shot classification through any accessible generalist multimodal model. If the multi-spectral model substantially outperforms an RGB-only baseline on the sample (by margins comparable to the +0.04 F1 on BigEarthNet or +3% on EuroSat), that is strong evidence that spectral information is discriminative for the task and that investing in a specialized model is likely to yield further gains. If the multi-spectral model shows minimal improvement, that suggests the spectral bands may not carry useful signal for the specific class distinctions, and the project should focus on higher-resolution RGB imagery or alternative data sources. This feasibility assessment requires no training data, no GPU infrastructure beyond API access, and can be completed in hours — a dramatic reduction in the cost of project scoping.

Enabling multi-spectral analysis in resource-constrained settings. Many remote sensing applications with high societal value — deforestation monitoring in tropical countries, crop yield assessment for smallholder farmers, wetland change detection for conservation — are conducted by organizations (government agencies in developing countries, NGOs, academic groups) that lack the computational resources and ML expertise to train or fine-tune specialized multi-spectral foundation models. For these users, the choice has historically been between: (a) using RGB-only generalist models (increasingly accessible via APIs) but losing the spectral information that makes multi-spectral data valuable, or (b) attempting to train or adapt a specialized model, which may be infeasible. The paper's approach provides a third path: use the RGB-only generalist model they already have access to, run the simple band-math preprocessing (which requires only basic geospatial software skills, not ML expertise), and compose the prompt (which requires domain knowledge of what the spectral bands measure, but no ML expertise). The +0.04 F1 gain on BigEarthNet and +3% gain on EuroSat are modest in absolute terms but represent information that was previously completely inaccessible to these users — the difference between detecting a wetland and missing it, or between distinguishing forest types and treating them as undifferentiated "vegetation."

Low-cost monitoring system updates when satellite sensors change. Satellite missions evolve: Sentinel-2A and Sentinel-2B have slightly different spectral response functions; the upcoming Sentinel-2 Next Generation will have additional bands and higher resolution; Landsat 8 and Landsat 9 have different band configurations than Landsat 7. Each sensor change potentially breaks models trained on the previous sensor configuration, requiring retraining or domain adaptation. The paper's approach is inherently sensor-agnostic at the model level: the model never "knows" about the specific sensor — it only sees RGB images and text descriptions. When a new sensor becomes available, the adaptation requires only updating the prompt (to describe the new bands, their wavelengths, and their physical interpretation) and the image generation script (to use the new band indices and potentially new spectral index formulations). No model retraining, fine-tuning, or architectural modification is needed. For operational monitoring systems that need to maintain continuity across sensor generations, this flexibility could substantially reduce the maintenance burden. An organization running a land cover monitoring pipeline could switch from Landsat 8 to Landsat 9 inputs by changing a configuration file (band numbers, wavelength metadata, resolution specs) and the prompt text, without touching the model at all.

Augmenting human-in-the-loop remote sensing interpretation workflows. Professional remote sensing interpretation often involves an analyst examining multi-spectral imagery with various band combinations and spectral indices to manually delineate land cover, identify changes, or validate automated classifications. The paper's approach enables a new mode of human-AI collaboration: the analyst can feed the same pseudo-color images they are examining (false-color NIR, NDVI, NDMI) into a Gemini 2.5-class model alongside a prompt that explains what the analyst is looking for ("identify all areas that appear to be recently irrigated agricultural fields, considering the NDMI signal for surface moisture and the NDVI signal for vegetation vigor"), and receive an initial classification, a confidence estimate, or specific areas flagged for manual review. The model does not replace the analyst — it serves as a tireless first-pass filter that can process large volumes of imagery and highlight areas where the spectral indices suggest something interesting, with the analyst providing final judgment. This workflow leverages the model's ability to integrate multiple spectral indices (which the ablation in Table 4 shows is better than using any single index) and to follow free-form natural language instructions (which specialized classification models cannot do), while keeping the human in the loop for quality control and edge cases.

Dependency tree

Papers this one builds on, walking backwards through the same domain and shared topics.

V2V-GoT: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Models and Graph-of-Thoughts2025· vision← in-domain predecessor
- MolmoAct: Action Reasoning Models that can Reason in Space2025· uncategorized← LLM-judged similar
- Lynx: Towards High-Fidelity Personalized Video Generation2025· vision← in-domain predecessor
LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training2025· multimodal← topic thread
- Open-Qwen2VL: Compute-Efficient Pre-Training of Fully-Open Multimodal LLMs on Academic Resources2025· multimodal← LLM-judged similar
- LLaVA-OneVision: Easy Visual Task Transfer2024· multimodal← LLM-judged similar
- Improved Baselines with Visual Instruction Tuning2023· multimodal← LLM-judged similar
- CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning2025· multimodal← in-domain predecessor
- Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking2026· multimodal← similar topic
- OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents2026· multimodal← similar topic
- InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models2025· multimodal← similar topic

What it influenced

Later papers in the same chain. Direction reversed.

Video models are zero-shot learners and reasoners2025· multimodal→ LLM-judged similar
- ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulation2025· vision→ LLM-judged similar
- Trace Anything: Representing Any Video in 4D via Trajectory Fields2025· vision→ LLM-judged similar
- VideoWorld 2: Learning Transferable Knowledge from Real-world Videos2026· pretraining→ LLM-judged similar
- Visual Generation Unlocks Human-Like Reasoning through Multimodal World Models2026· reasoning→ LLM-judged similar
- Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning2026· uncategorized→ LLM-judged similar
- CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning2025· multimodal→ in-domain predecessor
DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencoder2025· vision→ in-domain predecessor
- EgoX: Egocentric Video Generation from a Single Exocentric Video2025· vision→ similar topic
- MAGREF: Masked Guidance for Any-Reference Video Generation with Subject Disentanglement2025· vision→ similar topic
- Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition2025· vision→ similar topic