TRUST-SQL: Tool-Integrated Multi-Turn Reinforcement Learning for Text-to-SQL over Unknown Schemas

ArXiv: 2603.16448

🎯 Pitch

Text-to-SQL models trained on the standard assumption of pre-loaded schemas fail when deployed in real enterprise databases containing hundreds of tables, but we show an autonomous agent that actively explores and verifies schema metadata can match or surpass those privileged methods—without ever seeing the full schema upfront. The key is a novel training strategy that isolates the credit for successful exploration from the quality of the final SQL query, preventing the agent from hallucinating plausible table names and improving execution accuracy by nearly 10% over standard reinforcement learning.

---

1. Executive Summary

This paper introduces TRUST-SQL, an autonomous agent framework that reformulates Text-to-SQL parsing under the Unknown Schema setting — where databases contain hundreds of tables with noisy metadata and no pre-loaded schema — as a Partially Observable Markov Decision Process with a structured four-phase interaction protocol (Explore, Propose, Generate, Confirm). The core technical contribution is Dual-Track GRPO, a training strategy extending Group Relative Policy Optimization that uses token-level masked advantages to isolate exploration rewards from execution outcomes, resolving the credit assignment bottleneck that standard single-reward RL conflates across long interaction trajectories — yielding a 9.9% relative improvement in execution accuracy over standard GRPO on BIRD-Dev. Across five benchmarks with Qwen3-4B and Qwen3-8B base models, TRUST-SQL achieves average absolute improvements of 30.6% and 16.6% respectively over their base models under the Unknown Schema setting, and remarkably matches or surpasses strong baselines that rely on full schema prefilling — establishing that autonomous database exploration can substitute for privileged metadata access only when a structural cognitive checkpoint (the Propose phase) enforces verified schema grounding and prevents parametric hallucination, while on problems outside the agent's exploration horizon the benefit vanishes.

2. Context and Motivation

The Core Problem: The Full Schema Assumption Is a Fiction

Text-to-SQL parsing — translating natural language questions into executable SQL queries — has made remarkable progress in recent years. However, this progress rests on a premise that the paper argues is fundamentally incompatible with real-world enterprise environments: the Full Schema Assumption. Under this assumption, the complete database schema (all table names, column names, data types, and foreign key relationships) is pre-loaded into the model's input context before it generates a SQL query. The task then reduces to a static translation problem: given a question and a fully specified schema, produce the correct SQL.

This assumption fails catastrophically in actual enterprise settings for several reasons the paper identifies (Section 1):

• Scale overwhelms context windows. Enterprise databases routinely contain hundreds of tables. The BIRD benchmark alone includes databases with complex, multi-table schemas, and real production systems are far larger. Pre-loading every table name, column, and relationship quickly exceeds the finite context limits of even the largest language models.
• Metadata is noisy and distracting. Injecting massive amounts of schema information — much of it irrelevant to the current question — actively harms performance. Irrelevant or stale table structures distract the model during schema linking, increasing the probability that it selects wrong tables or columns when constructing the SQL query.
• Schemas evolve continuously. In production environments, tables are added, columns are renamed, and relationships are restructured through regular database migrations (Zhang et al., 2026). Pre-loaded schemas become stale, and models relying on them generate queries referencing columns that no longer exist, leading to execution failures that are invisible to models operating under the static translation paradigm.

The paper formalizes this gap as the Unknown Schema setting, depicted in Figure 1: an agent must actively explore an unobservable database to identify and verify only the relevant subset of metadata, rather than passively consuming a pre-provided schema. This is not a minor variant — it represents a fundamental paradigm shift from "the schema is given" to "the schema must be discovered."

Why This Problem Matters: Practical and Theoretical Stakes

The practical importance is immediate: as LLMs are deployed as database interfaces, the Full Schema Assumption constitutes a hard deployment blocker. The paper notes in Section 4.3 that base Qwen3 models experience a 17.0% absolute collapse in execution accuracy on BIRD-Dev when schema prefilling is removed — from 46.3% to 29.3% for the 4B variant. Without explicit mechanisms for active exploration, models simply cannot function in realistic environments where schemas are not provided upfront.

But the significance runs deeper than deployment practicality. The Unknown Schema setting forces a reconceptualization of the Text-to-SQL task itself. Under the Full Schema Assumption, a model's primary challenge is schema linking — mapping natural language references to the correct table and column names from a provided list. Under the Unknown Schema setting, the challenge expands to include discovery under partial observability: the agent does not know which tables exist, must decide what metadata to query, and must ground its SQL generation only in information it has actively verified. This transforms Text-to-SQL from a single-turn translation problem into a sequential decision-making problem under uncertainty, connecting it to the broader literature on tool-integrated reasoning and autonomous agents.

The paper also identifies a subtler consequence: schema prefilling not only provides necessary information but also encourages parametric hallucination. When a model has the full schema in context, it can sometimes correctly answer questions by pattern-matching without truly understanding the database structure. When that crutch is removed, the model's tendency to hallucinate non-existent table or column names based on its pre-training priors is exposed and must be explicitly countered. This connects the problem to the broader challenge of grounding LLM outputs in verifiable facts rather than parametric knowledge.

Where Prior Approaches Fall Short

The paper identifies three categories of prior work, each with specific limitations that the Unknown Schema setting exposes:

Single-turn methods under the Full Schema Assumption. The dominant paradigm in Text-to-SQL research — encompassing both supervised fine-tuning approaches (OmniSQL, Li et al., 2025; STAR, He et al., 2025; ROUTE, Qin et al., 2024) and single-turn reinforcement learning methods (Ma et al., 2025; Yao et al., 2025; Zhang et al., 2025; Pourreza et al., 2025) — treats the model as a passive translator. These methods optimize end-to-end SQL generation given a static input containing the question and the full schema. They internalize schema linking and SQL composition capabilities, but fundamentally lack any interactive capability. As the paper states in Section 2:

"Constrained to a single-turn interaction paradigm, these models act as passive translators. Consequently, they fundamentally fail in unobservable enterprise environments where active database exploration is strictly required."

The limitation is architectural, not merely a matter of training data: a single-turn model has no mechanism to say "I need more information" or to condition subsequent decisions on the results of prior metadata queries. Even if trained on perfect data, it cannot adapt to novel schemas it has never seen because it cannot ask about those schemas.

Training-free tool-augmented frameworks. Recent work (MAC-SQL, Wang et al., 2025a; TASQL, Wang et al., 2024) has explored equipping frozen LLMs with tool access for database exploration. These agents can query metadata by executing SQL statements against system tables, then iteratively refine their queries based on execution feedback. However, the paper identifies a critical limitation: without gradient-based training, these agents "remain susceptible to parametric hallucinations and cannot strictly enforce verification protocols" (Section 2). A frozen model prompted to explore a schema may still fabricate table names when the search space is large or when the prompts fail to constrain its behavior — a phenomenon the pilot study in Section 3.1 quantifies, showing that without explicit structural constraints, hallucination accounts for 26.4% of all failures in a basic Explore-Confirm agent.

The issue is not that frozen models cannot use tools, but that they have no learned incentive to verify before generating. The Propose phase introduced in TRUST-SQL is not merely a prompting convention — it is enforced through reinforcement learning, backed by a schema reward that penalizes unverified metadata. Training-free approaches have no mechanism to learn this discipline.

Multi-turn RL with conflated rewards. The most recent and directly comparable work applies multi-turn reinforcement learning to Text-to-SQL. MTIR-SQL (Xu et al., 2025) and SQL-Trail (Hua et al., 2026) embed SQL execution into the training loop, allowing models to learn from the consequences of their queries across multiple interaction turns. However, the paper identifies a fundamental credit assignment problem that these approaches fail to resolve. As stated in Section 2:

"By relying on a single terminal reward or naively aggregating intermediate signals, these methods conflate the quality of schema exploration with SQL generation, making it impossible to attribute the final execution outcome to specific actions."

Consider a trajectory where the agent correctly explores the schema and proposes verified metadata, but generates a logically flawed SQL query. A single terminal reward of 0 (execution failure) penalizes the entire trajectory equally, including the exploration phase that was actually correct. Conversely, if the agent proposes an incomplete schema but happens to generate correct SQL by coincidence, the terminal reward rewards the poor exploration. This conflation of two distinct sub-tasks — schema discovery and query composition — prevents the RL algorithm from learning that schema verification is independently valuable. The model has no signal that proposing verified metadata is good regardless of whether the SQL succeeds.

The Credit Assignment Bottleneck in Detail

The paper's framing of this as a credit assignment problem deserves deeper analysis because it motivates the entire Dual-Track GRPO design. In a standard multi-turn RL trajectory for Text-to-SQL, the sequence of actions includes:

1. Metadata queries (Explore actions) — decisions about which tables and columns to investigate.
2. A schema commitment (Propose action) — the agent declares what it believes to be the relevant schema.
3. SQL generation (Generate action) — the agent produces a candidate query.
4. Answer submission (Confirm action) — the agent finalizes its output.

The final execution outcome — whether the SQL produces the correct result — depends on the quality of all these actions. But the actions serve fundamentally different purposes: steps 1–2 are about information gathering and grounding (does the agent correctly identify the necessary schema elements?), while steps 3–4 are about query composition (does the agent correctly express the user's intent in SQL?). A failure in step 1 (missing a necessary table) and a failure in step 3 (incorrect JOIN logic) both produce execution failure, but require very different learning signals to fix.

Standard GRPO (Group Relative Policy Optimization), which TRUST-SQL extends, computes an advantage for each trajectory by comparing its reward to the mean reward of a group of sampled trajectories. This advantage is then broadcast to all tokens in the trajectory. For the execution reward, every token — whether part of a correct exploration action or a flawed SQL generation — receives the same positive or negative signal. The paper's key insight is that the Propose phase provides a natural structural boundary that enables separating these signals: everything before the Propose belongs to schema exploration, and everything after belongs to query generation. By computing separate advantages for each sub-trajectory with independent rewards — the Schema Reward for exploration quality and the Execution Reward for generation quality — Dual-Track GRPO gives each action type its own, non-conflated optimization signal.

The pilot study in Section 3.1 empirically motivates this separation. The authors constructed three agent variants with increasing structural constraints:

• EC (Explore-Confirm): The minimal baseline. The agent freely queries metadata and directly submits SQL without intermediate verification. Hallucination accounts for 26.4% of all failures.
• EGC (Explore-Generate-Confirm): Adds an explicit Generate phase where the agent executes a candidate SQL and observes the result before confirming. Hallucination drops to 14.2% — execution feedback provides a weak correction mechanism.
• EPGC (Explore-Propose-Generate-Confirm): Adds the Propose phase as a mandatory cognitive checkpoint. Hallucination plummets to 2.8% — a 9.4× reduction from EC.

The Propose checkpoint works because it forces the agent to explicitly commit to a verified schema before generating SQL. This makes the exploration process auditable (the agent declares exactly what it believes the relevant schema to be) and creates a clear boundary for credit assignment during RL training.

However, Obs. 2 from the pilot study reveals a critical complication: Schema linking errors remain consistently high across all variants. Even when hallucination is nearly eliminated by the Propose checkpoint, the agent still selects wrong or missing tables and columns. This means that suppressing hallucination is necessary but insufficient — the agent also needs to learn better exploration strategies, which requires an independent optimization signal for the schema discovery phase. This directly motivates the dual-track approach: the Schema Track receives its own reward and advantage computation, allowing the model to learn what constitutes good exploration regardless of whether the subsequent SQL generation succeeds.

How TRUST-SQL Positions Itself

The paper positions TRUST-SQL at the intersection of three research threads — each partially addresses the Unknown Schema challenge, but none provides a complete solution:

Against single-turn methods, TRUST-SQL argues that interaction is not optional but necessary. The Unknown Schema setting cannot be reduced to a smarter static translation, because the agent literally does not know what tables exist at the start of the interaction. Any method that cannot query the database cannot solve this problem.

Against training-free tool-augmented frameworks, TRUST-SQL argues that gradient-based training is required to enforce verification discipline. Prompting alone, no matter how carefully engineered, cannot prevent a model from hallucinating when the alternative (querying metadata) requires effort and the model's parametric priors offer shortcuts. The Prove checkpoint must be learned as a policy, not merely suggested as a convention.

Against prior multi-turn RL approaches, TRUST-SQL argues that the credit assignment problem must be explicitly resolved through structural decomposition. Simply adding more interaction turns with a single terminal reward does not teach the model to distinguish good exploration from good generation, because the reward signal conflates both.

The paper's contribution is therefore not a single technique but an integrated framework: the four-phase protocol provides the architectural scaffolding for structured exploration, while Dual-Track GRPO provides the algorithmic machinery for learning within that scaffolding. The POMDP formulation elevates this from an engineering solution to a principled framework: the database schema is the hidden state, metadata queries are observations, and the Propose checkpoint is the point where the agent's belief state (its verified schema knowledge) is explicitly crystallized and made available for credit assignment.

3. Technical Approach

3.1 Reader Orientation

TRUST-SQL is an autonomous agent system that actively explores an unseen database to gather the specific metadata needed to answer a natural language question, rather than relying on a pre-injected schema. It solves the problem of Text-to-SQL parsing when the database structure is initially hidden (the Unknown Schema setting) by combining a structured four-phase interaction protocol with a reinforcement learning strategy that separately optimizes two sub-skills: discovering the right schema elements, and writing the correct SQL query.

3.2 Big-Picture Architecture (Diagram in Words)

The system has four major components working together in a closed loop:

1. The Agent (Policy Model $\pi_\theta$). A fine-tuned LLM (Qwen3-4B or Qwen3-8B) that decides what action to take at each step. It maintains an internal context—the question, the history of interactions, and the Verified Schema Knowledge ($K_t$)—and outputs structured actions.
2. The Four-Phase Action Protocol. A rigid behavioral scaffold enforced by training. The agent can only choose from four actions: explore_schema (query database metadata), propose_schema (commit to a verified schema), generate_sql (write a candidate SQL query and execute it to see results), and confirm_answer (submit the final SQL).
3. The Training Engine (Dual-Track GRPO). An extension of Group Relative Policy Optimization (GRPO) that splits each interaction trajectory at the propose_schema action into two tracks. It assigns an independent reward and advantage to each track—a Schema Reward for exploration quality and an Execution Reward for the final answer—and uses token-level masking to optimize them separately without mixing signals.
4. The Environment (Database and Tools). The unobservable database that reacts to SQL queries. For explore_schema, it returns metadata (table names, schemas, sample values). For generate_sql, it returns query results. This feedback forms the agent's observations.

Information flows in a cycle: the Agent proposes an Action $\rightarrow$ the Environment executes it and returns an Observation $\rightarrow$ the Agent updates its internal context $\rightarrow$ the cycle repeats until the Agent submits a Confirm action.

3.3 Roadmap for the Deep Dive

This explanation will follow the conceptual stack of the system, from the formal problem it solves down to the specific loss function that trains it. This order helps because each layer provides the motivation and vocabulary for the next:

• First, the POMDP Formulation (Section 3.2): The mathematical model of the problem. This defines what constitutes a state, action, observation, and reward in an environment where the database schema is hidden.
• Second, the Four-Phase Protocol: The architectural solution to the POMDP. This explains the design and strict rules of the agent's action space, and presents the pilot study that empirically justifies why the Propose checkpoint is necessary.
• Third, the Reward Components (Section 3.3): The training signals. This details the three distinct rewards—Execution, Format, and Schema—and their precise definitions, which are the foundation for the optimization algorithm.
• Fourth, Dual-Track GRPO (Section 3.4): The optimization framework. This explains how Group Relative Policy Optimization is adapted to create two separate learning tracks, how token-level masked advantages solve the credit assignment problem, and how the final combined loss function is constructed.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and training methodology paper whose core idea is that Text-to-SQL over unknown schemas requires a structured interaction protocol whose phases serve as explicit boundaries for a multi-track reinforcement learning objective, thereby isolating the distinct challenges of schema discovery and query generation.

---

Problem Formulation: The Unknown Schema POMDP

The paper frames the task not as a single-step translation but as sequential decision-making under uncertainty. This formalism is the theoretical bedrock for the entire design.

The core idea is that the database schema is a hidden state, and the agent can only perceive it through incomplete, query-based observations. The task is therefore formalized as a Partially Observable Markov Decision Process (POMDP). A POMDP models situations where an agent's knowledge of the world is incomplete, and it must actively gather information through actions that provide partial glimpses of the true state.

Formally, the POMDP is defined by a tuple $(\mathcal{S}, \mathcal{A}, \mathcal{T}, R, \Omega, \mathcal{Z}, \gamma)$ over discrete time steps $t = 0, 1, \ldots, T$.

State and Observation Spaces ($\mathcal{S}$, $\Omega$, $\mathcal{Z}$). The true, hidden state $s_t \in \mathcal{S}$ represents the complete, ground-truth database schema—every table, column, data type, and relationship. This state is never directly visible to the agent. Instead, at each step, the agent receives a partial observation $o_t \in \Omega$ dictated by the observation function $\mathcal{Z}$. The observation consists exclusively of the results of a specific query the agent just executed (e.g., the list of table names, the schema of a single table, or the result set of a trial SQL query). To manage this uncertainty, the agent relies on an internal context state $c_t = (q, h_t, K_t)$, which is its own summary of everything it knows. $q$ is the original user question (an immutable constant). $h_t$ is the full interaction history up to step $t$—the sequence of actions and observations the agent has taken and received. $K_t$ is the Verified Schema Knowledge, a living document that stores only the metadata the agent has explicitly verified through explore_schema calls. It is initialized as an empty set ($K_0 = \emptyset$), forcing the agent to start from zero knowledge.

Action Space ($\mathcal{A}$). To prevent the agent from taking shortcuts that could lead to hallucination, its action space is strictly constrained to four categories, which are executed in a structured sequence. The policy $\pi_\theta(a_t | c_t)$ selects an action based on the current internal context. The four actions are:

1. Explore (explore_schema): Queries database metadata. This is how the agent populates $K_t$ by asking the environment about available tables, columns, or sample data values.
2. Propose (propose_schema): Acts as a mandatory cognitive checkpoint at a specific step $t_{\text{propose}}$. The agent commits to a final, verified schema $\hat{K} = K_{t_{\text{propose}}}$ based on its exploration. This output is a structured JSON-like specification of tables, columns, and potential joins.
3. Generate (generate_sql): Produces a candidate SQL query that is strictly grounded in the schema $\hat{K}$ it just proposed. This action also executes the candidate query against the database, and the results become the observation $o_t$, allowing the agent to see if the query runs and what data it returns.
4. Confirm (confirm_answer): Submits the final, validated SQL query $y$ at the terminal step $T$. This action ends the interaction.

Transition and Objective. When the agent executes an action $a_t$, the environment transitions to a new true state $s_{t+1}$ according to the transition function $\mathcal{T}$ and emits an observation $o_t$. The agent updates its context to $c_{t+1}$. A complete interaction sequence from the agent's perspective is a trajectory $\tau = \{(c_t, a_t, o_t)\}_{t=0}^T$. The fundamental goal is to learn a policy $\pi_\theta$ that maximizes the expected cumulative discounted reward.

The paper does not provide a single equation for the cumulative return in the POMDP section, but introduces it in the context of the RL objective in Section 3.4. However, it's implicitly the standard POMDP objective, which we can state clearly here for completeness:

$J(\theta) = \mathbb{E}_{\tau \sim \pi_\theta}\left[\sum_{t=0}^T \gamma^t R(c_t, a_t)\right]$

where $\gamma \in [0, 1]$ is the discount factor (though the paper's primary rewards are terminal and sparse, making $\gamma$ effectively 1 for the final reward and 0 for intermediate rewards except where Schema Rewards are applied).

What it computes: the expected sum of discounted rewards over a trajectory generated by the policy $\pi_\theta$. The expectation $\mathbb{E}_{\tau \sim \pi_\theta}$ means we average over many possible interaction sequences the agent might generate for a given question and database. The inner sum adds up the reward $R$ received at each step, with future rewards worth slightly less (discounted by $\gamma^t$). This single scalar $J(\theta)$ is the value the agent is trained to maximize.

Why this form: This is the canonical objective for reinforcement learning in sequential decision-making problems. The alternative—a single-turn supervised objective—would be correct only if the optimal action at each step was independent of the history. In the Unknown Schema setting, the generate_sql action's success is causally dependent on the quality of the preceding explore_schema and propose_schema actions. The POMDP formalization with a cumulative objective is necessary to train the entire multi-step process jointly, allowing a good exploration strategy to be reinforced even when it's many steps removed from the final success signal.

---

The Four-Phase Interaction Protocol

The POMDP defines the problem, and the four-phase protocol is the architectural solution. It is not just a prompting template; it is a rigid behavioral schema that is enforced at both inference and training time.

The protocol's structure was empirically derived from a pilot study, which is crucial for understanding why it has this specific four-phase shape and not a simpler one. The study tested three agent variants on the BIRD-Dev dataset using Qwen3-8B as a base model, each with increasingly stringent structural constraints on its interaction behavior.

Protocol Variants in the Pilot Study (Section 3.1). The authors built three agents to isolate the effect of each added phase:

1. EC (Explore-Confirm): The simplest interactive baseline. The agent is allowed to freely query metadata using an explore_schema tool. After exploring, it can directly submit a final SQL answer with a confirm_answer action. There are no intermediate verification steps—no explicit schema proposal and no trial execution of a candidate SQL.
2. EGC (Explore-Generate-Confirm): Adds an explicit generate_sql phase. After exploring, the agent must execute a candidate SQL query and observe its result before it can proceed to confirm_answer. This introduces a weak feedback loop; if the query is malformed or returns unexpected results, the agent can see this and potentially loop back to explore more.
3. EPGC (Explore-Propose-Generate-Confirm): The full TRUST-SQL protocol. It adds a propose_schema phase between exploration and generation. The agent is forced to commit to a verified schema list before it is allowed to write any SQL. This makes its internal hypothesis about the relevant database structure explicit and auditable.

The results of this study, visualized in Figure 3, directly motivated the protocol's design. A careful analysis of the error taxonomy led to three key observations.

Obs. 1: Schema verification is critical to suppress hallucination. The study classified failures into specific types: Hallucination (fabricating non-existent tables/columns), Schema Linking (selecting wrong tables/columns despite correct exploration), Semantic (logically incorrect SQL), Syntax (malformed SQL), and Generation (failing to produce complete SQL due to turn limits). In the minimal EC variant, Hallucination accounted for 26.4% of all failures. The generate_sql phase in EGC reduced this to 14.2% by providing some execution-based feedback. However, the most dramatic reduction came from the propose_schema phase in EPGC, which drove hallucination down to just 2.8%—a 9.4× reduction from EC. The paper's explanation is that the propose action forces a cognitive commitment to a concrete set of metadata, making it much harder for the model to drift into fabricating phantom columns later in the SQL generation phase. The act of writing down K_t creates a grounding anchor.

Obs. 2: Schema linking is the persistent bottleneck. While the EPGC protocol nearly eliminated hallucination, the rate of Schema Linking errors remained stubbornly high across all three variants (33%, 31%, and 38% of failures for EC, EGC, and EPGC, respectively). This is the critical insight that motivates the Dual-Track training strategy. The EPGC protocol solves the "hallucination" problem but does not automatically teach the agent how to explore effectively. An agent could perfectly follow the protocol, commit to a schema with no hallucinated tables, and still fail because it linked the wrong ones. This demonstrates that exploration quality is a separate capability that requires its own, targeted optimization signal—exactly what the Dual-Track GRPO Schema Track is designed to provide.

Obs. 3: Suppressing hallucination reveals semantic errors. A fascinating distributional shift occurred. As hallucination errors were suppressed (from 253 in EC to 24 in EPGC), the absolute count of semantic errors increased (from 268 to 330). This is not a regression; it's a revelation. When the agent was hallucinating, it was failing before its semantic reasoning could even be tested. Once the propose checkpoint forced it to play by the rules of the real schema, the true difficulty of writing logically complex queries over that schema became the dominant failure mode. This demonstrates that schema grounding and SQL logic are coupled but distinct challenges, further supporting the need for a joint optimization strategy.

Design justification summary. The EPGC protocol was chosen as the final architecture because the Propose phase serves three simultaneous functions: (1) it acts as a cognitive commitment device that directly suppresses hallucination, (2) it provides a clear structural boundary in the trajectory that can be used for credit assignment during training, and (3) it makes the agent's schema hypothesis an explicit, rewardable output.

---

The Reward Components

The training signals for the agent are defined by three distinct reward functions. Their definitions are precise and crucial, as they are the only source of truth for the RL optimization.

Execution Reward ($R_{\text{exec}}$). This evaluates the final predicted SQL query $y$ against the ground-truth SQL $y^*$ through database execution, not string matching. It is the primary task-completion signal.

\begin{cases} 1.0 & \text{if } \text{Exec}(y) = \text{Exec}(y^*) \\ 0.2 & \text{if } \text{Exec}(y) \neq \emptyset \\ 0.0 & \text{if } \text{Exec}(y) = \emptyset \end{cases}$$ where `$\text{Exec}(y)$` is the result set of executing query `$y$` on the database. **What it computes:** This function assigns a scalar reward by comparing the output of the generated SQL to the output of the ground-truth SQL. The ideal case yields a reward of 1.0. To provide a nuanced signal for non-matching but syntactically valid queries, a partial reward of 0.2 is given if the query executes without error (`$\text{Exec}(y) \neq \emptyset$`) but produces a wrong result. A complete failure to execute (e.g., due to a syntax error) yields 0.0. **Why this form:** The use of execution-based comparison is standard and necessary for evaluating SQL queries with semantic equivalence, where different `ORDER BY` clauses or equivalent `JOIN` structures can produce the same result. The partial reward of 0.2 for executable but incorrect queries is a design choice to create a smoother reward landscape. A sparse binary reward (1.0 for correct, 0.0 otherwise) gives no positive signal for nearly-correct queries that fail on a minor logical detail. The 0.2 reward encourages the model to generate *valid, runnable* SQL even when it's not perfectly correct, preventing the policy from collapsing into generating syntactically broken strings that always yield 0.0 and provide no gradient for improvement. **Format Reward (`$R_{\text{fmt}}$`).** This is a trajectory-level reward that enforces strict adherence to the four-phase protocol's output structure. $$R_{\text{fmt}}(\tau) = \begin{cases} 0.1 & \text{if protocol is fully adhered to} \\ 0.0 & \text{otherwise} \end{cases}$$ **What it computes:** A small, constant bonus of 0.1 for a trajectory that perfectly follows the rules. "Fully adhered to" is a strict condition requiring that: (1) every action `$a_t$` conforms to its prescribed output format (e.g., the `propose_schema` action must contain a valid `<schema>` block, as defined in the agent configuration in Appendix C), (2) all four action categories in `$\mathcal{A}$` appear at least once in the trajectory, and (3) no execution errors occur in the observations `$o_t$` (meaning the agent didn't submit a malformed query that crashed). **Why this form:** This is a form of reward shaping. The core Execution Reward is very sparse and difficult to achieve. The Format Reward is easier to obtain and teaches the agent the basic grammar of interaction and tool use before it has mastered the task itself. The value of 0.1 is intentionally small relative to the 1.0 maximum of `$R_{\text{exec}}$` to ensure it functions as a *guide* and not the primary objective. Making it trajectory-level rather than per-step encourages the agent to maintain structural discipline across the entire interaction, not just in isolated turns. **Schema Reward (`$R_{\text{schema}}$`).** This is the critical innovation that enables the Dual-Track structure. It evaluates the quality of the exploration phase independently of the final SQL. $$R_{\text{schema}}(\hat{K}, K^*) = f_{\text{match}}(\hat{K}, K^*)$$ where `$\hat{K}$` is the verified schema proposed by the agent at step `$t_{\text{propose}}$`, and `$K^*$` is the minimal ground-truth schema derived from the ground-truth SQL `$y^*$`. **What it computes:** This function measures how well the agent's proposed schema `$\hat{K}$` overlaps with the true necessary schema `$K^*$`. The paper explores several formulations for the `$f_{\text{match}}$` function in the analysis section (Section 5.2), with the final chosen method being a "Sparse + Coupled" reward. In this form, `$f_{\text{match}}$` is a binary check (1.0 or 0.0) for perfect structural overlap, but the `$R_{\text{schema}}$` reward is *conditioned* on the Execution Reward. The schema reward is only applied for trajectories where `$R_{\text{exec}} = 1.0$`. The paper also extracts the ground-truth schema `$K^* = (K^*_{\text{table}}, K^*_{\text{col}})$` using a multi-model consensus strategy from GPT-4.1, LongCat-Flash, and Gemini-2.5-Pro to ensure reliability. **Why this form:** A dense or graduated reward function (e.g., giving partial credit for finding 3 out of 4 necessary columns) was tested and found to be suboptimal, as it introduces conflicting gradients between maximizing recall and minimizing unnecessary columns. The binary signal provides an unambiguous optimization target. The critical design choice, however, is **coupling** it to `$R_{\text{exec}} = 1.0$`. An uncoupled schema reward—rewarding perfect schema proposals even if the final SQL fails—was found to incentivize redundant exploration, as the agent could rack up schema rewards without ever successfully completing the task. Coupling the reward establishes a direct causal chain in the learning signal: a perfectly proposed schema is only rewarded if it genuinely led to a successful task outcome, preventing the agent from becoming a "perfect explorer, terrible query writer." --- #### Resolving Credit Assignment via Dual-Track GRPO This is the core algorithmic contribution. It takes the structural boundary from the protocol (the `propose` action) and the independent reward signals (`$R_{\text{schema}}$` and `$R_{\text{exec}}$`) and uses them to create a novel training objective. The base algorithm it extends is Group Relative Policy Optimization (GRPO), a variant of policy gradient methods that uses a group of sampled trajectories to normalize rewards. **Track Formulation and Rewards.** For a given user question `$q$`, the system samples a group of `$G$` trajectories `$\{\tau^1, \tau^2, \dots, \tau^G\}$` from the current policy. Each complete trajectory `$\tau^i$` is decomposed into two sub-trajectories, called tracks, indexed by `$k \in \{\text{schema}, \text{full}\}$`: - The **Schema Track** (`$k = \text{schema}$`) spans from the start of the interaction to the `propose` action at step `$t_{\text{propose}}$`. Its length is `$T_{\text{schema}} = t_{\text{propose}}$`. This track represents the exploration and schema grounding phase. - The **Full Track** (`$k = \text{full}$`) spans the entire interaction from beginning to end `$T_{\text{full}} = T$`. This track represents the complete task. Crucially, a dedicated, independent reward `$R^i_k$` is assigned to each track for each trajectory `$i$`: $$R^i_k = \begin{cases} R_{\text{schema}}(\hat{K}^i, K^*) & \text{if } k = \text{schema} \\ R_{\text{exec}}(y^i, y^*) + R_{\text{fmt}}(\tau^i) & \text{if } k = \text{full} \end{cases}$$ where `$\hat{K}^i$` is the schema proposed in trajectory `$i$`, and `$y^i$` is the final SQL query submitted in that trajectory. **What it computes:** This assignment creates two separate reward streams for the same trajectory. The Schema Track is rewarded purely on the quality of its exploration and schema proposal, irrespective of the eventual SQL execution outcome. The Full Track is rewarded on the final task success (execution accuracy) plus the structural bonus. This means a trajectory with perfect exploration but a flawed SQL query will receive a high Schema Reward and a low Execution Reward, while a trajectory with incomplete exploration that still manages to guess the correct query might receive a low Schema Reward and a high Execution Reward. **Why this form:** This is the direct solution to the credit assignment problem. Under standard GRPO, a single reward per trajectory would conflate these two scenarios, failing to tell the model that the perfect exploration was, in itself, a valuable behavior to repeat. By decoupling the rewards, the model can learn good exploration strategies (Schema Track optimization) and good query generation strategies (Full Track optimization) from the same set of trajectories, using different signals for different parts of the action sequence. **Masked Advantage Computation.** With the tracks and rewards defined, the next step is to compute a learning signal for each token. This is where the token-level masking operates. For each track `$k$`, an advantage `$A^i_k$` is computed for trajectory `$i$` using group-relative normalization: $$A^i_k = \frac{R^i_k - \mu_k}{\sigma_k + \epsilon}$$ where `$\mu_k$` and `$\sigma_k$` are the mean and standard deviation of the rewards `$R^j_k$` across the entire group of `$G$` trajectories for that specific track, and `$\epsilon$` is a small constant for numerical stability. **What it computes:** This is a Z-score for the reward of a single trajectory, calculated with respect to the cohort of trajectories sampled for the same question. A positive advantage `$A^i_k > 0$` means trajectory `$i$` performed better than the group average for that track, and its tokens should be reinforced. A negative advantage means it performed worse, and its tokens should be penalized. **Why this form:** Group-relative normalization provides an adaptive baseline that automatically scales with task difficulty. For an easy question where all trajectories get a high reward, `$\mu_k$` will be high, and even a perfect trajectory's advantage will be small, preventing the model from overfitting to easy examples. For a hard task where most trajectories fail, even a modestly successful trajectory will have a large positive advantage, providing a strong learning signal. This is a standard and effective technique from GRPO. The critical innovation is how this advantage is applied. The scalar advantage `$A^i_k$` is broadcast to the loss function, but **only to the tokens generated within the active steps of the track**. This is a strict token-level mask. For the **Schema Track** (`$k = \text{schema}$`), the advantage `$A^i_{\text{schema}}$` is applied only to tokens generated from step `$0$` to `$T_{\text{schema}}$` (the `propose` step). Tokens generated after the `propose` step receive an effective schema advantage of zero. For the **Full Track** (`$k = \text{full}$`), the advantage `$A^i_{\text{full}}$` is applied to all tokens in the trajectory from step `$0$` to `$T_{\text{full}}$`. This masking is the exact mechanism that prevents the exploration reward from incorrectly crediting generation tokens, and vice versa. If the Schema Track has a high positive advantage because the agent proposed a perfect schema, that positive signal will boost the probabilities of the `explore_schema` and `propose_schema` actions that led to it. The subsequent `generate_sql` action, which might have been flawed, will remain unaffected by this specific signal. **Dual-Track Loss Function.** The final objective function combines the independent loss terms from both tracks. Let `$\mathcal{L}_k(\theta)$` be the standard GRPO loss computed over the active tokens for track `$k$` using the masked advantages `$A^i_k$`. The total training objective is a weighted sum: $$\mathcal{L}(\theta) = \mathcal{L}_{\text{full}}(\theta) + \lambda \cdot \mathcal{L}_{\text{schema}}(\theta)$$ where `$\lambda$` is a hyperparameter controlling the relative contribution of the Schema Track's loss to the overall gradient update. **What it computes:** The total loss is the sum of two independent GRPO losses, each computed over different token spans of the same trajectories with different reward signals. The `$\mathcal{L}_{\text{full}}(\theta)$` term drives the model to maximize task completion, while `$\mathcal{L}_{\text{schema}}(\theta)$` term drives it to maximize exploration quality. `$\lambda$` acts as a mixing coefficient. **Why this form:** This additive composition is the model's way of performing multi-task learning on two interdependent sub-tasks within a single training loop. The ablation study (Section 5.1) shows that the optimal value is `$\lambda = 0.25$`, demonstrating that the schema signal is an effective auxiliary objective. Setting `$\lambda$` too high (e.g., 0.375) over-weights exploration, causing the agent to spend all its time on metadata retrieval and never learn to generate SQL, which is reflected by a spike in average interaction turns and a collapse in execution accuracy. Setting `$\lambda = 0$` eliminates the decoupled credit assignment, reverting to a pure execution-based objective that was shown to be 3.6% worse. ## 4. Key Insights and Innovations ### Innovation 1: The "Unknown Schema" as a Distinct Problem Formulation, Not a Missing Feature Prior work treats the absence of pre-loaded schema as a deployment inconvenience — something to be solved by injecting more context or training on larger synthetic datasets that mimic the full-schema setting. TRUST-SQL's most fundamental conceptual move is to reframe this absence as a **first-class problem in its own right**: the Unknown Schema setting where the database structure is genuinely hidden from the agent, not merely omitted from training data. This is not incremental — it represents a fundamental reconceptualization of what Text-to-SQL means. Under the Full Schema Assumption (the default in virtually all prior benchmarks and methods, from Spider to BIRD to OmniSQL), the task is a **static mapping** from (question, schema) pairs to SQL queries. Schema linking — mapping natural language entities to provided candidate table and column names — is the primary challenge, and the space of possible errors is bounded by the provided schema list. The model may choose the wrong table, but it cannot choose a table that doesn't exist in its input. The Unknown Schema setting changes the nature of the problem in three ways that compound: (1) the space of possible schemas is effectively infinite at the start of the interaction, since the agent knows nothing about what tables exist; (2) the agent must decide **what metadata to query**, introducing an exploration-exploitation tradeoff absent from prior formulations; (3) the agent can **hallucinate** non-existent schema elements based on parametric priors, creating an error mode that cannot even occur when the full schema is provided. The 17.0% absolute collapse in Qwen3-4B's BIRD-Dev accuracy when schema prefilling is removed (Table 2 in Section 4.3) quantifies just how much this shifts the problem: nearly a fifth of all questions become impossible for the same model simply because the schema is no longer handed to it. The POMDP formalization (Section 3.2) is the mathematical expression of this reframing. By defining the database schema as the hidden state and metadata queries as partial observations, the paper connects Text-to-SQL to the broader literature on sequential decision-making under uncertainty — a connection that simply does not exist under the Full Schema Assumption. This is not merely a notation choice; it redefines the evaluation criterion. A good agent is no longer just one that writes correct SQL given a schema, but one that efficiently discovers the *minimum necessary information* to succeed. The significance of this reframing extends beyond the paper's immediate results. It opens Text-to-SQL to the entire toolkit of POMDP-solving techniques (belief state tracking, information-gathering rewards, exploration bonuses) that were previously irrelevant. It also provides a formal language for comparing different exploration strategies — something the field lacked entirely. The decision to treat "not knowing the schema" as the problem definition rather than a missing input feature is what makes the rest of TRUST-SQL's contributions coherent: the four-phase protocol is a POMDP policy architecture, Dual-Track GRPO is a POMDP credit assignment solution, and the Schema Reward is an information-gathering auxiliary objective. ### Innovation 2: The Propose Phase as Both Behavioral Scaffold and Credit Assignment Boundary Many agentic frameworks include intermediate reasoning or planning steps. What distinguishes TRUST-SQL's Propose phase is its **dual function as both a hallucination guardrail and a structural boundary for credit assignment** — and the paper's pilot study (Figure 3, Section 3.1) provides the empirical evidence that both functions are necessary and neither alone is sufficient. The behavioral function is intuitive and well-known from software engineering: making an agent commit to an explicit intermediate representation before acting on it reduces errors. The 9.4× hallucination reduction from EC to EPGC (26.4% to 2.8% of failures) demonstrates this powerfully. But this alone is not conceptually novel — many systems enforce similar checkpoints through prompting or programmatic constraints. What makes this innovation distinctive is the recognition that this checkpoint also **solves a training problem that afflicts all prior multi-turn RL approaches for Text-to-SQL**. In MTIR-SQL (Xu et al., 2025) and SQL-Trail (Hua et al., 2026), the entire interaction trajectory receives a single terminal reward (execution accuracy). This conflates exploration quality with generation quality — a trajectory with perfect schema discovery but a flawed SQL query looks identical (reward 0) to one with terrible exploration and terrible SQL. The model cannot learn that schema verification is valuable independently of query success because *it has no signal that distinguishes the two*. These prior approaches implicitly assume that optimizing the terminal reward will indirectly improve exploration, but the pilot study's Obs. 2 (persistently high Schema Linking errors across all protocol variants) shows this assumption is false: better execution accuracy does not mechanically improve schema linking. TRUST-SQL's insight is that the Propose phase provides a **natural structural boundary** where the trajectory can be split into two semantically meaningful sub-trajectories — before Propose (exploration) and after Propose (generation) — each amenable to independent reward assignment. This transforms the Propose from a mere output convention into a **training architecture component**. The Schema Track and Full Track in Dual-Track GRPO (Section 3.4) are not arbitrary splits; they align with the cognitive boundary the Propose checkpoint already enforces at inference time. This is a conceptually clean design: the same mechanism that prevents hallucinations at inference time enables decoupled optimization at training time. The separation is not an additional complexity burden — it emerges from the protocol architecture already demonstrated to be necessary for behavioral reasons. This tight coupling between behavioral constraints and training objectives is what elevates the Propose phase from a good engineering practice to a principled contribution. Prior work treated interaction protocols and training algorithms as independent design axes; TRUST-SQL shows they can and should be designed together, with the protocol providing the structural scaffolding the training algorithm exploits. ### Innovation 3: Execution-Coupled Schema Rewards as a Solution to the Exploration-Exploitation Credit Assignment Problem The paper's ablation on schema reward design (Section 5.2, Figure 5) reveals a non-obvious finding that challenges intuitive assumptions about how to reward intermediate sub-goals: **a binary, execution-coupled schema reward significantly outperforms both a dense (graduated) schema reward and an uncoupled schema reward**. This finding is a diagnostic contribution to the broader question of how to provide auxiliary rewards in multi-step reinforcement learning for language agents. The intuitive approach to rewarding schema exploration would be to provide partial credit: give the agent a fractional reward based on how many of the necessary tables and columns it correctly identified, regardless of whether the final SQL succeeds. This "Dense + Coupled" variant (a graduated `$f_{\text{match}}$` function conditioned on `$R_{\text{exec}} = 1.0$`) reduced average interaction turns to 5.03 — the agent learned to be efficient — but converged to a suboptimal 64.0% execution accuracy compared to 64.5% for the sparse, binary variant. The paper's explanation is instructive: a dense reward creates conflicting gradients between *maximizing recall* (finding all necessary columns) and *minimizing unnecessary information* (not including irrelevant columns), because the graduated function provides partial credit that can be optimized by slightly overshooting the optimal schema. The "Sparse + Uncoupled" variant (binary reward, but applied regardless of final SQL success) fared even worse at 52.7% accuracy despite the highest average turn count of 6.71. This reveals a dangerous pathology: decoupling the schema reward from the execution outcome incentivizes the agent to become a **perfect explorer but ineffective problem-solver**. It learns to propose impeccable schemas and collect the associated reward, but since this success signal is independent of task completion, it never learns to transition from exploration to generation. The spike in interaction turns confirms this — the agent lingers in the exploration phase because exploring is adequately rewarded on its own. The winning configuration — Sparse + Coupled — succeeds for a subtle reason: by conditioning `$R_{\text{schema}}$` on `$R_{\text{exec}} = 1.0$`, it establishes a **direct causal prerequisite**. The schema reward is only given when a correct schema proposal *actually led to* a correct final answer. This prevents the agent from optimizing the auxiliary objective in isolation and ensures that the schema reward reinforces exploration strategies that genuinely contribute to task success, not merely strategies that check the right boxes. This is a form of *causal credit assignment*: the reward for the exploration phase is mediated by the downstream consequences of that exploration. This finding has implications beyond Text-to-SQL. Any multi-step agentic system that decomposes into information-gathering and action-execution phases faces the same tension: how to reward good information-gathering without creating a degenerate policy that gathers information indefinitely. The paper's resolution — binary, execution-coupled sub-goal rewards — provides a simple but principled template that may generalize to other domains where intermediate verification checkpoints can be defined. ### Innovation 4: Empirical Demonstration That Active Exploration Can Substitute for Privileged Schema Access The paper's headline result — that TRUST-SQL operating without pre-loaded metadata matches or surpasses schema-prefilled baselines — is an empirical finding with substantive implications for how Text-to-SQL systems should be architected. It challenges the implicit assumption that schema prefilling is the gold standard from which other approaches deviate at a cost. The evidence is clearest in the breakdown by benchmark type (Table 1, Section 4.2). On BIRD-Dev, the most schema-intensive benchmark (with large, multi-table databases), TRUST-SQL-4B achieves 64.9% greedy accuracy without prefilling compared to 63.1% for MTIR-SQL-4B *with* full schema access. The 8B variant similarly leads at 65.8%. But the pattern on robustness benchmarks is even more revealing: TRUST-SQL-8B outperforms all prefilled baselines on Spider-Syn (75.4% vs. 69.7% for the second-best prefilled model, OmniSQL-7B) and Spider-Realistic (82.1% vs. 79.6%). These benchmarks introduce perturbations (synonym substitution) and ambiguities that make surface-level schema matching brittle. The actively exploring agent's advantage here suggests that iterative, verification-driven discovery is inherently more robust than one-shot schema linking from a large static context — a finding with direct practical implications for deployment in environments with messy or evolving schemas. The Schema Prefill ablation (Table 2, Section 4.3) provides the most direct evidence. When the full schema is injected into TRUST-SQL-4B as a synthetic initial turn, performance is essentially unchanged (64.9% to 64.8% on BIRD), and actually *degrades* on Spider-DK (71.6% to 69.2%) and Spider-Syn (74.7% to 72.5%). This is not a ceiling effect — the model has room to improve — but rather evidence that the pre-loaded schema introduces noise. The agent, trained to value only verified metadata, treats the pre-loaded schema as unverified, and the additional context becomes a distraction rather than an aid. The iterative policy has learned to be **self-sufficient**: it retrieves precisely the metadata it needs, and receiving extraneous information actively harms performance on tasks requiring value-level grounding (as the case study in Appendix E demonstrates, where schema prefilling causes the model to miss a critical data value predicate that iterative exploration discovers). This finding reframes the deployment question. Rather than asking "how can we fit the schema into the context window more efficiently?", the TRUST-SQL results suggest asking "how can we make the agent discover the schema more efficiently?". The second question leads to a very different research agenda — focused on exploration strategies, information-gathering efficiency, and verification mechanisms — than the first, which leads to schema compression, retrieval-augmented schema injection, and context optimization. The paper does not claim active exploration is universally superior (Section 4.3 shows it provides negligible benefit on the simplest schemas and may add inference overhead), but it establishes active exploration as a **credible alternative to prefilling** rather than a fallback for when prefilling is impossible — a reversal of the conventional wisdom in the field. ## 5. Experimental Analysis ### Evaluation Methodology - **Dataset.** The primary evaluation uses BIRD-Dev (Li et al., 2024) for large-scale schema grounding and Spider-Test (Yu et al., 2018) for compositional generalization. Three robustness variants stress-test generalization: Spider-Syn (Gan et al., 2021a) evaluates lexical robustness via synonym substitution, Spider-DK (Gan et al., 2021b) probes for implicit domain knowledge, and Spider-Realistic (Deng et al., 2021) assesses ambiguity resolution. An additional experiment on Spider 2.0's SQLite subset (Lei et al., 2024; 135 questions) tests enterprise-grade database complexity. All benchmarks use SQLite exclusively. - **Base model(s).** Two model scales from the Qwen3 family (Qwen, 2025): Qwen3-4B and Qwen3-8B. The paper argues these are representative of contemporary open-weight LLMs at their respective scales, and their non-trivial but far-from-saturated performance under the Unknown Schema setting (Qwen3-4B achieves only 29.3% on BIRD-Dev without schema prefilling, per Table 2) leaves substantial room for improvement through interaction and training. - **Metrics.** The primary metric is **Execution Accuracy (EX)** — the predicted SQL must produce exactly the same database result set as the ground-truth SQL. This is a semantic comparison (not string matching), which correctly handles equivalent queries with different syntax. Single-sample performance is measured via greedy decoding at temperature 0. Execution-based Majority Voting across multiple sampled trajectories (temperature 0.8) is also reported. For the Pass@K analysis in Section 5.3, a trajectory is considered correct if its final SQL yields the correct execution result. - **Baselines.** The paper compares against several categories: - **Single-turn methods with schema prefilling:** OmniSQL-7B (Li et al., 2025), trained on 2.5M synthetic samples; SQL-R1-3B and SQL-R1-7B (Ma et al., 2025), RL-trained on 5k synthetic samples with execution rewards; and CHESS (Talaei et al., 2024), a training-free pipeline method using frozen LLMs for schema selection. - **Multi-turn RL methods with schema prefilling:** MTIR-SQL-4B and MTIR-SQL-8B (Xu et al., 2025), trained on 18.1k samples from Spider and BIRD; SQL-Trail-3B and SQL-Trail-7B (Hua et al., 2026), trained on 0.8k SFT + 1k RL samples. - **Proprietary models** for the Spider 2.0 experiment: GPT-4o (OpenAI, 2024), DeepSeek-V3 (DeepSeek-AI, 2025), and OpenSearchSQL paired with Arctic-Text2SQL-R1-7B (Yao et al., 2025). - **Base models with and without schema prefilling:** Qwen3-4B and Qwen3-8B evaluated both with full schema injected and under the Unknown Schema setting, serving as the critical baseline for measuring TRUST-SQL's improvement. - **Generation budget / compute accounting.** The paper measures compute implicitly through several axes. The maximum interaction turn budget `T` is the primary constraint — the agent can take at most `T` steps per question. Training uses a 10-turn budget; inference is evaluated at both 10-turn and 15-turn budgets (Section 5.3, Figure 6b). For the Pass@K analysis (Figure 6c), `K` independently sampled complete trajectories per question are generated. Token consumption statistics (Table 10, Appendix D.1) report total output tokens per query and average turn counts with tool call frequencies. Latency in seconds per query is also reported for efficiency comparison. - **Cross-validation / statistical protocol.** No explicit cross-validation or statistical significance testing is reported. The paper uses fixed train/test splits from the standard benchmarks (BIRD training set for training, BIRD-Dev for evaluation; Spider training for RL question filtering, Spider-Test for evaluation). The RL data filtering strategy (Appendix A.3) uses an 8-trajectory rollout with a pass rate threshold (< 6/8) to retain only questions where the SFT policy is not already near-perfect. Training curve stability is assessed by monitoring execution accuracy and average turn counts across training steps (Figures 4, 5, 6a), but no confidence intervals or multiple-seed results are reported. --- ### Main Quantitative Results #### Overall Performance Across Benchmarks (Table 1) The headline result is that TRUST-SQL operating entirely without pre-loaded metadata (Unknown Schema, denoted × in the prefilling column) matches or outperforms strong baselines that rely on full schema prefilling (✓). At the 4B scale, TRUST-SQL-4B achieves 64.9% greedy execution accuracy on BIRD-Dev, compared to 63.1% for MTIR-SQL-4B (the best prefilled baseline at this scale) and 50.1% for SQL-Trail-3B. With majority voting at temperature 0.8 and a 15-turn inference budget, TRUST-SQL-4B reaches 67.2%, surpassing MTIR-SQL-4B's 64.4%. On Spider-Test, the pattern reverses: TRUST-SQL-4B's 82.8% greedy falls slightly behind MTIR-SQL-4B's 83.4%, though majority voting closes this gap (85.0% for TRUST-SQL-4B vs. unreported for MTIR-SQL-4B). The robustness benchmarks tell a more decisive story. On Spider-Syn (synonym substitution), TRUST-SQL-4B achieves 74.7% greedy, compared to a best prefilled baseline of 69.7% (OmniSQL-7B). On Spider-Realistic, the advantage is 79.9% vs. 78.7% (MTIR-SQL-4B) for greedy, and 82.5% vs. unreported for majority voting. On Spider-DK, TRUST-SQL-4B (71.6%) edges above the best prefilled baseline SQL-R1-3B (70.5%). At the 8B scale, TRUST-SQL-8B achieves the highest BIRD-Dev greedy accuracy (65.8%) and majority voting accuracy (67.7%), surpassing MTIR-SQL-8B (63.6% greedy, 64.6% majority). On Spider-Syn (75.4% vs. the next-best 72.8% from SQL-Trail-7B) and Spider-Realistic (82.1% vs. 79.6% from SQL-Trail-7B), the 8B model similarly leads the prefilled baselines. The pattern is consistent: TRUST-SQL's largest relative advantages appear not on the standard Spider-Test (where schema linking from a provided list is a well-learned skill), but on benchmarks that perturb surface-level schema information, suggesting that active exploration provides robustness that static schema linking cannot replicate. #### Schema Prefill Effect (Table 2, Section 4.3) The most diagnostic result for TRUST-SQL's core capability is the Schema Prefill ablation (Table 2). Base Qwen3 models without schema prefilling collapse: Qwen3-4B drops from 46.3% to 29.3% on BIRD-Dev (a 17.0 absolute percentage point decline), and Qwen3-8B drops from 49.9% to 47.9% on BIRD but also drops on all robustness benchmarks (e.g., Spider-Syn falls from 64.5% to 58.4%, a 6.1 point decline). This confirms that base models fundamentally lack autonomous exploration capability — the Full Schema Assumption is not a convenience but a hard requirement for them. TRUST-SQL eliminates this dependence. Across all five benchmarks, the 4B variant achieves an average absolute improvement of 30.6% over the base Qwen3-4B under the Unknown Schema setting, while the 8B variant achieves 16.6% average improvement. The per-benchmark breakdown (Table 2): on BIRD-Dev, TRUST-SQL-4B improves from 29.3% to 64.9% (+35.6 points); on Spider-Test, from 51.2% to 82.8% (+31.6 points); on Spider-DK, from 43.7% to 71.6% (+27.9 points); on Spider-Syn, from 47.4% to 74.7% (+27.3 points); on Spider-Realistic, from 49.2% to 79.9% (+30.7 points). Crucially, injecting the full schema into TRUST-SQL provides negligible benefit and sometimes hurts. For TRUST-SQL-4B: BIRD goes from 64.9% to 64.8% (−0.1), Spider-Test from 82.8% to 83.1% (+0.3), Spider-DK from 71.6% to 69.2% (−2.4), Spider-Syn from 74.7% to 72.5% (−2.2), Spider-Realistic from 79.9% to 80.1% (+0.2). The 8B variant shows similar behavior: BIRD drops from 65.8% to 65.5% (−0.3), Spider-Realistic drops from 82.1% to 80.5% (−1.6), while Spider-DK improves from 72.1% to 74.4% (+2.3). The paper interprets these mixed and often-negative deltas as evidence that the pre-loaded schema introduces noise — the agent, trained to value only explicitly verified metadata, treats pre-loaded information as unverified, and the additional context can distract from the value-level probing that the case study (Appendix E) shows is critical for correct answers. #### Dual-Track GRPO Training Dynamics (Sections 5.1–5.2, Figures 4–5) **The λ hyperparameter controlling schema track weight (Section 5.1, Figure 4).** The Schema Track's contribution to the total loss is controlled by λ (Equation 6). The paper ablates λ ∈ {0.125, 0.25, 0.375} against two single-track baselines where λ = 0. Both single-track baselines share the same underlying GRPO optimization but differ in reward construction: one uses only the execution-based terminal reward ("λ = 0 w/o schema"), achieving 60.9% on BIRD-Dev; the other naively adds a schema reward weighted at 0.25 to the terminal reward without track separation ("λ = 0 w/ schema"), achieving 58.7%. The latter is worse than the pure execution baseline — the paper interprets this as evidence that "conflating exploration and generation obscures the reward signal," since the mixed single reward cannot distinguish whether a trajectory succeeded because of good exploration or despite poor exploration. The optimal Dual-Track setting is λ = 0.25, which peaks at 64.5%, yielding a 5.8 percentage point gain over naive aggregation (58.7%) and a 3.6 point gain over the pure execution baseline (60.9%). At λ = 0.125, performance reaches 64.0%, close to the optimum. At λ = 0.375, performance degrades severely to 54.2%. Figure 4b reveals the mechanism: average interaction turns spike to 6.66 at λ = 0.375, compared to 5.64 at λ = 0.25 and 4.99 at λ = 0.125. The over-weighted schema reward incentivizes the agent to "remain perpetually in the exploration phase," maximizing metadata retrieval at the expense of ever transitioning to SQL generation. The relationship between λ and turn count is monotonic across the tested range, confirming that the Schema Track directly shapes exploration behavior. **Schema reward design (Section 5.2, Figure 5).** Three formulations of `f_match` in the Schema Reward are compared, all using λ = 0.25 and the Qwen3-4B base model. The "Sparse + Coupled" variant (binary match, conditioned on `R_exec = 1.0`) is TRUST-SQL's chosen configuration and achieves 64.5% with 5.64 average turns. "Sparse + Uncoupled" assigns the same binary schema reward regardless of final execution outcome and achieves only 52.7% — the worst of all variants — with the highest turn count of 6.71. The paper's interpretation: decoupling the schema reward from execution outcome incentivizes the agent to collect schema rewards through perfect exploration without ever learning to complete the task, explaining both the low accuracy and the inflated turn count. "Dense + Coupled" uses a graduated `f_match` (partial precision-based rewards while enforcing full recall as a hard gate) conditioned on `R_exec = 1.0`; it reduces turns to 5.03 (the lowest of all variants) but converges to 64.0%, 0.5 points below the sparse variant. The paper attributes this to conflicting gradients: the graduated function simultaneously encourages the agent to maximize recall (finding all necessary columns) and minimize unnecessary information, creating optimization tension that the binary signal avoids by providing an unambiguous, all-or-nothing target. These results collectively demonstrate that (1) track separation is necessary — mixing schema and execution rewards into a single terminal signal is worse than using no schema reward at all, (2) the schema reward's weight must be carefully tuned to prevent exploration from dominating generation, and (3) a binary, execution-coupled schema reward provides a cleaner optimization signal than a graduated one. #### Test-Time Scaling Behavior (Section 5.3, Figure 6) **Training turn budget (Figure 6a).** Expanding the training turn budget from 8 to 10 yields substantial accuracy gains on BIRD-Dev, with the 10-turn setting achieving the peak. Further increasing to 12 turns causes training instability: the average turn count spikes and execution accuracy sharply declines. The paper interprets this as the model failing to penalize redundant exploration when given an overly permissive horizon. The 10-turn budget is selected as optimal for all subsequent experiments. **Interaction between training and inference budgets (Figure 6b).** The paper cross-evaluates policies trained with 8, 10, and 12 turn budgets at inference budgets of 8, 10, 12, and 15 turns. The 10-turn training budget consistently yields the strongest baseline policy across all inference settings. Notably, providing additional inference turns beyond the training horizon improves performance: the 10-turn-trained policy achieves 63.49% at inference turn = 10 (matching its training budget), 64.86% at inference turn = 12, and peaks at 64.93% at inference turn = 15. This demonstrates that the learned policy generalizes to longer horizons — the agent can effectively utilize extra test-time compute to recover from early exploration mistakes, even though it was never trained on trajectories that long. The 12-turn-trained policy underperforms at all inference budgets, confirming that the training instability at 12 turns is a genuine degradation of the policy, not merely a mismatch with inference settings. **Pass@K scaling with repeated sampling (Figure 6c).** For both 4B and 8B models at both 10-turn and 15-turn inference budgets, Pass@K increases monotonically as the number of sampled trajectories K grows from 1 to 8. The absolute numbers on BIRD-Dev at temperature 0.8: TRUST-SQL-4B with maxTurn=10 achieves Pass@1 = 64.4%, Pass@4 = 72.2%, Pass@6 = 74.0%, Pass@8 = 75.1%. With maxTurn=15, the corresponding values are 64.9%, 72.9%, 74.3%, and 75.2%. The 8B model follows the same pattern at a slightly higher baseline: Pass@1 = 65.8%, Pass@4 = 72.8%, Pass@6 = 74.4%, Pass@8 = 75.4% at maxTurn=10, and 65.8%, 73.2%, 74.7%, 75.6% at maxTurn=15. Extended Pass@K results for the remaining four benchmarks (Table 11, Appendix D.2) confirm monotonic scaling across all evaluation settings. The persistent gap between Pass@K and greedy performance (e.g., 75.1% Pass@8 vs. 64.9% greedy for the 4B model at 15 turns) — where the model can generate correct solutions through repeated sampling but does not consistently produce them under greedy decoding — indicates that the policy has not fully converged. The paper interprets this as headroom for further RL training. #### Cold-Start SFT Necessity (Section 5.4, Table 3) The full TRUST-SQL pipeline consists of SFT warm-up followed by Dual-Track GRPO. Three configurations are compared: SFT only, RL only (Dual-Track GRPO without SFT initialization), and the full SFT + RL pipeline. SFT alone achieves 46.2% on BIRD-Dev and 66.7% on Spider-Test — reasonable but well below the full pipeline. Applying Dual-Track GRPO directly to the base model without SFT warm-up achieves 59.9% on BIRD and 79.6% on Spider — superficially competitive but, the paper argues, largely illusory. Without SFT initialization, the model quickly learns a degenerate strategy: it exhaustively queries all tables and columns in the first turn, completing the entire interaction in roughly four actions. This bypasses genuine active exploration entirely, essentially converting the Unknown Schema setting back into a disguised Full Schema scenario. The full SFT + RL pipeline achieves 64.9% on BIRD (+5.0 over RL-only) and 82.8% on Spider (+3.2 over RL-only), demonstrating that the SFT warm-up instills structured exploration behavior that the subsequent RL phase can refine rather than circumvent. #### Efficiency and Cost (Table 10, Appendix D.1) The inference cost analysis on BIRD-Dev (Table 10) provides crucial context for TRUST-SQL's practical applicability. Training-free pipeline methods like CHESS achieve 61.5% accuracy but consume 251.3 seconds and 320.8K tokens per query on average — a computation budget that makes them impractical for real-world deployment. TRUST-SQL-4B achieves a higher accuracy of 64.9% under the Unknown Schema setting with only 0.6 seconds latency and 2.83K output tokens, representing approximately a 500× reduction in latency and a 113× reduction in token consumption compared to CHESS. The average turn count is 5.89, with 3.66 tool calls per query on average. Compared to schema-prefilled baselines of similar scale, TRUST-SQL's efficiency is comparable rather than substantially worse. MTIR-SQL-4B (with schema prefilling) consumes 2.9K tokens with 1.34 tool calls at 0.5 seconds latency. TRUST-SQL-4B (without prefilling) consumes 2.83K tokens with 3.66 tool calls at 0.6 seconds, confirming that the active exploration policy retrieves only the necessary metadata and does not incur prohibitive overhead. The base Qwen3 models reveal their dependence on prefilling: Qwen3-4B without schema goes from 1.82K tokens (with prefilling) to 4.93K tokens (without), and its accuracy drops from 46.3% to 29.3% — the model tries to explore but does so inefficiently and often incorrectly. #### Complex Benchmark (Spider 2.0, Appendix D.3, Table 12) The Spider 2.0 experiment evaluates TRUST-SQL under enterprise-grade complexity on the SQLite subset (135 questions). Strong proprietary models like GPT-4o and DeepSeek-V3 achieve only 15.6% execution accuracy with full schema prefilling, and the specialized OmniSQL-7B reaches 10.4%. These low numbers reflect the benchmark's genuine difficulty — databases with significantly more complex schemas and larger table counts than standard Spider, requiring extended multi-step reasoning. TRUST-SQL-8B, operating without any pre-loaded metadata, achieves 14.8% greedy accuracy and 24.9% Pass@8, surpassing the OpenSearchSQL + Arctic-Text2SQL-R1-7B combination (14.1% greedy, 20.7% Pass@8) that uses full schema access. The non-saturating Pass@8 curve (a 10.1 point gap from greedy to Pass@8) suggests substantial room for improvement with increased sampling budgets. This result on a substantially more challenging and realistic benchmark corroborates the pattern from the robustness benchmarks: active exploration provides advantages that scale with schema complexity. --- ### Ablation Studies and Robustness Checks - **Schema track weight λ (Section 5.1, Figure 4):** λ = 0.25 is optimal (64.5% on BIRD), compared to 60.9% for pure execution-based GRPO (λ = 0 without schema) and 58.7% for naively aggregated schema reward (λ = 0 with schema). λ = 0.375 causes severe degradation (54.2%) due to over-exploration (spikes to 6.66 average turns). The non-monotonic relationship between λ and performance — where moderate schema reward helps but aggressive schema reward catastrophically hurts — demonstrates that the Schema Track's auxiliary objective must remain subordinate to the primary execution objective, and that Dual-Track GRPO's structural separation (track-level advantage masking) is what prevents the schema reward from contaminating the generation optimization. - **Schema reward coupling and sparsity (Section 5.2, Figure 5):** Binary (sparse) schema reward coupled to execution success (64.5%, 5.64 turns) outperforms both uncoupled binary (52.7%, 6.71 turns) and dense coupled (64.0%, 5.03 turns). The uncoupled variant's failure (lowest accuracy, highest turns) reveals a reward hacking pathology: the agent learns to optimize schema proposals independently of task completion, collecting easy schema rewards through exhaustive exploration without learning to succeed at the actual SQL task. The dense coupled variant's suboptimality (0.5 points below sparse) is attributed to conflicting gradients between maximizing recall and minimizing extraneous columns introduced by the graduated `f_match` function. These results demonstrate that the choice of auxiliary reward design — not just its presence — is critical for multi-track RL. - **Cold-start SFT necessity (Section 5.4, Table 3):** RL-only training (without SFT warm-up) achieves superficially competitive 59.9% on BIRD and 79.6% on Spider, but the paper demonstrates this is achieved through a degenerate strategy — exhaustive first-turn metadata queries that bypass genuine exploration, completing interactions in ~4 turns. The full SFT + RL pipeline (64.9% on BIRD, 82.8% on Spider) produces a qualitatively different policy that engages in structured, efficient exploration. This shows that SFT warm-up provides necessary behavioral priors (the four-phase protocol structure) that pure RL optimization exploits rather than circumvents. It also serves as a cautionary negative result: RL applied to complex tool-use tasks without behavioral initialization can produce policies that superficially achieve high reward while circumventing the intended task structure. - **Schema Prefill interaction (Table 2):** Injecting full schema into TRUST-SQL produces negligible or negative effects on most benchmarks — BIRD drops 0.1 and 0.3 points for 4B and 8B, Spider-Syn drops 2.2 points for 4B and 0.0 for 8B, Spider-Realistic drops 1.6 points for 8B. Only Spider-DK for 8B shows a positive effect (+2.3). This is not a ceiling effect (accuracy is far from 100%) but evidence that the pre-loaded schema acts as noise for an agent trained to verify metadata. Importantly, this test validates that TRUST-SQL's policy has genuinely learned self-sufficient exploration, not merely learned to perform well without prefilling when prefilling would help. - **Pass@K across all benchmarks (Appendix D.1, Table 11):** The monotonic Pass@K scaling observed on BIRD-Dev (Figure 6c) generalizes to all four additional benchmarks. For TRUST-SQL-8B: Spider-Test Pass@1 = 83.9%, Pass@8 = 87.5%; Spider-DK Pass@1 = 72.1%, Pass@8 = 81.3%; Spider-Syn Pass@1 = 75.4%, Pass@8 = 84.0%; Spider-Realistic Pass@1 = 82.1%, Pass@8 = 87.0%. The consistency across diverse benchmarks — standard, synonym-substituted, domain-knowledge-requiring, ambiguity-stressed — indicates that the exploration diversity captured by repeated sampling is a general property of the trained policy, not an artifact of a particular dataset. - **RL data quality control (Appendix A.3, Table 7):** The difficulty-based filtering for RL training (requiring pass rate < 6/8 on 8 rollouts from the SFT policy) retains 11,642 questions from 18,078 candidates, rejecting 6,436 (35.6% rejection rate). This is a critical engineering detail: questions already nearly solved by the SFT policy provide negligible learning signal for RL, and including them would waste training compute. The paper does not ablate this filtering threshold, but its effect is implicit in the SFT vs. SFT+RL comparison (Table 3): without filtering, the RL gain might be smaller or slower to emerge. - **Ground-truth schema extraction quality (Appendix A.3):** The multi-model consensus strategy using GPT-4.1, LongCat-Flash, and Gemini-2.5-Pro (requiring at least 2 out of 3 models to agree) for extracting `K*` from `y*` is not ablated. This is a potential source of noise in the Schema Reward, and an ablation comparing single-model vs. consensus extraction or oracle (human-annotated) schemas would strengthen confidence in the Schema Track's robustness to label quality. The paper acknowledges the consensus approach but does not quantify its error rate relative to human annotation. --- ### Critical Assessment The experimental evaluation provides strong evidence for TRUST-SQL's core capability — autonomous database exploration substituting for pre-loaded schemas — with several important qualifications about the scope and generalizability of these findings. **Claim: TRUST-SQL achieves massive performance leaps over base models under the Unknown Schema setting.** This claim is unambiguously supported. The 30.6% and 16.6% average improvements for 4B and 8B variants (Table 2) are measured directly against the same base models evaluated without schema prefilling on the same benchmark splits. There is no circularity or leakage in this comparison. The improvement is present across all five benchmarks, with the smallest absolute gain being 27.3 points (Spider-Syn, 4B) and the largest being 35.6 points (BIRD-Dev, 4B). The contrast is stark: base Qwen3 models are non-functional (29.3% on BIRD) without pre-loaded schemas, while TRUST-SQL is state-of-the-art on the same setting. However, the claim is specifically about improvement *over the base model in the Unknown Schema setting*, not over all possible approaches. The 30.6% figure should not be interpreted as "TRUST-SQL is 30.6% better than the best possible Text-to-SQL system" — it is the gap between a model without any exploration capability and a model trained for exploration. This is the right and most honest comparison, but it is important to recognize it as measuring the elimination of a specific deficit rather than a general superiority over all methods. **Claim: TRUST-SQL matches or surpasses strong baselines that rely on schema prefilling.** This claim is supported but requires careful qualification about *which* baselines and *which* benchmarks. On BIRD-Dev, TRUST-SQL-4B (64.9%) does edge above MTIR-SQL-4B (63.1%), but this is a 1.8 point difference on a single benchmark with a 500-question test set, and no statistical significance testing is reported. On the Spider-Test, TRUST-SQL-4B (82.8%) is actually slightly below MTIR-SQL-4B (83.4%), though majority voting (85.0%) flips this. The more convincing evidence for this claim comes from the robustness benchmarks: Spider-Syn (74.7% vs. next-best 69.7%), Spider-Realistic (79.9% vs. next-best 78.7%), and Spider 2.0 (14.8% vs. 14.1% for the best open-source baseline). Here the gaps are larger and more consistent, suggesting that the active exploration advantage is most pronounced when surface-level schema matching is made difficult — precisely the scenario the Unknown Schema setting is designed for. A notable limitation: the baselines are not all evaluated under identical conditions. MTIR-SQL and SQL-Trail use different training data volumes and sources (Table 4), and some majority voting results are unreported (e.g., MTIR-SQL's Spider-Test majority). The paper's claim that TRUST-SQL uses a "highly efficient data recipe" (9.2k SFT + 11.6k RL samples) is noteworthy — this is 1–2 orders of magnitude less than OmniSQL's 2.5M SFT samples — but the data efficiency comparison is confounded by the fact that multi-turn trajectory data is inherently more expensive to collect and annotate per sample than single-turn data. A more principled comparison would match total training FLOPs rather than sample counts. **Claim: Dual-Track GRPO yields a 9.9% relative improvement over standard GRPO.** The figure 9.9% appears to be computed from the λ = 0 w/o schema baseline (60.9%) to the optimal λ = 0.25 Dual-Track configuration (64.5%), yielding (64.5 - 60.9) / 60.9 ≈ 5.9% absolute, which as a percentage of the baseline's accuracy gap from ceiling — or as a different relative computation — arrives at the 9.9% figure. [*Note: the paper states "yielding a 9.9% relative improvement in execution accuracy over standard GRPO on BIRD-Dev" in the abstract and introduction, but the exact computation is not explicitly walked through in the main text. Based on the numbers in Section 5.1, the improvement from 60.9% to 64.5% represents a 3.6 percentage point absolute gain. The "9.9% relative" likely refers to (64.5 - 58.7) / 58.7 ≈ 9.9%, where 58.7% is the "λ = 0 w/ schema" baseline — the version that uses a schema reward but mixes it into a single terminal reward without track separation. This matches the paper's emphasis that Dual-Track GRPO's specific contribution is the *decoupling via track separation*, not merely the addition of a schema reward.*] This is a meaningful improvement, but it is measured at a single optimal λ value on a single benchmark, with no error bars provided. The gap between the second-best configuration (λ = 0.125 at 64.0%) and the optimum (λ = 0.25 at 64.5%) is only 0.5 points, suggesting the result is somewhat sensitive to hyperparameter choice, though the broad trend (Dual-Track > single-track) is robust across λ values in the effective range. **Claim: The Propose checkpoint reduces hallucination by 9.4×.** This claim from the pilot study (Section 3.1, Figure 3) is measured on Qwen3-8B base model without any training — it reflects the effect of adding protocol constraints to an untrained model through prompting alone. As such, it demonstrates the behavioral necessity of the Propose phase in the architecture, but it does not measure how much the Propose checkpoint contributes to TRUST-SQL's final performance after training. An ablation of the full trained TRUST-SQL with and without the Propose phase (which would require a different protocol architecture, likely EGC) is not reported, so it is unclear whether the trained RL policy has internalized the anti-hallucination discipline to the point where the explicit Propose checkpoint becomes redundant. This is an important missing experiment. **What experiments would strengthen the paper:** - **Multi-seed training runs with variance reporting.** All main results are reported as point estimates without confidence intervals. Given the training instability observed at T=12 turns (Figure 6a), it is plausible that different random seeds could produce meaningfully different outcomes, particularly for the λ comparisons where differences are on the order of 0.5–1.0 points. Even 2–3 seeds would substantially improve confidence in the relative rankings. - **Ablation of the Propose phase in the trained model.** The pilot study (Figure 3) shows the Propose phase is necessary for untrained models. An experiment comparing full trained TRUST-SQL (EPGC) to a version trained with the EGC protocol (no Propose phase, using the same Dual-Track GRPO but with the track boundary at the Generate action instead) would reveal whether the trained policy still depends on the explicit Propose checkpoint, or whether RL has internalized the anti-hallucination behavior to the point where the checkpoint is primarily serving as a credit assignment boundary rather than a behavioral necessity. - **Cross-model-family evaluation.** All experiments use Qwen3 as the base model family. The finding that Schema Prefill hurts TRUST-SQL (Table 2) could be specific to Qwen3's pre-training biases or its tendency to hallucinate certain types of schema elements. Replicating key results (especially the Schema Prefill ablation and the λ sweep) on a different model family (e.g., Llama, DeepSeek) would establish whether the "prefilling as noise" effect is a general property of exploration-trained agents or a Qwen3-specific artifact. - **Oracle schema reward ablation.** The Schema Reward depends on `K*` extracted via multi-model consensus (Appendix A.3). Replacing this with human-annotated gold `K*` for a subset of the evaluation (e.g., the 500-question BIRD-Dev set) would quantify the noise floor of the Schema Reward and determine whether the observed 0.5 point gap between Sparse and Dense schema rewards (Figure 5) is attributable to reward noise or to a genuine optimization difference. - **Scaling analysis with model size.** The paper reports 4B and 8B variants, which is a 2× scale difference. The results show diminishing returns: the 8B model's absolute improvement over its base model (16.6% average) is roughly half of the 4B model's (30.6%), because the 8B base model starts from a higher baseline (47.9% vs. 29.3% on BIRD). A smaller model (e.g., 1.5B–2B) or a larger model (14B–32B) would help characterize whether TRUST-SQL's benefit is largest at a specific capability threshold where the base model has enough reasoning ability to benefit from structured exploration but not enough to succeed without it — a "sweet spot" analogous to the difficulty-dependent scaling in the example paper's compute-optimal test-time scaling analysis. **Overall assessment.** The experimental design cleanly tests TRUST-SQL's primary contribution — autonomous exploration substituting for schema prefilling — through a combination of benchmark evaluations, controlled ablations, and cost analysis. The results consistently demonstrate that TRUST-SQL eliminates the performance cliff that base models face when schemas are withheld, and that the trained policy achieves competitive or superior performance to prefilled baselines, particularly on schema-perturbed benchmarks. The Dual-Track GRPO ablation convincingly demonstrates that track separation provides a genuine optimization benefit over naive reward aggregation, though the exact magnitude of this benefit is modest (3.6 absolute points over the better baseline) and its robustness to seed variance is untested. The most significant limitation is the single-model-family evaluation — the central finding that active exploration can substitute for schema prefilling needs validation on at least one other model architecture before it can be considered a general principle rather than a Qwen3-specific success story. ## 6. Limitations and Trade-offs ### 6.1 The Inference Overhead of Multi-Turn Interaction Is Partially Characterized but Not Fully Resolved **The constraint.** The four-phase protocol requires multiple rounds of database interaction per question. Each `explore_schema` or `generate_sql` action executes a live SQL query against the database, and the agent must wait for results before proceeding to its next action. The paper reports average turn counts of 5.89 (4B model) and 5.62 (8B model) on BIRD-Dev (Table 10, Appendix D.1), meaning each question requires approximately 6 sequential round-trips to the database. The paper explicitly acknowledges this in the Limitations section: > "The multi-turn interaction paradigm naturally incurs higher inference cost compared to single-turn methods, as each interaction step involves a live database call." **The consequence.** Latency, not just total FLOPs or tokens, emerges as a hard deployment constraint that the paper's primary metric (execution accuracy) obscures. A single-turn model like SQL-R1-7B produces its answer in 0.4 seconds; TRUST-SQL-4B requires 0.6 seconds (a 50% increase) despite consuming fewer output tokens (2.83K vs. 3.1K). For the latency-sensitive deployment scenarios that motivate the Unknown Schema setting — interactive database querying, real-time dashboards, customer-facing analytics — this serial dependency fundamentally limits throughput in a way that parallel sampling cannot mitigate. While the paper frames this as "modest in practice" (Limitations), the comparison point matters: 0.6 seconds per query for a single user is indeed modest, but for a system processing thousands of concurrent queries, the accumulated serial database round-trips create a throughput bottleneck that single-turn models avoid entirely. A subtler consequence: the latency overhead makes the test-time scaling strategy (Figure 6c, Section 5.3) less practically relevant than it appears. The Pass@K gains from sampling 8 trajectories (e.g., 75.1% Pass@8 vs. 64.9% greedy for the 4B model) require 8 independent multi-turn interactions, each with ~6 serial database calls. This is not 8× the cost — it is 8× the number of sequential dependencies, and in a production system with database connection pooling and query queuing, these parallel trajectories may contend for the same database resources, increasing the latency of each individual call beyond what isolated benchmarks measure. **What evidence exists.** Table 10 (Appendix D.1) reports latency and token consumption for all methods. The paper's own numbers show that TRUST-SQL without schema prefilling (0.6s latency) is slower than every schema-prefilled baseline at comparable scale: MTIR-SQL-4B achieves 0.5s, SQL-R1-7B achieves 0.4s, OmniSQL-7B (latency unreported but presumably similar to single-turn models), and even the same TRUST-SQL model with schema prefilling achieves 0.4s. The 0.2s difference (50% relative increase) is attributable entirely to the additional interaction turns. No throughput benchmark (queries per second under concurrent load) is reported, and no analysis of how latency scales with database size or query complexity is provided. **Mitigation status.** The paper does not attempt to reduce the turn count beyond the λ tuning that prevents pathological over-exploration (Section 5.1). It flags inference efficiency optimization as "a practical direction for future work" but proposes no concrete mechanisms — such as batching multiple metadata queries into a single tool call, caching schema information across questions targeting the same database, or training the model to predict when further exploration is unnecessary. The Schema Prefill experiment (Table 2) shows that injecting the full schema reduces trivially-easy exploration turns to zero, but the paper does not propose or evaluate a hybrid strategy that combines lightweight prefilling (e.g., table names only) with selective deep exploration. --- ### 6.2 The SQLite-Only Constraint Leaves Dialect-Dependent Behaviors Uncharacterized **The constraint.** The paper states explicitly: > "Both training and evaluation are conducted on SQLite-based benchmarks, as BIRD and Spider exclusively use SQLite. Extending to other SQL dialects such as PostgreSQL or MySQL remains a valuable direction for future work." This is not merely a benchmark coverage issue — it is a potential capability boundary. All metadata exploration (`explore_schema` actions) queries SQLite's specific system tables (`sqlite_master`) and relies on SQLite's particular information schema conventions. The tool definitions (Appendix C.3) hard-code the `execute_sql_query` function with a `db_id` parameter but no dialect specification, and the prompt template assumes SQLite-specific querying patterns. **The consequence.** Dialect-specific schema discovery is a fundamentally different task than SQLite schema discovery. PostgreSQL and MySQL have different system catalogs (`information_schema.tables` vs. `pg_catalog.pg_tables`), different naming conventions for metadata columns, and substantially different SQL dialects for the queries themselves (e.g., PostgreSQL's `STRING_AGG` vs. MySQL's `GROUP_CONCAT`). A model trained exclusively on SQLite system table queries has no exposure to these conventions and may fail entirely when the `explore_schema` tool's expected query patterns don't match the target database's metadata interface. Beyond metadata queries, the SQL generation itself — which the Dual-Track GRPO optimizes for execution accuracy against SQLite's query engine — may produce SQL that is syntactically valid in SQLite but rejected by PostgreSQL's stricter type checking or MySQL's different function names. The execution-based reward signal (Equation 1) would silently penalize these queries during training on SQLite, but the learned policy would have no mechanism to adapt its generation to a different dialect at inference time. A more subtle failure mode: the `generate_sql` phase includes executing the candidate query and observing results. Different database engines may return results in different formats or with different column name conventions. The model's ability to interpret execution feedback — which is part of the learned interaction policy — may not transfer across dialects if the feedback format changes. **What evidence exists.** None. The paper does not report any cross-dialect evaluation, even zero-shot. The Spider 2.0 experiment (Appendix D.3, Table 12) uses the "SQLite subset" specifically, which means even the most realistic enterprise benchmark is constrained to SQLite. No ablation tests whether the learned exploration strategy (querying `sqlite_master` to discover tables) would generalize to a different metadata interface without retraining. The ground-truth schema extraction pipeline (Appendix A.3), which uses GPT-4.1, LongCat-Flash, and Gemini-2.5-Pro, operates on ground-truth SQL `y*` that is also SQLite-specific, so even the Schema Reward's supervision signal assumes SQLite semantics. **Mitigation status.** Not addressed. The paper identifies this as future work and provides no evidence about transferability. A practitioner deploying TRUST-SQL against a PostgreSQL or MySQL database would need to either retrain from scratch on dialect-appropriate benchmarks (which currently don't exist at comparable scale and quality to BIRD and Spider) or trust that zero-shot transfer works — a gamble with no supporting evidence in the paper. --- ### 6.3 Fixed Training Turn Budget Creates a Hard Cap on Exploration Capability and a Tension with Inference-Time Scaling **The constraint.** The maximum interaction turn `T` is fixed at training time — the optimal configuration identified in Section 5.3 is `T = 10` turns. The paper acknowledges this limitation directly: > "The maximum interaction turn T is fixed at training time, which may limit exploration thoroughness for databases with exceptionally complex schemas. Adapting the turn budget dynamically based on database complexity remains an interesting direction for future work." **The consequence.** This creates two related problems. First, there is a hard capability ceiling: databases more complex than anything in the training distribution — requiring more than 10 interaction turns to adequately explore — will be partially or completely unsolvable by the trained policy, because the policy has never seen trajectories longer than 10 steps and has not learned how to allocate exploration effort across a longer horizon. The Spider 2.0 results (Appendix D.3, Table 12) hint at this: on enterprise-grade databases, TRUST-SQL-8B achieves 14.8% greedy accuracy — better than the best open-source baseline but far below saturation, and the 10.1% gap between greedy and Pass@8 (24.9%) suggests the model often generates trajectories that run out of turns before completing adequate exploration. With more complex schemas, this gap would likely widen. Second, the tension between training and inference budgets (Figure 6b, Section 5.3) reveals brittleness. The 10-turn-trained policy benefits from additional inference turns (peak at 15-turn inference), which the paper interprets positively as evidence that "the agent effectively utilizes extra test-time compute to recover from early exploration mistakes." But this cuts both ways: the policy was never optimized for 15-turn horizons, and the 12-turn-trained policy degrades severely (training becomes unstable, average turns spike, accuracy declines). This means there is no reliable way to extend the policy's exploration horizon beyond the training budget — attempting to train at T=12 produces a worse policy, and inference at T=15 with a T=10-trained policy is an untested extrapolation whose behavior is not guaranteed. A database that requires 12 turns for adequate exploration cannot be handled by training at 12 turns (which fails) or by inference at 12+ turns with a 10-turn policy (which was never trained to handle that horizon). **What evidence exists.** The training turn budget analysis (Figure 6a) shows the collapse at T=12: execution accuracy drops sharply and average turns spike. The training vs. inference interaction experiment (Figure 6b) shows that a 10-turn-trained policy benefits from extended inference (up to 15 turns), but the peak accuracy is achieved at a specific combination (10-turn training, 15-turn inference), and the relationship is not simple — 12-turn inference with a 10-turn policy is better than 10-turn inference, but 12-turn training is worse than 10-turn training regardless of inference budget. The Spider 2.0 results (Table 12) provide circumstantial evidence of turn budget insufficiency, but no direct measurement of how many TRUST-SQL trajectories on Spider 2.0 exhaust the turn budget without success. **Mitigation status.** Not addressed beyond the acknowledgment. The paper does not propose or evaluate any mechanism for dynamic turn allocation — the agent has no way to request additional turns when it recognizes that exploration is incomplete, and the policy has no training signal that penalizes premature confirmation (the `confirm_answer` action is always terminal). Techniques like adaptive computation time, hierarchical policies that can call sub-policies for extended exploration, or curriculum learning that gradually increases the turn budget during training are all left unexplored. --- ### 6.4 The Difficulty Estimation and Data Filtering Pipeline — Critical to RL Training Efficiency — Is Not Ablated and May Introduce Hidden Biases **The constraint.** The RL training data filtering strategy (Appendix A.3) retains questions only if the SFT-initialized policy achieves a pass rate strictly below 6/8 on 8 rollouts — questions already "easy" for the SFT model are excluded from RL training. The paper reports a 64.4% keep rate (11,642 retained from 18,078 candidates). This filtering is critical to the training pipeline because, as the paper states, questions already nearly solved "provide negligible learning signal for RL, and including them would waste training compute." The ground-truth schema extraction (`K*` for computing `R_schema`) uses a multi-model consensus strategy (GPT-4.1, LongCat-Flash, Gemini-2.5-Pro, requiring at least 2/3 agreement) which is similarly critical — noisy or incorrect `K*` labels would corrupt the Schema Track's optimization signal. **The consequence.** Both design choices create potential biases that the paper does not measure or control. The difficulty-based filtering systematically excludes exactly the question types where the SFT policy already performs well — which may be the same question types where RL could learn to be *more efficient* (fewer turns, fewer tool calls) rather than more accurate. The RL-trained policy may therefore specialize on harder questions at the expense of efficiency on easier ones, a tradeoff invisible in the aggregate accuracy metric. If the SFT policy's pass rate is correlated with schema complexity (questions with simpler schemas are easier for SFT), the RL training distribution skews toward complex-schema questions, which could explain why TRUST-SQL underperforms schema-prefilled baselines on Spider-Test (simpler schemas, less benefit from exploration) but outperforms on Spider 2.0 (complex schemas, more benefit). This is not necessarily a bug — it may be a feature that concentrates RL training compute where it matters most — but without an ablation comparing filtered vs. unfiltered training, the effect of this filtering on the final policy's behavior is unknown. The multi-model consensus for `K*` extraction is also unvalidated. The paper states that an annotation "is accepted only when at least two out of three models produce consistent results," but does not report the inter-model agreement rate, the percentage of questions where all three models disagree (meaning no `K*` is extracted), or the accuracy of the consensus `K*` relative to human-annotated ground truth. If the consensus `K*` is noisy — e.g., missing a necessary column that all three models happened to omit — the Schema Reward will incorrectly penalize trajectories that correctly propose that column, introducing systematic noise into the Schema Track's optimization signal. The 0.5 point gap between Sparse and Dense schema rewards (Figure 5) could partially reflect noise in the `K*` labels rather than a genuine optimization difference between reward formulations. **What evidence exists.** The paper reports the filtering statistics (Table 7: 11,642 retained, 6,436 rejected, 64.4% keep rate) and describes the consensus strategy (Appendix A.3) but provides no ablation or validation of either component. There is no comparison of RL training with vs. without difficulty filtering, and no measurement of `K*` extraction accuracy (e.g., agreement with human annotations on a held-out subset). The question difficulty distribution of retained vs. rejected questions is not reported, so it is impossible to assess what types of questions are systematically excluded. **Mitigation status.** Not addressed. The paper does not ablate the filtering threshold, validate the `K*` extraction quality, or acknowledge these as potential sources of bias. Both components are engineering decisions that are sensible and well-motivated but whose impact on final performance is assumed rather than measured. --- ### 6.5 The Single Model Family Evaluation Limits the Generality of the Central Finding **The constraint.** All experiments — the pilot study, the main benchmark results, every ablation, the cost analysis, the Spider 2.0 evaluation — use Qwen3 (4B and 8B variants) as the base model family. The paper does not report any results with Llama, DeepSeek, Mistral, Gemma, or any other model architecture. The justification for Qwen3 is that "these models are representative of contemporary LLMs at their respective scales" (Section 4), but this claim is not tested. **The consequence.** The paper's central finding — that active exploration can substitute for schema prefilling, and that the Propose checkpoint eliminates hallucination — may be partially or entirely specific to Qwen3's pre-training characteristics. Different model families exhibit different tendencies toward hallucination, different in-context learning capabilities, and different baseline schema-linking accuracy. A model family that is less prone to parametric hallucination (e.g., one trained with more aggressive factuality objectives) might show a smaller benefit from the Propose checkpoint, because it hallucinates less even without structural constraints. Conversely, a model family with stronger in-context reasoning might learn effective exploration strategies with less structured training, reducing the gap between Dual-Track GRPO and single-track approaches. The Schema Prefill result — that injecting the full schema into TRUST-SQL provides no benefit and sometimes hurts — is particularly vulnerable to this concern. If Qwen3 has a specific tendency to be distracted by large contexts or to over-rely on pre-loaded information at the expense of value-level probing (as the case study in Appendix E demonstrates), the "prefilling as noise" finding may not replicate on model families with different attention patterns or context utilization characteristics. A practitioner using Llama or DeepSeek as their base model cannot assume that TRUST-SQL would exhibit the same indifference to schema prefilling, nor that the optimal `λ` value (0.25) would transfer. The two model scales tested (4B and 8B) differ by only 2× in parameter count, and the diminishing returns pattern — the 8B model's 16.6% average improvement is roughly half the 4B model's 30.6% — suggests that TRUST-SQL's benefit shrinks as base model capability increases. It is unknown whether this trend continues at larger scales (14B, 32B, 70B) to the point where the benefit becomes negligible, or whether it asymptotes at some positive value. If the benefit is largest at small-to-medium scales where the base model has sufficient reasoning ability to execute structured exploration but insufficient capability to succeed without it, then TRUST-SQL's value proposition is strongest for on-device or edge-deployed models and weakest for datacenter-scale models — a nuanced deployment consideration that the paper's data cannot confirm or refute. **What evidence exists.** The paper reports two model sizes from one family. The diminishing returns between 4B and 8B are visible in Table 2: the 4B model gains 35.6% absolute on BIRD, the 8B gains 17.9% (from a higher baseline of 47.9%). But this is insufficient to extrapolate to other model families or larger scales. The pilot study (Section 3.1) establishing the Propose checkpoint's必要性 uses Qwen3-8B only — no other model's hallucination rate without the checkpoint is reported. **Mitigation status.** Not addressed. The paper does not acknowledge the single-model-family limitation or suggest cross-family replication as future work. This is a significant gap for a paper whose primary contribution is a training methodology claimed to be general. --- ### 6.6 The Untested Assumption That Schema Reward Quality Is Robust to Extraction Noise **The constraint.** The Schema Track depends entirely on the quality of `K*` — the ground-truth minimal schema extracted from the ground-truth SQL `y*` — which is computed automatically via multi-model consensus (Appendix A.3). The paper states that `K*` extraction is performed by three strong models (GPT-4.1, LongCat-Flash, Gemini-2.5-Pro) with a 2/3 agreement criterion, but provides no validation of this pipeline against human annotations. The Schema Reward `R_schema(Â, K*)` (Equation 3) is then computed as a binary match between the agent's proposed schema `Â` and this automatically extracted `K*`. **The consequence.** This creates a silent failure mode: if `K*` is systematically biased — e.g., the extraction models consistently omit certain types of columns, or consistently include unnecessary columns — the Schema Track will optimize toward a distorted target. The model may learn to propose schemas that match the *extracted* ground truth, which may not be the *actual* minimal schema necessary to answer the question. In the best case, this introduces noise into the training signal, slowing convergence or capping the achievable Schema Track performance. In the worst case, it teaches the model to systematically under- or over-specify schemas relative to the true task requirements — a bias that would persist even if the model's exploration is otherwise perfect. This limitation is particularly concerning because schema extraction is a non-trivial task. Determining the *minimal* set of tables and columns necessary for a SQL query requires understanding the query's semantics, distinguishing columns that are functionally necessary from those that are merely referenced (e.g., in SELECT clauses that could be rewritten), and correctly handling aliases, subqueries, and views. The three extraction models may agree frequently — but if they share common failure modes (e.g., both GPT-4.1 and Gemini-2.5-Pro may handle nested subqueries in similar ways), the consensus criterion provides false confidence. The coupling of `R_schema` to `R_exec = 1.0` (Section 5.2) partially mitigates this by ensuring that incorrect `K*` extraction only corrupts the Schema Track signal for questions where the agent's SQL execution happened to fail for other reasons — if `K*` is wrong but `R_exec = 1.0` anyway (because the agent's SQL matched the execution result despite proposing a non-minimal schema that `K*` would penalize), the schema reward would be withheld despite task success, creating a conflicting signal. The frequency of this scenario depends on the extraction pipeline's error rate, which is unmeasured. **What evidence exists.** None. The paper describes the extraction procedure but provides no evaluation of its accuracy. No comparison of extracted `K*` against human-annotated schemas (even on a small subset) is reported. No ablation tests whether training with oracle (human-verified) `K*` produces a better policy than training with consensus-extracted `K*`. The paper's conclusion that a binary schema reward outperforms a dense one (Section 5.2, Figure 5) could be influenced by extraction noise — a dense reward might amplify noise in `K*` more than a binary reward, and the observed 0.5 point gap might reflect differential sensitivity to label quality rather than an inherent property of the reward formulation. **Mitigation status.** Not addressed. The paper acknowledges the consensus strategy as a design choice but does not validate it, does not measure its error rate, and does not ablate it against stronger baselines (human annotation, single-model extraction, oracle extraction from database metadata directly). A practitioner implementing TRUST-SQL would need to either replicate the multi-model consensus pipeline exactly (with the same three models and the same agreement threshold) or validate their own extraction pipeline independently — the paper provides no guidance on what accuracy level is sufficient for effective Schema Track training. ## 7. Implications and Future Directions ### How This Work Changes the Landscape This paper effects a genuine **paradigm shift in how the Text-to-SQL community conceptualizes the task itself**, not merely an incremental improvement to an existing pipeline. The shift is from "the schema is given" to "the schema must be discovered" — and the paper demonstrates that this reframing is not a concession to messy real-world conditions but a **more robust foundation** that can match or exceed the performance of systems with privileged schema access. The magnitude of this shift is easiest to see through the asymmetry the paper documents. Base models without schema prefilling collapse entirely — Qwen3-4B drops 17.0 absolute percentage points on BIRD-Dev when the schema is withheld (Table 2, Section 4.3) — while TRUST-SQL with its active exploration policy achieves the same accuracy regardless of whether the schema is pre-injected (64.9% vs. 64.8% for the 4B variant). This inverts the conventional wisdom: the field has implicitly treated schema prefilling as the gold standard that all methods should aspire to use, and "unknown schema" as a degraded fallback scenario. TRUST-SQL demonstrates that the relationship is more nuanced — on robustness benchmarks like Spider-Syn and Spider-Realistic, the actively exploring agent actually *outperforms* prefilled baselines (74.7% vs. 69.7% on Spider-Syn for the 4B model, Table 1), because iterative, verification-driven discovery is inherently more robust to surface-level perturbations than one-shot schema linking from a large static context. This finding reframes the research question from "how do we fit more schema into the context window?" to "how do we make the agent discover the right schema more efficiently?" — a question that connects Text-to-SQL to the broader literatures on information-gathering agents, exploration under partial observability, and tool-integrated reasoning. The POMDP formulation in Section 3.2 is the formal expression of this reframing, and it opens the door to techniques from reinforcement learning, active learning, and Bayesian experimental design that were previously irrelevant to the Text-to-SQL community. **Reconciling prior contradictions.** The paper's pilot study (Section 3.1, Figure 3) provides a mechanistic explanation for conflicting claims in the prior literature about whether multi-turn interaction helps or hurts Text-to-SQL. The answer depends on *what structural constraints are imposed*. In the minimal EC variant (free exploration, direct SQL submission), interaction without constraints actually enables hallucination — the agent fabricates 26.4% of its failures as non-existent schema elements. Adding execution feedback (EGC) partially mitigates this (14.2% hallucination), but only the explicit Propose checkpoint (EPGC) nearly eliminates it (2.8%). Prior work that found negative or mixed results for interactive Text-to-SQL likely lacked this verification checkpoint; prior work that found positive results may have implicitly enforced similar grounding through their specific prompting or evaluation design. The paper resolves this apparent contradiction by identifying the Propose phase as the **necessary and sufficient structural condition** for interaction to be helpful rather than harmful. **Research directions that become more attractive.** The paper's central finding — that active exploration can substitute for, and sometimes outperform, schema prefilling — makes **verifier design for exploration quality** a first-class research priority. The Schema Track in Dual-Track GRPO demonstrates that providing an independent optimization signal for exploration quality yields measurable gains (9.9% relative improvement over standard GRPO, Section 5.1), but the Schema Reward itself depends on an automatically extracted `K*` whose quality is unvalidated. Improving schema extraction accuracy, developing more sophisticated `f_match` functions, and exploring alternative auxiliary signals (information gain, coverage, efficiency) for exploration all become high-leverage research investments — each improvement to the Schema Reward directly translates to better exploration policies. The robustness benchmark results also make **adversarial schema perturbation** a newly important evaluation dimension. The fact that TRUST-SQL gains its largest advantages on Spider-Syn (synonym substitution) and Spider-Realistic (ambiguity resolution) suggests that active exploration provides a form of robustness that static schema linking cannot replicate. This implies that future Text-to-SQL benchmarks should include schema perturbation tests as standard, and that methods should be evaluated on their ability to *verify* rather than merely *match* schema elements. The Spider 2.0 results (Appendix D.3, 14.8% greedy accuracy from TRUST-SQL-8B vs. 15.6% from GPT-4o with full schema access) further validate that enterprise-grade schema complexity is a distinct challenge where exploration-driven approaches have room to lead. **Research directions that become less attractive.** The paper's results diminish the case for two previously active lines of work. First, **schema compression and retrieval methods** that aim to fit ever-larger schemas into fixed context windows (e.g., schema linking with learned embeddings, hierarchical schema representations, schema summarization) become less compelling if an agent can discover the necessary subset through interaction. The paper shows that for TRUST-SQL, full schema prefill actually *degrades* performance on robustness benchmarks (Table 2), suggesting that the problem is not "how to compress the schema" but "how to ignore irrelevant schema" — and interaction solves the latter more directly than compression solves the former. Second, **training-free exploration approaches** that rely on prompting frozen models to use database tools face a harder justification after TRUST-SQL's results. The pilot study (Section 3.1) shows that even a structured prompt (the EC variant) cannot prevent an untrained model from hallucinating at a 26.4% rate, and the RL-only ablation (Section 5.4, Table 3) shows that without SFT warm-up, the model discovers a degenerate strategy that bypasses genuine exploration entirely. These results collectively demonstrate that exploration discipline — the willingness to verify before generating — is a learned behavior that requires gradient-based training, not merely a prompting convention. Training-free approaches may still have value for rapid prototyping or zero-shot evaluation, but the paper strongly implies that deployment-grade systems will require trained exploration policies. --- ### Follow-Up Research This Work Enables **Cross-model-family replication to establish the generality of active exploration as a substitute for schema prefilling.** The paper's central claim — that active exploration can match or exceed schema prefilling — is demonstrated exclusively on Qwen3 models. This is a direct consequence of the single-model-family evaluation limitation, but it points to a critical follow-up: does the same pattern hold for Llama, DeepSeek, Mistral, or Gemma? A strong study would replicate the full TRUST-SQL pipeline (SFT warm-up on the same 9.2k trajectories, Dual-Track GRPO on the same 11.6k RL questions) on at least two additional model families at comparable scales (7B–8B parameters). The key measurement would be the Schema Prefill ablation from Table 2: for each model family, does injecting the full schema into the trained agent produce negligible or negative effects, or do some model families still benefit from prefilling? A negative result — e.g., Llama-3-8B-trained TRUST-SQL still gains +5 points from schema prefilling — would suggest that Qwen3's specific pre-training biases (perhaps toward hallucination under uncertainty, or toward reliance on context-provided information) are responsible for the "prefilling as noise" effect, and that the exploration-prefilling substitution is not a universal property. A positive result across two additional families would substantially strengthen the paper's paradigm-shift claim. **Validating the schema extraction pipeline to quantify the noise floor of the Schema Reward.** The Schema Track's optimization signal depends entirely on `K*` — the ground-truth minimal schema extracted from `y*` via multi-model consensus (GPT-4.1, LongCat-Flash, Gemini-2.5-Pro, requiring 2/3 agreement, Appendix A.3). The paper provides no validation of this pipeline against human annotations. A direct follow-up would sample 200–300 questions from the RL training set (stratified by difficulty and schema complexity), have 2–3 human SQL experts independently annotate the minimal necessary schema, and compute: (1) the inter-model agreement rate among the three extraction models, (2) the agreement rate between the consensus `K*` and the human-annotated gold standard, and (3) the types of systematic errors the extraction pipeline makes (e.g., consistently omitting columns used only in WHERE clauses, consistently including columns referenced in SELECT but not functionally necessary). If the extraction error rate is above 5–10%, the next step would be to retrain TRUST-SQL with human-verified `K*` and measure whether the Schema Track produces a meaningfully different policy (higher accuracy? fewer turns? different exploration patterns?). This would establish a quality threshold for schema extraction in Dual-Track RL systems, providing essential guidance for practitioners. **Dynamic turn budget allocation to address the fixed-horizon brittleness documented in Figure 6a.** The paper shows that training at T=10 turns is optimal, T=12 turns causes training instability, but inference at T=15 turns with a T=10-trained policy yields the best performance. This reveals a fundamental tension: the optimal inference horizon exceeds the maximum trainable horizon. A strong follow-up would implement an adaptive turn budget: the agent maintains a running estimate of its schema coverage (e.g., fraction of proposed tables that have been verified through value-level probing) and is allowed to request up to N additional turns when coverage is below a threshold. The training objective would penalize unnecessary turn extensions (e.g., a small negative reward per additional turn requested). The evaluation would measure whether the adaptive policy can solve Spider 2.0 questions that require more than 10 turns of exploration without collapsing into the degenerate over-exploration behavior observed at λ=0.375 (Figure 4b) or the instability at T=12 (Figure 6a). This directly addresses the practical limitation described in Section 6.3 while building on the paper's existing infrastructure. **Combining exploration with multi-model diversity to push the Pass@K ceiling.** The Pass@K analysis (Figure 6c, Table 11) reveals a persistent gap between greedy and Pass@8 performance — e.g., 64.9% greedy vs. 75.1% Pass@8 for TRUST-SQL-4B on BIRD-Dev — indicating the model can generate correct solutions through repeated sampling but cannot consistently identify them. A follow-up could train a separate verifier model (analogous to the PRM in the example paper's test-time compute framework) that scores partial trajectories at the Propose checkpoint, predicting whether the proposed schema is complete and correct. At inference time, the system would sample K parallel exploration trajectories, use the verifier to select the most promising schema proposals, and then generate SQL from only the top-M proposals — a best-of-K-weighted strategy applied to the Schema Track. The evaluation would measure whether this approach closes the greedy-to-Pass@K gap more efficiently than simply increasing K, and whether the verifier's quality itself improves with more training data (scaling behavior). This connects TRUST-SQL directly to the test-time compute scaling literature and tests whether the exploration diversity captured by repeated sampling can be filtered rather than merely averaged. **Negative result: probing the limits of the Propose checkpoint's anti-hallucination effect on out-of-distribution database structures.** The pilot study (Section 3.1) demonstrates the Propose checkpoint reduces hallucination by 9.4× on BIRD-Dev schemas, but the evaluation databases share similar naming conventions and structural patterns. A stress test would construct a synthetic benchmark of 200–300 databases with deliberately adversarial naming: table and column names that conflict with common-sense priors (e.g., a table named `users` that does not contain user IDs, a column named `price` that stores quantities, foreign keys that reference non-obvious parent tables). A model relying on parametric priors would be strongly pulled toward hallucinating the "obvious" schema rather than verifying the actual one. The experiment would measure whether TRUST-SQL's hallucination rate spikes on this adversarial benchmark, and whether the effect is mitigated by simply increasing the inference turn budget (giving the agent more opportunities to verify) or whether the Propose checkpoint's grounding effect is fundamentally dependent on the schema being *plausible enough* that the agent is willing to verify rather than assume. A negative result — TRUST-SQL hallucinates at >20% on adversarial schemas despite the Propose checkpoint — would establish a boundary condition for the paper's approach and motivate research into explicit "verify before trust" training objectives beyond the structural checkpoint alone. --- ### Practical Applications and Downstream Use Cases **On-device or edge-deployed database interfaces constrained by context window size.** The paper's most immediately actionable finding for practitioners is that TRUST-SQL-4B, a compact model trained with only 9.2k SFT + 11.6k RL samples, achieves 64.9% execution accuracy on BIRD-Dev without any pre-loaded schema — matching the 63.1% of MTIR-SQL-4B *with* full schema access. In deployment scenarios where the database schema is too large to fit in the context window of a small on-device model (e.g., mobile SQL assistants, embedded analytics in IoT devices, offline-capable enterprise tools), the conventional approach — schema compression or retrieval — adds latency and complexity. TRUST-SQL offers a simpler architecture: deploy the small model with database query access, and let it discover what it needs. The cost analysis (Table 10, Appendix D.1) is encouraging in this context: 0.6 seconds latency and 2.83K output tokens per query, compared to 251.3 seconds and 320.8K tokens for the training-free pipeline method CHESS. The 500× latency reduction and 113× token reduction make TRUST-SQL deployable in interactive settings where CHESS is not. **Robust Text-to-SQL for databases with evolving or noisy schemas.** The paper's robustness benchmark results — particularly the 74.7% vs. 69.7% advantage over the best prefilled baseline on Spider-Syn (synonym-substituted table and column names) for the 4B variant — directly apply to production databases undergoing migration, renaming, or restructuring. In enterprise environments where database schemas evolve through regular migrations (additions, deletions, renames; see Zhang et al., 2026, cited in the paper's introduction), a schema-prefilled system using stale metadata will silently fail on renamed columns. TRUST-SQL's active exploration policy, which probes actual values and verifies column names through live queries, is inherently robust to these changes — the agent discovers the current schema at query time, not the schema that was current when the context was constructed. The Schema Prefill result (Table 2) showing that pre-injecting the schema actually degrades TRUST-SQL's performance on Spider-DK (−2.4% for 4B) and Spider-Syn (−2.2%) further supports this use case: the system works better when it *doesn't* have stale or overwhelming schema information, precisely the condition that holds in evolving production databases. **Cost-efficient data generation for Text-to-SQL self-improvement pipelines.** The RL training data filtering strategy (Appendix A.3) — retaining only questions where the SFT policy achieves pass rate < 6/8 — provides a template for data-efficient self-improvement. In a production Text-to-SQL system, the vast majority of user queries may be "easy" (simple schemas, straightforward mappings) and provide negligible training signal. TRUST-SQL's filtering approach can be applied online: deploy the agent, log trajectories and outcomes, and only retain for fine-tuning those queries where the agent struggles (execution failure or excessive turn count). The paper's demonstration that 9.2k SFT + 11.6k RL samples suffice for strong performance (compared to OmniSQL's 2.5M SFT samples) suggests that targeted, difficulty-filtered data collection is dramatically more sample-efficient than indiscriminate data synthesis. A production pipeline using TRUST-SQL's filtering strategy with a continuously updating model could achieve strong performance with a fraction of the annotation cost of large-scale synthetic data generation. --- ### When to Prefer This Method The paper articulates a clear tradeoff between TRUST-SQL's active exploration approach and two alternatives: **single-turn methods with schema prefilling** (the dominant paradigm) and **training-free tool-augmented frameworks** (like CHESS and MAC-SQL). The decision rules emerge directly from the experimental results: - **Prefer TRUST-SQL over schema-prefilled single-turn models when:** (1) the database schema is too large or too dynamic to reliably pre-load into context (the Unknown Schema setting that motivates the paper); (2) the deployment environment has restricted context windows (on-device models, edge deployment); (3) robustness to schema perturbations is critical — TRUST-SQL-4B outperforms the best prefilled baselines by 5.0% on Spider-Syn and 1.2% on Spider-Realistic (Table 1); (4) the inference latency budget tolerates ~0.6 seconds per query with 5–6 sequential database round-trips (Table 10). The 30.6% average improvement over base models without schema prefilling (Table 2) makes this a strong preference when schema availability is constrained. - **Prefer TRUST-SQL over training-free frameworks when:** (1) hallucination suppression is critical — training-free agents lack the learned verification discipline that reduces hallucination from 26.4% to 2.8% (pilot study, Figure 3); (2) inference cost matters — CHESS consumes 500× more latency and 113× more tokens than TRUST-SQL-4B for lower accuracy on BIRD-Dev (Table 10: 61.5% vs. 64.9%); (3) the deployment requires a compact model (3B–8B parameters) — training-free frameworks rely on much larger frozen models for tool-use capability. The 9.9% relative improvement from Dual-Track GRPO over standard GRPO (Section 5.1) is realized only with training; training-free agents cannot benefit from this credit assignment mechanism. - **Prefer schema-prefilled single-turn methods when:** (1) latency is the absolute binding constraint and single-turn generation (0.4 seconds, Table 10) is required; (2) the schema is small, static, and reliably known at deployment time — under these conditions the prefilled baselines like OmniSQL-7B are competitive (63.9% on BIRD) and simpler to deploy; (3) the database dialect is not SQLite — TRUST-SQL's training is SQLite-only, while single-turn models trained on synthetic data may have broader dialect coverage. The paper does not test TRUST-SQL on non-SQLite databases, making single-turn models the safer choice for PostgreSQL, MySQL, or other dialects until cross-dialect training is demonstrated. - **Prefer neither when:** the base model's capability is fundamentally insufficient for the task difficulty. The paper's diminishing returns from 4B to 8B (30.6% vs. 16.6% average improvement over base models, Section 4.3) suggest that TRUST-SQL's benefit is largest when the base model has enough reasoning ability to execute structured exploration but not enough to succeed without it. On extremely complex enterprise schemas (Spider 2.0, 14.8% greedy accuracy from TRUST-SQL-8B), both exploration and generation remain challenging, and neither approach comes close to solving the problem — larger-scale models or fundamentally different architectures may be necessary.