Universal Deep Research: Bring Your Own Model and Strategy

🎯 Pitch

Existing deep research tools rigidly lock you into one model and one hard-coded search strategy. Universal Deep Research breaks this open by letting you write plain-language strategies that compile into code, turning any LLM into a deep research agent without fine-tuning—and it all fits in an 8k context window.

1. Executive Summary

This paper introduces Universal Deep Research (UDR), a generalist agentic system that wraps around any language model without additional fine-tuning and enables users to create, edit, and refine entirely custom deep research strategies. UDR operates by converting user-written natural-language research strategies into executable code—treating LM reasoning as a callable utility for localized tasks like summarization or ranking rather than as the orchestrator of the full research process, and maintaining all intermediate state in code variables to operate within small context windows (8k tokens sufficed across all workflows). The system is demonstrated with three example strategies—minimal, expansive, and intensive—on a range of research prompts using Llama 3.3 70B, establishing that it is feasible to attach a well-functioning deep research tool to virtually any general-enough generative language model while giving the end user programmatic control over the research strategy, though the paper finds that devising sufficiently sophisticated strategies remains a tedious process for end users.

2. Context and Motivation

The Core Problem: Deep Research Tools Are Rigid and Non-Composable

The paper addresses a specific architectural limitation in the rapidly emerging class of tools known as deep research tools (DRTs). A DRT, by the paper's definition, is a system that takes a research prompt from a user, conducts extensive search across available resources relevant to the task, and produces a structured, formatted research report—typically containing headings, tables, extensive formatting, and references to sources (Section 1). These tools have quickly become popular for search-intensive tasks in both professional and personal contexts.

The fundamental gap the paper identifies is that every existing DRT is hard-coded to a particular research strategy and a particular underlying language model, with essentially no room for user customization beyond the research prompt itself. The authors frame this through three concrete problems (Section 1, "Problem statement"):

P1 — Restricted user control over the research process. Users of existing DRTs cannot enforce a hierarchy of preferred resources, automate cross-validation of information against reputable sources, or control the expenses associated with each search operation. The paper argues this restriction contributes directly to the functionality gap between consumer-oriented and enterprise-oriented DRTs. Consumer tools like Gemini, Perplexity, and OpenAI Deep Research operate with wide resource scopes (the entire web), while enterprise tools like NVIDIA AI-Q and SambaNova Deep Research must operate within limited internal document databases and therefore require more specialized, structured approaches. The inability to customize strategy means neither class of tool can adapt to the other's use case.

P2 — Inability to create specialized document research strategies. In high-value industries—finance, legal, healthcare, real estate, government and public administration—research workflows often require specific, repeatable methodologies that existing DRTs cannot accommodate. The paper argues this leaves large portions of professional research workloads to be automated by "costly agentic solutions made to measure" (Section 1). Essentially, organizations must either accept a generic research strategy that doesn't fit their needs or build entirely custom agentic systems from scratch—there is no middle ground where a general tool can be configured with domain-specific strategies.

P3 — Models and tools are not interchangeable. The user of an existing DRT cannot swap the underlying language model for a more recent or more capable one, nor can they pair a particular research agent with a model of their choice. Models and DRTs are bundled together, preventing "unspecialized competition among models and DRTs independently of each other" (Section 1). This coupling prevents the natural ecosystem dynamic where model improvements and strategy improvements can advance independently and be combined arbitrarily.

Why This Problem Matters: Real-World Impact

The paper makes the case that resolving these three problems has direct practical consequences (Section 1, "Problem importance and impact"):

Resolving P1 would increase research report quality for individual users and help close the consumer-enterprise DRT gap, making powerful enterprise-style research methodologies available to anyone who can write a natural-language strategy.
Resolving P2 would enable automation of substantial amounts of high-value, labor-intensive specialized research in regulated and expertise-heavy industries. This represents not just a convenience improvement but a potential structural shift in how professional research is conducted—moving from bespoke per-organization agentic builds to a common platform that can be configured declaratively.
Resolving P3 would allow the most competitive models to be paired with the most competitive research strategies independently, creating a composable ecosystem akin to separating a database engine from the query planner. This would also accelerate iteration: improvements to strategies could be tested across model families, and improvements to models could be evaluated against a fixed strategy baseline.

There is also a theoretical significance to the gap. The trend in agentic systems over recent years has been toward increasing autonomy—models that plan, reason, and decide their own actions. UDR pushes in the opposite direction: it asks what happens when the user retains explicit, programmatic control over the agent's behavior, reducing the LM from orchestrator to utility. This explores an under-examined point in the design space of agentic architectures, one that separates the what (the research strategy) from the how (the code execution) from the who (the language model performing localized reasoning). The paper's demonstration that this separation is feasible and functional is itself a contribution to understanding agentic system design tradeoffs.

Prior Approaches and Where They Fall Short

The paper surveys both the general and enterprise landscapes of existing DRTs (Section 1, "General landscape" and "Enterprise landscape") to establish what exists and what is missing. Critically, the paper does not claim existing DRTs don't work—they do work, and are popular—but rather identifies a systematic design choice that limits their flexibility.

General-Purpose DRTs: Fixed Iterative/Expansive Strategies

The paper describes three major consumer DRTs, each of which employs a research strategy that cannot be modified by the user:

Gemini Deep Research (Google). Transforms user prompts into a research plan and browses the web autonomously, at each step finding information and starting new searches based on previous results. The core mechanism is iterative: search results inform subsequent searches, and this loop continues until the system deems sufficient data has been gathered. The strategy—how many iterations, when to stop, how to evaluate sufficiency—is entirely hard-coded.

Perplexity Deep Research. Similarly operates with an autonomous iterative process. The paper does not detail differences from Gemini's approach, suggesting they share the same fundamental architecture of fixed-strategy iterative web research.

OpenAI Deep Research. Again transforms prompts into research plans with autonomous browsing. Like Gemini and Perplexity, the research strategy is embedded in the system design and not exposed to the user.

Grok 3 DeepSearch (xAI). Employs a different architecture: a two-tier crawling system where a distributed network of crawler bots continuously indexes the web, and an on-demand agent performs targeted searches by generating specific sub-queries and fetching relevant pages in real time. The system additionally leverages chain-of-thought LM reasoning to evaluate source credibility and cross-verify claims across multiple sources. Despite the architectural differences, the research strategy is again fixed—users cannot modify the crawling behavior, the credibility evaluation criteria, or the cross-verification methodology.

The unifying characteristic across all these tools is that the research strategy is a fixed part of the product. Users provide a prompt and receive a report; the intermediate process is a black box. There is no mechanism for a user to say, "search these sources first, then cross-validate against these," or "when researching this topic, prioritize government databases over news articles," or "do not spend more than N search operations on this query."

Enterprise DRTs: Specialized but Equally Rigid

Enterprise-oriented DRTs are designed for the constrained environment of internal document databases rather than the open web, and as a result employ more specialized strategies. But they share the same fundamental rigidity:

NVIDIA AI-Q Research Assistant. Follows a structured five-step process: (1) creating a report plan from the prompt, (2) searching data sources, (3) writing a report, (4) reflecting on gaps for further queries if needed, and (5) finishing with a comprehensive list of document sources. This is a more structured and transparent pipeline than the consumer tools, with explicit gap-reflection and source-listing steps. However, the five-step pipeline is fixed—a user whose research needs require a different sequence (e.g., source-listing before gap analysis, or different stopping criteria) cannot modify it.

SambaNova Deep Research. Operates through a document-oriented five-step pipeline: (I) parsing the prompt for research scope, (II) generating a detailed document outline with section-level planning, (III) gathering information using web tools and APIs, (IV) delegating tasks to specialized research agents (such as a financial agent for hierarchical analysis), and (V) synthesizing findings into Markdown-formatted reports with citations. The delegation step (IV) is notable for introducing specialized sub-agents, but the overall pipeline structure and agent specialization logic are built into the system.

ERP AI Deep Research. Takes an entirely different approach, employing a "Graph-Based AI Architecture" that represents enterprise data through knowledge graphs accessed using Graph Neural Networks. It performs rounds of information aggregation across relevant graph elements rather than traditional document/web traversals combined with LLM text comprehension. This is the most idiosyncratic of the surveyed tools, fundamentally graph-oriented rather than document-oriented, making it the least adaptable to general-purpose research.

The enterprise tools demonstrate that specialized research strategies are valuable and necessary—the existence of finance-specific agents, graph-based architectures, and gap-reflection steps proves that different domains benefit from different research methodologies. But each tool bakes its strategy into its architecture. An organization that needs elements from multiple tools (say, SambaNova's section-level planning combined with AI-Q's gap reflection) has no way to compose them.

The Deeper Architectural Problem: LM-as-Orchestrator

Beneath the surface-level observation that strategies are fixed lies a deeper architectural choice shared by most existing DRTs: the language model serves as the orchestrator of the entire research process. The LM decides what to search, when to stop searching, how to evaluate sufficiency, and when to begin report generation. This has two consequences:

Model dependence: If the orchestration logic is expressed through the LM's reasoning capabilities, changing the underlying model changes the research behavior, even if the strategy is nominally the same. The strategy and the model are inextricably linked.
Opacity: Because the LM's decision-making is not fully interpretable, the user cannot audit why certain searches were performed, why certain sources were prioritized, or why the research stopped when it did.

The paper argues that both consequences are undesirable—users should be able to specify strategy independently of model choice, and they should be able to audit and understand the research process.

How This Paper Positions Itself

UDR is positioned not as a better DRT in the conventional sense (better reports, faster search, more accurate citations) but as a different architectural paradigm that resolves P1–P3 by design. The paper frames its contribution around three design decisions that distinguish it from existing work:

1. Strategy as explicit, user-authored input. Rather than embedding the research strategy in the system architecture (as existing DRTs do) or in the LM's prompt (as LM-as-orchestrator approaches do), UDR makes the research strategy a first-class input alongside the research prompt. The user writes a natural-language description of the research procedure they want followed, and the system converts it into executable code. This directly resolves P1 (user control) and P2 (specialized strategies): any strategy expressible in natural language can be executed, with no architectural limitations on what sequences of steps, cross-validation logic, or resource hierarchies are permissible.

2. LM-as-utility, not LM-as-orchestrator. UDR treats language model reasoning as a callable utility—summarization, ranking, extraction—rather than as the controlling entity. The orchestration of the research process is handled entirely by generated code executing on CPU. This directly resolves P3 (model interchangeability): because the LM is not responsible for strategy execution, any model capable of the localized reasoning tasks specified in the strategy can be substituted. The paper explicitly states that UDR "wraps around any language model" and "does not require additional fine-tuning to function" (Section 1).

3. State in code, not in context. Rather than maintaining a single growing context window as most DRTs do (appending each search result to the conversation history), UDR stores all intermediate information and text fragments as named variables in the code execution state. This architectural choice has several implications: it enables operation within small context windows (8k tokens sufficed regardless of workflow complexity), it makes the research process fully auditable (all intermediate state is inspectable), and it prevents the degradation in LM performance that can occur with very long contexts. Section 2.2 explicitly contrasts this with typical DRTs where "the LM often orchestrates the full research process" through a growing context window.

The paper positions this architecture not as a tentative experiment but as a deliberately chosen alternative to the prevailing LM-orchestrator paradigm. The authors report that earlier prototypes—embedding the strategy directly within a prompt to a reasoning-oriented LM, or decomposing the strategy into individual steps with isolated code fragments—"proved fragile and error-prone" (Section 2.2, "Reliability"). The final approach of generating a single end-to-end function from the strategy description, with enforced comment-code structure, was found to be significantly more reliable, minimizing synchronization overhead and avoiding cascading failures typical of fragmented orchestration.

Scope and target audience. The paper is explicitly a research prototype (Section 1, "Target audience"), not a production-ready consumer tool. Its intended audience is developers working on agentic solutions and members of the research community investigating agentic system architectures. This framing is important: the paper is demonstrating a design point in the space of agentic architectures, not competing with existing DRTs on end-to-end report quality. The example outputs in Appendix B serve as existence proofs that the approach produces reasonable research reports, not as benchmarked claims of superiority.

Relationship to the broader agentic landscape. The paper situates UDR within a broader ecosystem trend that the authors (Belcak, Molchanov) have written about elsewhere—specifically, they cite their own work arguing that "small language models are the future of agentic AI" (Reference 1). UDR can be seen as an architectural instantiation of that argument: by separating orchestration (CPU-executable code) from reasoning (focused LM calls), the system reduces dependence on large, expensive models for the orchestration layer, making it compatible with smaller, cheaper models for the reasoning tasks that remain. The paper does not belabor this connection, but the citation signals continuity with the authors' broader research agenda on agentic system design.

3. Technical Approach

3.1 Reader Orientation

UDR is a system that converts a user's natural-language description of how they want research conducted into executable Python code that orchestrates search operations and language model calls to produce a structured research report. The paper is fundamentally an architectural demonstration paper whose core idea is that by separating the orchestration layer (code execution) from the reasoning layer (LM calls) and making the research strategy a first-class user input rather than a hard-coded system behavior, the system can simultaneously solve three problems: giving users control over strategy (P1), enabling domain-specific methodologies (P2), and allowing arbitrary model-strategy combinations (P3).

3.2 Big-Picture Architecture (Diagram in Words)

The system has four major components:

Strategy Compiler — receives the user's natural-language research strategy and converts it into a single executable Python function using a language model, enforcing a disciplined comment-code structure to ensure fidelity to the user's intent.
Isolated Code Execution Environment — runs the generated strategy function, maintaining all intermediate state (search results, extracted information, partially constructed report) as named Python variables rather than in a growing LM context window.
Tool Interface (Search Function) — exposes search capabilities as synchronous function calls accessible within the execution environment, with results returned as structured data that the strategy code can manipulate.
Language Model Utility Layer — provides the strategy code with callable LM access for localized reasoning tasks (summarization, ranking, extraction, query generation) but explicitly NOT for orchestration decisions, which remain in the generated code.

Information flows as follows: the user provides a research strategy (natural language list of steps) and a research prompt → the strategy compiler generates a Python generator function from the strategy description → the function executes in an isolated environment, making search calls and invoking the LM for specific reasoning subtasks per the strategy's instructions → the function continuously yields progress notifications to the user interface → the function assembles and yields a final research report as the terminal notification.

3.3 Roadmap for the Deep Dive

First, the strategy processing pipeline (Phase 1) — how natural-language strategy descriptions are converted into executable code, including the specific prompting technique that prevents models from taking shortcuts.
Second, the strategy execution environment (Phase 2) — how the generated code runs, how state is managed outside the LM context, how tools are accessed, and how notifications flow to the user.
Third, the design decisions that distinguish UDR from the prevailing LM-as-orchestrator paradigm, including the reliability and efficiency arguments the paper makes.
Fourth, the security considerations inherent in a system that generates and executes user-defined code.
Fifth, the three example strategies (minimal, expansive, intensive) as concrete instantiations that demonstrate the range of research behaviors the system supports. This ordering follows the system's operational phases (compile, then execute) and then steps back to analyze the architectural choices that make the system work.

3.4 Detailed, Sentence-Based Technical Breakdown

Phase 1: Strategy Processing — From Natural Language to Executable Code

The core technical mechanism of UDR is the conversion of a user-written natural-language research strategy into a single, coherent, executable Python function. This is not a trivial prompt-to-code translation; the paper describes specific failure modes encountered in earlier prototypes and the design decisions introduced to overcome them.

Inputs to the strategy compiler. The strategy compiler receives two things from the user: a research strategy (a natural-language list of steps, preferably formatted as a numbered or bullet-point list to facilitate parsing) and a research prompt (specifying the topic, content requirements, and formatting requirements for the desired report). The research strategy defines the behavior of the UDR instance "in its entirety" (Section 2.1)—every decision about how searches are conducted, how information is aggregated, when the LM is invoked, what notifications are sent, and how the report is assembled must be specified in the strategy. There are no implicit or default behaviors beyond what the strategy describes.

The paper provides three complete example strategies in Appendix A, ranging from approximately 10 steps (minimal, Appendix A.1) to a more elaborate iterative strategy with sub-contexts and search refinement loops (intensive, Appendix A.3). These serve both as documentation of the expected input format and as existence proofs of the range of research methodologies the system can express.

The code generation process. The research strategy is passed to a language model together with constraints on the available functions and permitted code structures (Section 2.2, "Phase 1 – Strategy processing"). The model's task is to produce a single callable function that accepts the research prompt as input and continuously returns output notifications. The paper enforces a specific output format: the generated function must be a Python generator (using yield statements), and every notification updating the user on research progress must be a yield statement returning a dictionary with the notification payload. The paper states this requirement explicitly:

"In our implementation, we enforced this behavior by insisting that the generated function returns a generator and that every notification sent to the user that updates them on the progress of the research is a yield statement returning a dictionary with the notification payload."

The shortcut problem and the comment-code discipline. A critical finding emerged during development: when given complete freedom to generate the code implementation, language models frequently took shortcuts — skipping strategy steps, taking liberties at imposing constraints where none were stipulated, and generally producing code that did not faithfully implement the user's intended strategy (Section 2.2, "Phase 1 – Strategy processing"):

"We found that regardless of the choice of the language model, giving it a completely free hand at generating the code implementation of the research strategy often resulted in the model taking shortcuts, skipping strategy steps, and taking liberties at imposing constraints where none were stipulated by the user."

To address this, the paper introduced a step-by-step comment-code discipline: the model is prompted to generate code that corresponds to the strategy step by step, explicitly prepending every segment of generated code with comments laying out the strategy step it corresponds to. This is not merely a documentation practice; it is a structural constraint on code generation that dramatically improved fidelity. The paper characterizes this as all but eradicating the shortcut behavior.

Why this works. The comment-code discipline forces the model to explicitly map each natural-language instruction to a code segment, reducing the degrees of freedom available for "interpretation." Rather than reading the entire strategy and producing code that "feels right" as a whole, the model must justify each code block by referencing the specific strategy step it implements. This is analogous to chain-of-thought reasoning applied to code generation, but with the chain-of-thought made explicit in the output code as comments so that both the model's reasoning and the code's traceability to the strategy are preserved.

Tool description. The available tools — primarily the search function — are described in a docstring contained in the user message to the code-generating model (Section 2.2, "Phase 1 – Strategy processing"). This means the model knows the function signature, parameters, and return format of the search tool when generating code, and can produce valid function calls directly. The paper does not enumerate the full tool API, but from the example strategies in Appendix A, the search function appears to accept a search phrase as input and return results whose contents can be iterated over and appended to context variables.

Validation before execution. Once the research strategy has been converted to code, its "conformance with execution requirements has been confirmed" before it is invoked (Section 2.2, "Phase 2 – Strategy execution"). The paper does not detail the validation procedure — it mentions only "basic syntactic and execution checks" in Section 4 (Limitations) — but the implication is that the generated code must at minimum be syntactically valid Python, define a generator function with the expected signature, and not import or access disallowed modules.

Why single-pass end-to-end generation over alternatives. The paper reports that two alternative approaches were attempted and found wanting:

Embedding the strategy directly within a prompt to a reasoning-oriented language model. This approach made the LM responsible for executing the strategy through its own reasoning, essentially making it the orchestrator. The paper describes this as "fragile and error-prone" (Section 2.2, "Reliability").
Decomposing the strategy into individual steps and generating isolated code fragments for each. This introduced synchronization overhead and risked cascading failures: if one step's code fragment produced output in an unexpected format, subsequent steps could fail silently. The fragmented nature also made it harder to maintain coherent state across steps.

The single-pass generation approach, by contrast, "ensures coherence across steps, minimizes synchronization overhead, and avoids the cascade of failures often seen in fragmented orchestration" (Section 2.2, "Reliability"). The generated code is also "fully interpretable and auditable by the user" — a property that follows from the comment-code discipline and the use of standard Python rather than domain-specific frameworks.

Phase 2: Strategy Execution — Code-Orchestrated Research

Once the strategy function has been generated and validated, it is invoked in an isolated code execution environment (Section 2.2, "Phase 2 – Strategy execution"). This section covers how the generated code operates, how state is managed, how tools are accessed, how LM reasoning is invoked, and how notifications are emitted.

State management: variables, not context windows. The most architecturally significant design decision in UDR is the approach to state management, and the paper devotes substantial attention to explaining it. Rather than maintaining a single growing context window — appending each search result to an accumulating conversation history, as most DRTs do — UDR stores all intermediate information and text fragments as named variables in the code execution state (Section 2.2, "State modifications"):

"Rather than maintaining a single growing context window, UDR stores all intermediate information and text fragments as named variables in the code execution state."

The implications of this choice are multi-layered:

First, it enables operation within small context windows. Because information is stored in variables rather than in the LM's context, the system never needs to pass the entire accumulated research history to the LM at once. Instead, the strategy code selectively passes only the relevant fragments to each LM call. The paper reports that "a context length of 8k tokens was sufficient to carry out full research workflows, regardless of their complexity" (Section 2.2). This is a striking claim: it means the complexity of the research workflow (number of searches, volume of accumulated information) does not increase the context length required for any individual LM call.

Second, it makes the state fully auditable. All intermediate information is stored in named Python variables, which can be inspected, logged, or debugged. This contrasts sharply with LM-agent-based approaches, where the state is the entire conversation history — a monolithic, semi-structured blob that is difficult to inspect programmatically.

Third, it prevents context-induced degradation. Long-context language model performance has been shown to degrade with increasing context length, particularly for information in the "middle" of the context. By keeping each LM call's context short and focused, UDR avoids this degradation entirely.

Fourth, it enables accurate cross-referencing. Because state persists in variables outside the LM context, "the system can accurately refer to and reuse information gathered in earlier steps, even if that information was processed well before the current point in execution" (Section 2.2). This is non-trivial: in a pure context-window approach, earlier information might be "forgotten" if the context window is truncated or if the LM's attention mechanism fails to retrieve it. In UDR, the strategy code explicitly passes the relevant earlier information to the LM when needed.

Tool use: synchronous function calls. All tools — primarily the search function — are accessed via synchronous function calls within the generated code (Section 2.2, "Tool use"):

"All tools are accessed via synchronous function calls, ensuring transparent and deterministic behavior."

The synchronous design means that tool calls are blocking: the strategy code calls search(phrase), waits for results, then proceeds to the next instruction. This has both advantages and disadvantages. The advantage is determinism and debuggability: the execution trace is a simple linear sequence, and every state transition is explicitly triggered by a line of code. The disadvantage is latency: the system cannot parallelize independent searches. The paper acknowledges this, noting that "the architecture also allows for a future upgrade to asynchronous tool use for improved performance" (Section 2.2).

LM reasoning: callable utility, not orchestrator. Language model reasoning is treated as a callable utility rather than a controlling entity (Section 2.2, "LM reasoning"):

"Language model reasoning is treated as a callable utility rather than a controlling entity. UDR uses the language model for localized reasoning tasks such as summarization, ranking, or extraction, in line with specific steps in the user-defined research strategy."

This is the architectural inversion that distinguishes UDR from existing DRTs. In a typical DRT, the LM decides what to search, evaluates the results, decides whether to search more, and decides when to start writing the report. In UDR, the generated code makes all these decisions. The LM is invoked only when the code explicitly calls it, and only for the specific reasoning subtask requested (e.g., "given this research prompt, generate 3 search phrases," or "given this context, produce a markdown report").

The practical consequence is that the strategy is model-agnostic. Any LM capable of performing the localized reasoning tasks (summarization, extraction, query generation) can be substituted. The paper demonstrates this concretely by running all example outputs using Llama 3.3 70B (Appendix B), but the architecture makes no assumption about which model is used, and the strategy code would execute identically regardless of the underlying LM (assuming it produces parseable output).

The paper does not specify whether the LM used for code generation (Phase 1) is the same as the LM used for reasoning during execution (Phase 2). The examples in Appendix B use Llama 3.3 70B for both, and the paper states that UDR "wraps around any language model" without additional fine-tuning, implying that a single model can serve both roles.

Notifications: structured real-time progress updates. Throughout strategy execution, the user is kept informed via structured notifications. These are explicitly defined by the strategy author and emitted using yield statements in the generated code (Section 2.2, "Notifications"):

"Throughout the execution of a research strategy, the user is kept informed via structured notifications, which are explicitly defined by the strategy author. These progress updates are emitted using yield statements in the generated code, allowing the user interface to display real-time updates without revealing raw intermediate outputs or internal state unless explicitly requested."

Each yielded notification is a dictionary object containing "a loosely pre-specified schema with fields such as type, timestamp, and description" (Section 2.3). The example strategies in Appendix A show a consistent notification vocabulary: types like "prompt_received", "prompt_analysis_started", "search_started", "search_result_processing_completed", "research_completed", "report_building", and "report_done" recur across strategies. The final notification is marked with the type "final_report" and contains the completed research report as a payload field, signaling to the user interface that execution is complete.

The notification mechanism gives the strategy author complete control over what the user sees and when. A strategy could, for example, emit detailed per-search notifications for transparency, or emit only coarse phase-level updates to avoid overwhelming the user, or selectively surface intermediate results for user review. This explicitly replaces "the implicit, often opaque progress tracking found in typical DRTs with a deterministic, user-auditable stream of events" (Section 2.3).

Design Decisions: Why This Architecture?

The paper is explicit about the rationale behind UDR's architecture, describing it as the result of iterative development where earlier approaches were tried and discarded. Several design decisions are worth analyzing in detail:

Why code generation over prompt-based orchestration? The most fundamental choice is that UDR generates and executes code rather than embedding the strategy in an LM prompt and letting the LM execute it through reasoning and tool-calling. The paper's reliability analysis (Section 2.2, "Reliability") provides the justification: the prompt-based approach "proved fragile and error-prone." The specific failure modes included skipping strategy steps, misapplying search instructions, invoking the language model out of sequence, and introducing spurious checks and constraints not present in the original strategy.

The code-generation approach addresses these failure modes through several mechanisms. First, code is deterministic in a way that LM reasoning is not: a for loop will execute exactly the specified number of times, an if condition will branch exactly as written, and a sequence of statements will execute in order. Second, the comment-code discipline makes the mapping from strategy to implementation explicit and auditable — one can read the generated code and verify that each strategy step has a corresponding code block. Third, because the code is a complete artifact rather than a stream of reasoning tokens, it can be validated before execution.

Why generator functions over other execution patterns? The choice to make the generated function a Python generator (using yield) rather than, say, a function that returns the report directly or a class with methods for each phase, serves multiple purposes. Generators naturally model the streaming nature of research progress: as the function executes, it periodically yields status updates without terminating, and the caller (the user interface) can consume these updates as they arrive. Generators also support the stop-and-resume pattern that the user interface requires: if the user clicks the "stop research" button, the caller can simply stop iterating the generator, and the function's state is cleaned up. Finally, generators are a standard Python pattern that any LM trained on sufficient Python code will be familiar with, making them a reliable target for code generation.

Why synchronous tool calls over asynchronous? The paper acknowledges this as a simplification that could be upgraded. Synchronous calls make the execution model simple: one operation happens at a time, in a known order, with no concurrency bugs. This is appropriate for a research prototype where reliability is prioritized over performance. The paper explicitly flags the asynchronous upgrade path, suggesting the synchronous design is a deliberate scoping decision rather than a permanent architectural commitment.

Why the comment-code discipline specifically? The paper reports that this discipline "all but eradicate[d]" the shortcut-taking behavior observed in free-form code generation. The mechanism is likely twofold. First, requiring the model to write a comment before each code block forces it to explicitly identify which strategy step it is implementing, acting as a form of self-verification. Second, the comments make the generated code more structured and predictable, reducing the degrees of freedom available for the model to invent behaviors not specified in the strategy.

Security Design: Sandboxed Execution

Because UDR generates and executes user-defined code — and because that code is generated by an LM that could potentially be influenced by prompt injection in the research strategy or prompt — the paper explicitly addresses security as a design requirement rather than an afterthought (Section 2.2, "Security"):

"Because UDR generates and executes user-defined code, it is essential to account for the risks associated with prompt injection and code-based exploits."

The security mechanism is sandboxed execution: each generated strategy function is executed within an isolated environment that prevents access to the host system. This isolation is "enforced by default" and "designed to eliminate the possibility of side effects beyond the execution context" (Section 2.2). The paper cites Piston, an open-source code execution engine, as an example of a ready-to-use solution for such execution environments.

The paper emphasizes that isolation is "a strict requirement for any deployment of UDR beyond that to a fully trusted audience" (Section 2.2). This is a measured statement: for internal use by developers who understand the system, less stringent isolation might be acceptable, but for any production or multi-tenant deployment, sandboxing is non-negotiable. The threat model includes both malicious user-provided strategies (intentionally exploiting the execution environment) and prompt injection attacks (where the LM generates malicious code in response to a crafted strategy description).

The paper does not detail the specific sandboxing mechanism beyond mentioning Piston, but the requirement is clear: the execution environment must prevent file system access beyond a designated workspace, network access beyond the approved tool interfaces, and any other system calls that could affect the host.

Efficiency: Separating Orchestration from Reasoning

A central efficiency argument in UDR's design is the separation of control logic (which runs on CPU via generated Python code) from language model reasoning (which runs on GPU via LM inference calls). The paper states this explicitly (Section 2.2, "Efficiency"):

"UDR achieves high computational efficiency by separating control logic from language model reasoning. The orchestration of the deep research process is handled entirely by generated code, which executes on the CPU without requiring orders-of-magnitude more expensive language model inference."

The cost asymmetry is substantial. Running a Python for loop, string comparison, or variable assignment is effectively free compared to an LM inference call. In an LM-orchestrator architecture, every decision — what to search next, whether to continue searching, how to structure the report — requires an LM call. In UDR, only the explicitly specified LM subtasks (summarization, extraction, query generation) consume GPU resources. The orchestration logic — looping over search phrases, appending results to context variables, deciding when to proceed to the next phase — runs on CPU at negligible cost.

A second efficiency dimension is context efficiency. Because each LM call operates on "compact, well-scoped textual fragments" rather than the entire accumulated research history, the per-call token count is bounded regardless of overall research complexity. This reduces both the GPU cost per call (fewer tokens to process) and the latency per call (shorter sequences to generate from).

The paper summarizes this as "dual-level efficiency — delegating orchestration to CPU-executable logic and limiting LLM use to focused, context-efficient invocations — not only reduces GPU usage but also minimizes the overall latency and cost of executing deep research tasks" (Section 2.2). The paper does not provide quantitative benchmarks comparing UDR's GPU utilization to that of LM-orchestrator DRTs, so the efficiency claim remains qualitative.

The Three Example Strategies: Instantiating the Architecture

The paper provides three complete research strategies in Appendix A as concrete demonstrations of the range of behaviors UDR can express. These are not merely illustrative; they represent distinct points in the space of possible research methodologies, and analyzing them reveals the expressive capabilities and limitations of the strategy language.

Minimal Research Strategy (Appendix A.1). This is the simplest strategy, approximately 10 steps. Its logic is: (1) notify the user that the prompt was received; (2) ask the LM to generate 3 search phrases from the research prompt; (3) for each phrase, perform a search and append results to a CONTEXT variable; (4) ask the LM to produce a markdown research report from CONTEXT addressing the prompt's instructions; (5) yield the report. The strategy has no iteration, no cross-validation, no quality checks on search results, and no refinement of search phrases based on intermediate findings. It represents the lower bound of strategy complexity: a single pass of search-then-synthesize.

Expansive Research Strategy (Appendix A.2). This strategy adds a layer of indirection: (1) the LM first generates 2 "topics" from the prompt (broader than search phrases); (2) for each topic, the LM generates up to 2 search phrases specific to that topic; (3) all searches are performed and results accumulated into a single context; (4) the report is generated from the accumulated context. The key difference from the minimal strategy is the topic decomposition: rather than generating search phrases directly from the prompt, the strategy first identifies distinct sub-topics and then generates search phrases for each. This is a breadth-oriented strategy — it aims to cover multiple angles of a topic by explicitly decomposing it before searching.

Intensive Research Strategy (Appendix A.3). This is the most sophisticated of the three, implementing an iterative search refinement loop. Its logic: (1) generate 2 initial search phrases from the prompt; (2) perform 2 iterations of the following: (a) search all current phrases and accumulate results into a "subcontext"; (b) ask the LM to generate 2 new phrases based on the subcontext and the original prompt, replacing the old phrases; (3) accumulate results from all iterations into a "supercontext"; (4) generate the report from the supercontext. This strategy models the kind of iterative research that tools like Gemini and Perplexity Deep Research perform: search results inform subsequent searches, and the search direction evolves as more information is gathered. The separation of "subcontext" (current iteration's findings) and "supercontext" (all findings across all iterations) is a deliberate design that allows the strategy to use recent findings for query refinement while preserving the full research history for report generation.

What these strategies demonstrate about expressiveness. Across the three strategies, several dimensions of variation are visible:

Search breadth vs. depth: The expansive strategy prioritizes breadth (more topics, more phrases), while the intensive strategy prioritizes depth (iterative refinement of fewer phrases).
Decomposition strategy: The minimal strategy has no decomposition (phrases directly from the prompt), the expansive strategy has static decomposition (topics identified once, then searched), and the intensive strategy has dynamic decomposition (phrases evolve based on search results).
Notification granularity: All three strategies emit similar notification types, but the number and content of notifications vary with the strategy's complexity. The intensive strategy yields more notifications per search cycle.
LM invocation patterns: The minimal strategy invokes the LM twice (once for phrase generation, once for report generation); the expansive strategy invokes it three times per topic plus once for the report; the intensive strategy invokes it once for initial phrases, twice per iteration for phrase refinement, and once for the report.
State architecture: The minimal and expansive strategies use a single CONTEXT variable; the intensive strategy uses a dual-context architecture (subcontext + supercontext) to separate current-iteration findings from the full history.

The strategies are not claimed to be optimal or even particularly sophisticated. The paper's point is that UDR can express all of them — and by extension, any strategy that a user can articulate in natural-language step-by-step instructions — without any modification to the system itself. The strategy is data (text input), not code (system architecture).

Summary of Design Choices and Their Justifications

Single-pass code generation over prompt-based orchestration or fragmented code generation: ensures coherence across steps, minimizes synchronization overhead, avoids cascading failures, and produces auditable code.
Comment-code discipline during generation: forces the LM to map each strategy step to a specific code block, dramatically reducing shortcut-taking and hallucinated constraints.
Generator functions with yield notifications over other execution patterns: naturally models streaming research progress, supports stop-and-resume, and is a familiar Python pattern for reliable code generation.
Variable-based state management over growing context windows: enables small context windows (8k tokens regardless of workflow complexity), makes state auditable, prevents context-induced LM degradation, and enables accurate cross-referencing of earlier information.
Synchronous tool calls over asynchronous: prioritizes determinism and debuggability for the research prototype, with explicit acknowledgment of the asynchronous upgrade path.
LM as callable utility rather than orchestrator: decouples strategy from model choice (resolving P3), makes behavior deterministic and auditable, and reduces GPU utilization by confining LM calls to specified subtasks.
Sandboxed execution as a hard requirement: addresses the prompt injection and code exploit risks inherent in executing LM-generated code.
Strategy as explicit user input rather than system parameter: resolves P1 and P2 by making the research methodology fully user-controllable without requiring programming expertise beyond the ability to write step-by-step instructions.

4. Key Insights and Innovations

Innovation 1: Inverting the Agentic Architecture — LM as Utility, Not Orchestrator

The dominant paradigm in deep research tools — and agentic systems more broadly — is that the language model serves as the orchestrator. In tools like Gemini Deep Research, Perplexity Deep Research, OpenAI Deep Research, and Grok 3 DeepSearch, the LM decides what to search, evaluates whether results are sufficient, determines when to stop searching, and controls the transition to report generation. Even enterprise DRTs like NVIDIA AI-Q and SambaNova Deep Research, despite their more structured pipelines, embed the orchestration logic either in the system architecture or implicitly in the LM's reasoning. This makes the research strategy and the model inextricable: change the model, and the research behavior changes, even if the nominal strategy is the same.

UDR's fundamental conceptual move is to invert this relationship. The language model is demoted from orchestrator to callable utility — it performs localized reasoning tasks (summarization, extraction, ranking, query generation) only when and how the strategy code explicitly instructs it to. The orchestration logic — the sequence of operations, the decision of what to search next, the accumulation and cross-referencing of information, the construction of the report — resides entirely in generated Python code executing on CPU.

This is not a minor architectural tweak; it is a fundamental shift in the design space of agentic systems. The prevailing research direction in agentic AI has been toward increasing model autonomy — giving LMs more control over planning, tool use, and decision-making (see Reference 1, the authors' own prior work arguing for small-model agentic AI). UDR pushes in the opposite direction, asking what becomes possible when the model is stripped of orchestration responsibility and reduced to a reasoning utility called by deterministic code. The result is an architecture where the research strategy and the reasoning engine are cleanly decoupled — the former expressed as user-authored natural language compiled to code, the latter as any model capable of the specific reasoning subtasks required.

Why this matters beyond UDR. The LM-as-orchestrator paradigm has a structural problem that the paper's approach circumvents: opacity. When an LM orchestrates research, the user cannot fully audit why certain searches were performed, why some sources were prioritized over others, or why the research stopped when it did — because those decisions emerge from the LM's internal reasoning, which is not fully interpretable. UDR's code-generation approach makes the orchestration logic explicit and auditable: the generated Python code, with its comment-code discipline mapping each block to a strategy step, is a complete and inspectable specification of the research process. This advances a broader argument about the value of deterministic, auditable control flow in agentic systems — a counterpoint to the trend of making agents more autonomous and less predictable.

The efficiency argument reinforces this conceptual position. By moving orchestration from GPU (LM inference) to CPU (Python execution), UDR reduces not only cost but also the coupling between research complexity and model capability. Research workflows that require many decision points or complex branching do not become more expensive or less reliable in UDR, because those decision points are executed in code rather than through additional LM calls. The paper's claim that 8k tokens sufficed for all workflows regardless of complexity is the empirical evidence for this decoupling.

The prior work comparison is stark: every existing DRT surveyed in Section 1 — consumer and enterprise alike — embeds orchestration in the LM or the fixed system architecture. UDR is the first system to treat research strategy as a compilable specification that generates code, and LM reasoning as a stateless utility that the code invokes. This is less a refinement of existing DRT design than a proposal for a different class of agentic system altogether.

Innovation 2: Strategy as First-Class User Input — Closing the Expressiveness Gap

Existing DRTs expose exactly one user-facing control over research methodology: the research prompt. Everything else — how searches are conducted, how sources are evaluated, when iteration stops, how information is validated — is hard-coded into the system. This creates the functionality gap the paper identifies between consumer DRTs (designed for wide-open web search with fixed iterative strategies) and enterprise DRTs (designed for constrained document databases with specialized pipelines). A financial analyst who needs to cross-validate claims against SEC filings before including them, or a legal researcher who needs to follow a specific hierarchy of precedent databases, cannot express these requirements to any existing DRT — because the DRT's research strategy is not an input; it's the product's architecture.

UDR's innovation is making the research strategy a first-class user input, on equal footing with the research prompt. The user writes a natural-language, step-by-step description of how they want research conducted — what to search, in what order, with what validation checks, with what notification granularity — and the system executes it. The three example strategies in Appendix A demonstrate the range this enables: a minimal single-pass strategy, an expansive topic-decomposition strategy, and an intensive iterative-refinement strategy — all executed by the same system without any code changes to UDR itself.

This is a reframing of the deep research problem. Previous DRTs answer the question "given a topic, what is the best research strategy?" — and each DRT answers it differently based on its designers' choices. UDR removes that question from the system's scope and delegates it to the user: "given a topic and a research methodology you specify, execute that methodology faithfully." This shifts the role of the DRT from researcher (making methodological choices) to research assistant (executing a user-specified methodology). The conceptual consequence is that strategy becomes data rather than architecture — users can share strategies, iterate on them, combine elements from different strategies, and build libraries of strategies for different domains, all without touching the DRT's codebase.

Why this resolves P1 and P2 in a way prior approaches cannot. The paper's three enumerated problems — restricted user control (P1), inability to create specialized strategies (P2), and model-tool bundling (P3) — cannot be resolved by making existing DRTs' fixed strategies "better" or more configurable through parameters. A configurable fixed strategy (e.g., "choose your search breadth on a slider") is still a single strategy with tunable knobs, not a general mechanism for expressing arbitrary research methodologies. UDR's approach of compiling natural-language strategies into code is the first demonstrated mechanism for achieving full user control over research methodology without requiring users to write actual code — they write natural-language instructions, and the system handles the conversion.

The evidence that this works is in Appendix B: the same UDR system, with zero architectural changes, executes a minimal strategy for four entirely different research prompts (ornithological humor, historical events, financial data, and biographical research) and produces structured reports with citations. The reports are not claimed to be superior to those from specialized DRTs — the paper is a research prototype — but they demonstrate that the expressiveness goal (arbitrary user-specified strategies) has been achieved, even if the strategy design process itself remains tedious (as the paper acknowledges in Section 5).

Innovation 3: Variable-Based State as an Alternative to Context-Window Accumulation

The predominant state management approach in LLM-based research tools — and in LLM applications generally — is context-window accumulation: as the system gathers information, it appends it to the conversation history or prompt context, and the LM's attention mechanism is responsible for retrieving and synthesizing relevant earlier information. This approach is simple but has well-documented failure modes: long-context degradation (performance drops for information in the middle of long contexts), context-window limits (even million-token windows have finite capacity), and the fundamental opacity of what the LM "attends to" versus what it ignores.

UDR introduces a fundamentally different state management paradigm: all intermediate information is stored as named Python variables in the code execution state, and the LM is called with only the specific, scoped textual fragments relevant to its current task. The strategy code is responsible for retrieving the right variables and passing them to each LM call, rather than relying on the LM's attention mechanism to find what matters in a monolithic context.

This is a conceptual innovation with architectural consequences that extend beyond UDR. The dominant assumption in LLM application design is that more context is better — hence the race to million-token context windows. UDR demonstrates that for a complex, multi-step workflow like deep research, a different assumption holds: selective, scoped context is more reliable than maximal context, provided the orchestration layer (the code) maintains the full state and can retrieve the right fragments for each subtask. The paper's empirical result — 8k tokens sufficed for full workflows "regardless of their complexity" — is the evidence supporting this claim.

The significance goes beyond efficiency. Variable-based state management makes the research process fully auditable in a way that context-window accumulation cannot. Every intermediate result — a search outcome, an extracted fact, a partially assembled report section — lives in a named variable that can be inspected, logged, or checkpointed. This is important for the enterprise use cases the paper targets (P2), where research outputs in regulated industries (finance, legal, healthcare) must be traceable: "this claim in the report came from this search result, processed by this LM call, at this point in the strategy." Context-window-based systems can attempt this traceability through prompt logging, but the relationship between "text in the context at step N" and "claim in the output at step N+1" is mediated by the LM's unobservable attention patterns. UDR's explicit variable passing makes the information flow deterministic and inspectable.

This innovation also enables the LM-as-utility paradigm (Innovation 1) to work reliably. If the LM were expected to maintain state implicitly through context accumulation, it would necessarily become an orchestrator — deciding what to "remember" and what to use. By externalizing state into code variables, UDR removes that burden from the LM, making it feasible to treat the LM as a truly stateless utility called with explicit inputs and producing explicit outputs.

The prior work contrast is clear: most agentic systems (including the DRTs surveyed in Section 1) use context-window accumulation as their state mechanism, either implicitly (through growing conversation history) or explicitly (through retrieval-augmented generation that appends retrieved passages to the prompt). UDR's variable-based approach is not a refinement of this pattern — it is a fundamentally different state architecture that separates storage (code variables) from reasoning (LM calls with scoped context), enabling capabilities (auditability, context-size independence, accurate cross-referencing) that the accumulation approach cannot provide.

Innovation 4: The Comment-Code Discipline as a Reliability Mechanism for Strategy-to-Code Translation

The paper's most practically instructive innovation may be its solution to a specific failure mode in natural-language-to-code generation: when given complete freedom, language models take shortcuts, skip steps, and impose constraints the user never specified. The paper reports this occurred "regardless of the choice of the language model" (Section 2.2), suggesting it is a general property of how LMs approach open-ended code generation from procedural descriptions — they optimize for "plausible code that accomplishes the general goal" rather than "faithful code that implements each specified step."

UDR's countermeasure is the comment-code discipline: the LM is prompted to generate code step-by-step, explicitly prepending each code segment with a comment identifying which strategy step it implements. This is not a prompting trick; it is a structural constraint on the generation process that changes what the model optimizes for. Instead of generating code holistically from the entire strategy description, the model must produce a comment-then-code pair for each instruction — effectively performing chain-of-thought reasoning where each reasoning step (the comment) must map to a specific strategy step before the corresponding code can be written.

The failure modes this addresses are specific and consequential for agentic systems: strategy steps being silently skipped, search instructions being applied incorrectly (e.g., wrong query format, wrong number of results), LM calls being invoked out of the specified sequence, and spurious validation checks being introduced that the user never requested. The paper characterizes the comment-code discipline as having "all but eradicating" these behaviors (Section 2.2).

Why this is more than an implementation detail. The shortcut problem is not unique to UDR — it is a general challenge for any system that translates natural-language procedural descriptions into executable behavior, whether through code generation, prompt-based orchestration, or task decomposition. The comment-code discipline is a generalizable reliability technique that can be applied to any LM-driven strategy-to-code translation, not just deep research. Its mechanism — forcing the model to explicitly link each generated code block to its source instruction — effectively adds a self-verification step to the generation process, reducing the degrees of freedom for hallucinated behavior.

The evidence comes from the paper's comparative development history (Section 2.2, "Reliability"): two prior approaches — prompt-based orchestration and fragmented code generation — were both found to be fragile, while the single-pass end-to-end generation with comment-code discipline proved reliable. The paper frames this as the key architectural decision that made the overall UDR approach feasible, moving from "fragile and error-prone" prototypes to a system that "rarely exhibits the failure modes encountered in our earlier prototypes."

This innovation is incremental in the sense that it is a generation technique rather than a new class of system, but it is practically fundamental: without it, the core UDR concept — translating arbitrary user-written strategies into faithful executable code — would not work reliably. It is the linchpin that makes Innovations 1–3 realizable in practice.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper does not evaluate against a standard benchmark dataset with ground-truth labels. Instead, UDR is demonstrated on four hand-crafted research prompts provided in Appendix B: (B.1) the airspeed velocity of an unladen swallow, (B.2) significant events on May 1st, 2025, (B.3) US stock movements on April 24th, 2025, and (B.4) a biographical report on Ulysses S. Grant. These prompts span distinct domains — popular culture, historical events, financial data, and biographical research — chosen to demonstrate the system's flexibility across different research types. The paper does not specify a separate training/validation/test split, as no model training or hyperparameter tuning is conducted.
Base model(s). All example outputs in Appendix B were generated using Llama 3.3 70B. The paper explicitly states that UDR "wraps around any language model" without additional fine-tuning (Section 1), and the choice of Llama 3.3 70B serves as a representative demonstration rather than a claim about optimal model selection. The model was used both for the strategy compilation phase (Phase 1, converting natural-language strategies to code) and for the localized reasoning tasks during strategy execution (Phase 2, generating search phrases and producing reports). The paper does not report experiments with alternative models, so the claim of model-agnosticity is asserted architecturally rather than demonstrated empirically through comparative results.
Metrics. The paper reports no quantitative performance metrics in the traditional sense. There are no accuracy scores, F1 measures, ROUGE scores, citation precision/recall rates, or human evaluation results. The demonstration relies entirely on qualitative assessment of the generated reports: the full Markdown outputs for each of the four research prompts are reproduced verbatim in Appendix B, allowing the reader to inspect report structure, relevance, factual accuracy, and citation quality directly. The paper does note one specific failure in the stock movements report (B.3.2), flagging that "Any hallucinations are not due to incorrect search results but to the incorrect comprehension of the search results by the language model." This is the only explicit quality assessment in the paper.
Baselines. The paper does not compare UDR against any baseline system. There are no head-to-head comparisons with existing DRTs (Gemini Deep Research, Perplexity, OpenAI Deep Research, Grok 3 DeepSearch, NVIDIA AI-Q, SambaNova Deep Research, or ERP AI Deep Research). There are no comparisons of UDR's reports against reports generated by those tools on the same prompts. There are no ablations comparing UDR's code-generation approach against alternative architectures (e.g., prompt-based LM orchestration, hard-coded strategy pipelines, or retrieval-augmented generation without explicit strategy control). The experimental section is a pure existence demonstration: the paper shows that UDR can produce plausible research reports using different strategies on different prompts, but does not establish whether these reports are better, worse, or comparable to those from existing systems.
Generation budget / compute accounting. The paper provides no quantitative compute accounting for the experimental demonstrations. There are no measurements of total GPU time, number of LM inference calls, tokens processed, wall-clock time, or monetary cost for any of the example runs. The efficiency claims in Section 2.2 ("dual-level efficiency," 8k context sufficiency, CPU-orchestration vs. GPU-reasoning separation) are architectural arguments rather than experimentally measured results. The paper does not report the actual token counts or inference latencies for the example reports in Appendix B.
Cross-validation / statistical protocol. There is no statistical protocol of any kind. No cross-validation, no confidence intervals, no significance testing, no multiple runs with different random seeds. Each research prompt in Appendix B was processed exactly once with a single strategy (the minimal strategy from Appendix A.1). The paper does not report variance across repeated runs, sensitivity to the specific LM inference, or robustness to variations in the research prompt phrasing.

Main Quantitative Results

The paper contains no quantitative experimental results in the conventional machine learning sense. There are no tables of numbers, no comparative bar charts, no scaling curves, no ablation measurements. The "experimental" content consists entirely of four qualitative demonstrations:

Demonstration of Strategy Flexibility

The paper demonstrates UDR's ability to execute different research strategies by running the minimal research strategy (Appendix A.1) on four distinct prompts (Appendix B). The key observable outcomes are:

Strategy fidelity: The generated code successfully implements the specified 10-step procedure across all four prompts, as evidenced by the consistent notification structure visible in the user interface screenshots (Figures 3 and 4) and the presence of all required structural elements (search phrase generation, sequential searching, report assembly from accumulated context) in the final outputs.
Cross-domain applicability: The same minimal strategy produces domain-appropriate outputs for popular culture (B.1 — Monty Python reference, swallow airspeed, cultural impact), historical events (B.2 — International Labour Day, Maharashtra Day, Gujarat Day), financial data (B.3 — S&P 500, DJIA, NASDAQ opening/closing prices with wider market context), and biographical research (B.4 — Ulysses S. Grant's military career, leadership style, political legacy). The strategy does not require domain-specific modification to function across these different topics.
Citation generation: All four reports include numbered reference links (e.g., [[0]], [[1]]) with corresponding URLs at the bottom of each report. B.1 cites two sources, B.2 cites five, B.3 cites five, and B.4 cites five. The paper does not evaluate whether these citations are accurate (do the cited sources actually contain the claimed information?) or comprehensive (are there important sources missing?).
Report structure: All four reports follow the structural requirements specified in their respective prompts. B.1 produces three sections (Origins and Popular Culture, Technical Analysis and Accuracy, Enduring Cultural Impact), B.2 produces sections on International Labour Day, Maharashtra/Gujarat Day, and Global Significance, B.3 produces sections on Opening Conditions, Closing Conditions, and Wider Perspective, and B.4 produces five sections (Introduction, Military Career, Leadership Style, Political Legacy, Conclusion). The strategy's report-generation step (Step 8) successfully extracts formatting requirements from the prompt and enforces them.

Example of Hallucination and Error

The stock movements report (B.3.2) contains the paper's only explicit acknowledgment of a quality issue:

"Any hallucinations are not due to incorrect search results but to the incorrect comprehension of the search results by the language model."

The report contains specific numerical claims (e.g., "S&P 500 opened at 5395.92," "closed at 5375.86, marking a 1.67% increase") attributed to sources. The paper flags that the LM's comprehension of search results (i.e., its interpretation and synthesis of retrieved text) can produce factual errors even when the underlying search results are correct. This is a standard LM hallucination pattern, not specific to UDR, but the paper's acknowledgment establishes that UDR inherits the factual reliability limitations of the underlying LM for reasoning tasks.

Strategy Complexity Range

The three strategies in Appendix A demonstrate a range of research methodologies executable by UDR:

Minimal strategy (A.1): linear, single-pass, 3 search phrases, 1 context variable, 2 LM calls (phrase generation + report generation), approximately 10 steps.
Expansive strategy (A.2): topic decomposition, topic-specific search phrase generation, single context accumulation, approximately 10 steps but with nested loops (2 topics × up to 2 phrases each = up to 4 searches plus phrase generation per topic).
Intensive strategy (A.3): iterative search refinement with dual-context architecture (subcontext + supercontext), 2 refinement iterations, phrase replacement between iterations, 3 LM calls per iteration (initial phrase generation + 2 refinement calls) plus final report generation.

The paper does not report the actual execution traces, token counts, or wall-clock times for any of these strategies on any prompt, so the complexity comparison remains qualitative and structural.

Reliability of Code Generation

The paper reports in Section 2.2 ("Reliability") that the final single-pass comment-code approach "rarely exhibits the failure modes encountered in our earlier prototypes, such as skipping strategy steps, misapplying search instructions, invoking the language model out of sequence, or introducing spurious checks and constraints not present in the original strategy." This is an empirical claim based on the authors' development experience, but no quantitative data is provided: no counts of strategy steps executed correctly vs. incorrectly, no comparison of generated code accuracy between the comment-code approach and the abandoned alternatives, and no measurement of failure rates across multiple strategy generations with different random seeds or model configurations.

Ablation Studies and Robustness Checks

The paper contains no formal ablation studies and no systematic robustness checks. There are no controlled experiments where individual design decisions are varied while holding others constant to measure their contribution. The following observations are drawn from the paper's qualitative descriptions of its development process rather than from structured experimental comparisons:

Comment-code discipline vs. free-form code generation: The paper reports (Section 2.2, "Phase 1 – Strategy processing") that free-form code generation "often resulted in the model taking shortcuts, skipping strategy steps, and taking liberties at imposing constraints where none were stipulated by the user," and that the comment-code discipline "all but eradicate[d]" this behavior. This is a comparative claim without quantitative evidence. No side-by-side comparison of generated code accuracy, strategy step completion rates, or output quality between the two approaches is reported.

Single-pass end-to-end generation vs. prompt-based orchestration: The paper states in Section 2.2 ("Reliability") that embedding the strategy directly within a prompt to a reasoning-oriented LM "proved fragile and error-prone," while the single-pass code generation approach "yielded significantly more reliable outcomes." Again, no quantitative comparison is provided — no reliability metrics, no counts of failure modes, no measurements of output quality differences between the two approaches.

Single-pass end-to-end generation vs. fragmented step-by-step code generation: The paper reports that decomposing the strategy into individual steps and generating isolated code fragments for each introduced "synchronization overhead" and risked "cascade of failures." The single-pass approach is claimed to "ensure coherence across steps" and "minimize synchronization overhead." No empirical evidence is presented to quantify these differences.

Model choice: The paper does not ablate across different language models to demonstrate UDR's claimed model-agnosticity. All examples use Llama 3.3 70B. There are no experiments showing that UDR functions equivalently (or at all) with other model families, smaller models, or models from different providers. The claim that UDR "wraps around any language model" (Section 1) is an architectural assertion rather than an experimentally verified property, and the paper's own limitation about "reliance on language model code generation" (Section 4) acknowledges that faithfulness "depends on the quality of code generated by the underlying language model" — implying that not all models may generate reliable code.

Strategy complexity sensitivity: The paper does not test whether strategy fidelity degrades with increasing strategy length or complexity. The three example strategies range from approximately 10 steps to a more complex iterative strategy, but all were executed with the same model, and no measurements of generated code correctness or strategy-step compliance are reported across strategies of different lengths.

Context window size: The paper claims that "a context length of 8k tokens was sufficient to carry out full research workflows, regardless of their complexity" (Section 2.2). No experimental evidence is provided for this claim — no measurements of actual token counts across different strategy-prompt combinations, no tests showing that 4k or 16k contexts produce different results, and no demonstration of what happens as research complexity (number of searches, volume of accumulated results) increases.

Search result quality: The paper does not investigate how UDR's output quality varies with the quality or quantity of search results. There is no experiment varying the search engine, the number of results per search, or the relevance threshold of returned results. The hallucination noted in B.3.2 is attributed to LM comprehension rather than search result quality, but no controlled experiment establishes this attribution.

Negative result — ReST $^{EM}$ revision training: This is the one area from the broader paper analysis where a negative result is reported (the earlier paper provided extensive experimental analysis of this failure mode). No analogous negative result is reported for UDR itself, beyond the brief mention that earlier prototypes were fragile.

Notification granularity: The paper does not experiment with different notification schemas or measure user comprehension or satisfaction with different notification densities. The notification structure is demonstrated as functional but not experimentally evaluated.

Critical Assessment

The paper makes no claims that require quantitative experimental validation in the traditional sense. Its primary claims are architectural and capability-oriented: that UDR can accept user-written natural-language strategies, convert them to executable code, execute them to perform research, and produce structured reports — all without additional training or fine-tuning, and with model-agnostic design. The demonstrations in Appendix B serve as existence proofs that this pipeline can produce plausible outputs on a small set of hand-chosen prompts.

However, the paper's framing implies stronger claims that remain untested:

Claim: UDR "wraps around any language model." The paper demonstrates this with exactly one model (Llama 3.3 70B). The claim requires demonstrating that (a) different model families (GPT, Claude, Gemini, open-source models of varying sizes) can reliably generate valid strategy code, and (b) different models can perform the reasoning subtasks (summarization, query generation, report writing) with acceptable quality. Neither is tested. The strategy code generation step is particularly sensitive — if a model produces syntactically invalid Python, or a function with the wrong signature, or a function that doesn't respect the yield-based notification protocol, UDR would fail at the validation step. The paper does not report success rates for code generation across models.

Claim: "8k tokens was sufficient to carry out full research workflows, regardless of their complexity." This is a strong architectural claim presented without evidence. The paper provides no token counts for any of the example runs, no demonstration that workflows with 100+ search results still fit in 8k per-call contexts, and no evidence that the "regardless of complexity" qualifier holds. The intensive strategy (A.3), with its dual-context architecture, was not demonstrated on any prompt in Appendix B — all four examples use the minimal strategy — so the claim that complex workflows fit in 8k tokens is entirely unsubstantiated by the reported experiments.

Claim about efficiency: "UDR achieves high computational efficiency by separating control logic from language model reasoning." This is presented as an achieved property, but no efficiency measurements are reported. There is no comparison of UDR's total GPU utilization, wall-clock time, or cost against any existing DRT on the same research task. The architectural argument is plausible — executing orchestration logic on CPU rather than through LM inference calls should reduce GPU usage — but the magnitude of the saving and its practical significance are unknown.

Claim about reliability: the comment-code discipline "all but eradicate[d]" shortcut-taking and strategy-skipping. This is based on the authors' qualitative development experience. Without reporting failure rates (e.g., "in 100 strategy-to-code generations, strategy steps were skipped in X% of cases without comment-code discipline and Y% with it"), the claim is anecdotal. The paper's frank acknowledgment of prior failures is credible, but the improvement is not quantified.

Missing experiments that would substantiate the claims:

Cross-model code generation success rate: Generate strategies with 3–5 different model families and report the percentage of generated functions that pass validation and execute without errors.
Cross-model output quality: For a fixed set of 3–5 research prompts and a fixed strategy, run UDR with different underlying LMs and conduct even a small-scale human evaluation comparing report quality.
Strategy complexity scaling: Measure code generation success rate and output quality as the number of strategy steps increases (e.g., 5, 10, 20, 50 steps) to establish whether there is a complexity ceiling.
Context token measurements: For the intensive strategy on a realistic prompt, report the actual token counts per LM call and demonstrate that even the largest calls remain below 8k tokens.
Comparative baselines: Run the same research prompts through one or more existing DRTs (Gemini, Perplexity, OpenAI) and compare report structure, citation quality, and factual accuracy, even qualitatively.
Factuality evaluation: For a prompt with ground-truth-checkable claims (e.g., the stock movements prompt has verifiable opening/closing prices), systematically check the factual accuracy of UDR's reports against the cited sources.

The fundamental limitation of the experimental section is that it demonstrates functionality (UDR can produce reports that look like deep research outputs) but not quality (these reports are accurate, comprehensive, or superior to alternatives) and not reliability (UDR will consistently produce correct strategy code across models and strategies). For a research prototype paper whose claimed contribution is architectural — making research strategies user-programmable and model-agnostic — this may be acceptable scoping. The paper presents UDR as opening a design space rather than as a benchmarked system ready for deployment. However, the claims about efficiency, reliability, and model-agnosticity go beyond pure architecture into empirical territory, and those claims lack experimental support.

The paper's own acknowledgment of limitations in Section 4 — particularly "reliance on language model code generation" where "occasional semantic drift or hallucinated logic may still occur" and "the system does not currently validate whether the specified steps produce a coherent or meaningful workflow" — implicitly recognizes that the reliability and quality questions are open. The experimental section does not close them.

6. Limitations and Trade-offs

Pre-Authored Strategies Are Tedious to Create and Refine

The assumption or constraint. UDR makes the research strategy a first-class user input, but this shifts a substantial design burden onto the user. The paper is transparent about the practical consequence:

"devising a research strategy sufficiently sophisticated to contend with the complexities of all user queries and topics has proven to be a rather tedious process for the end users of the applications – even those who appreciate the level of control this approach lends them" (Section 5)

The user must anticipate in advance what sequence of searches, validations, and synthesis steps will produce a good report for their topic — a non-trivial task that requires understanding of both the domain and effective research methodology.

The consequence. The expressiveness that UDR enables (P1, P2) is gated behind a usability barrier that the paper itself identifies as the primary friction point for adoption. A user who writes an inadequate strategy — missing important search terms, failing to cross-validate, stopping iteration too early — will receive an inadequate report, and UDR provides no mechanism to detect or correct strategic deficiencies. The paper's recommendation R1 — equipping a deployed system with a "library of research strategies for modification and customization rather than the requirement that the user bring their own strategy" (Section 5) — implicitly acknowledges that requiring users to author strategies from scratch is unrealistic for most audiences.

What evidence exists in the paper. Section 5 characterizes this as an explicit finding rather than a hypothetical concern. The three example strategies in Appendix A were authored by the paper's authors (presumably with deep familiarity with the system), yet the paper does not report how long each strategy took to write, how many iterations of refinement were needed, or whether different strategy authors would produce strategies of comparable quality. The recommendation to provide a pre-built strategy library rather than requiring from-scratch authorship is the paper's own conclusion about the severity of this limitation.

Mitigation status. The paper does not attempt to mitigate this limitation within UDR itself. Section 5 frames it as a deployment consideration (recommendation R1) rather than a technical problem to solve. Potential approaches — automatic strategy suggestion based on prompt analysis, strategy quality estimation before full execution, interactive strategy refinement where UDR suggests improvements — are not explored. The limitation is structural: separating strategy from execution is UDR's core contribution, and making strategy authoring easy is an orthogonal problem that the paper identifies but does not address.

No Evidence That Generated Reports Are Accurate or Competitive

The assumption or constraint. UDR is evaluated through four qualitative demonstrations (Appendix B) with no accuracy measurements, no human evaluation, and no comparison against any existing DRT. The paper acknowledges one specific failure — hallucinations in the stock report (B.3.2) attributed to "incorrect comprehension of the search results by the language model" — but provides no systematic analysis of how frequently such errors occur, how they vary across domains, or how they compare to error rates in existing tools.

The consequence. A practitioner evaluating UDR cannot answer the most basic deployment question: will the reports it produces be factually reliable enough for their use case? The paper demonstrates that UDR can produce reports that look like deep research outputs (structured, formatted, with citations), but provides no evidence that these reports are correct. The hallucination in B.3.2 is flagged as a single example, but the reader has no way to know how many other claims in B.3 — or in B.1, B.2, and B.4 — are inaccurate. For the enterprise use cases the paper targets (P2 — finance, legal, healthcare, government), factual reliability is non-negotiable, and this gap is the single largest barrier to practical deployment.

What evidence exists in the paper. Only the qualitative reports in Appendix B, with one explicit error flag. There is no factuality audit of any report against its cited sources, no measurement of citation accuracy (does [[0]] actually contain the claim it's attached to?), no comparison of UDR's report against a report produced by Gemini/Perplexity/OpenAI on the same prompt, and no human evaluation rubric or results. The paper does not report whether the Ulysses Grant report (B.4), which makes specific historical claims about battles, strategies, and Reconstruction policies, was fact-checked against the cited sources or against expert knowledge.

Mitigation status. The paper does not address report accuracy as a limitation. Section 4 (Limitations) discusses "reliance on language model code generation" and "trust in user-defined strategies" but does not mention the factual reliability of the generated reports themselves. This is a notable omission, as report accuracy is the ultimate measure of a deep research tool's usefulness. The paper's framing as a "research prototype" (Section 1) partially explains the scoping — architectural demonstrations do not typically require exhaustive accuracy benchmarks — but the failure to even qualitatively discuss report quality as an open question weakens the deployment-readiness narrative.

No Cross-Model Validation of the Model-Agnosticity Claim

The assumption or constraint. UDR's central architectural claim is that it "wraps around any language model" without additional fine-tuning (Section 1). This claim is substantiated by exactly one configuration: Llama 3.3 70B used for both code generation (Phase 1) and reasoning during execution (Phase 2). The paper does not test any other model family, any smaller model, or any model from a different provider.

The consequence. Model-agnosticity is asserted based on architecture (the LM is not the orchestrator, therefore any LM can fill the utility role) rather than demonstrated empirically. However, two distinct failure modes could violate this claim:

Code generation failure: Phase 1 requires the model to generate syntactically valid Python, respect a specific function signature (generator with yield), follow the comment-code discipline, and correctly implement the strategy's control flow. Smaller or less capable models may produce code that fails validation, omits steps, or contains silent logic errors. The paper reports that shortcut-taking occurred "regardless of the choice of the language model" during development (Section 2.2) — but this was observed on models the authors actually tested (likely Llama-family models), not proven across the full spectrum of available LMs.
Reasoning failure: Phase 2 requires the model to perform localized reasoning tasks (query generation, summarization, report synthesis) with bounded context. A model that performs adequately on general benchmarks may still struggle with the specific output formats UDR expects (e.g., exact newline-separated search phrases with no extraneous text), causing the strategy code's parsing logic to fail silently.

What evidence exists in the paper. None. The paper reports no experiments with any model other than Llama 3.3 70B. The claim that UDR "wraps around any language model" is an architectural assertion, not an empirically supported property. The paper's own limitation in Section 4 — "reliance on language model code generation" where "faithfulness ... depends on the quality of code generated by the underlying language model" — implicitly acknowledges that model choice matters, but this tension with the core model-agnosticity claim is not explored.

Mitigation status. The paper does not address this gap. The recommendation R3 in Section 5 — exploring how "a large set of user prompts could be automatically turned into deterministically controlled agents" — suggests the authors view broader model validation as future work. The paper would be strengthened by even a minimal cross-model experiment: running the same strategy on two different model families (e.g., Llama 3.3 70B vs. GPT-4o vs. Claude 3.5 Sonnet) and reporting whether code generation succeeds and whether resulting reports are qualitatively comparable.

Search Result Quality and Source Coverage Are Not Controlled or Evaluated

The assumption or constraint. UDR relies entirely on the quality of results returned by its search function — the system searches for user-specified (or LM-generated) phrases, accumulates results into context variables, and synthesizes reports from those results. The paper provides no information about what search engine is used, how many results are returned per query, what relevance or freshness thresholds are applied, or whether any source credibility filtering occurs. The search function is mentioned only as a tool described in a docstring to the code-generating model (Section 2.2, "Phase 1").

The consequence. The quality of UDR's research reports is fundamentally bounded by the quality and coverage of the search results. If the search engine returns outdated, irrelevant, or low-credibility sources, the report will reflect those deficiencies regardless of how well the strategy is designed. Furthermore, different strategies make different assumptions about search result quality: the minimal strategy performs 3 searches and builds the report entirely from those results, making it highly sensitive to search quality, while the intensive strategy's iterative refinement partially compensates by using earlier results to generate better subsequent queries. Without characterizing the search backend, it is impossible to disentangle strategy quality from search quality in the example outputs.

A specific concern: the example reports in Appendix B cite primarily popular web sources (Know Your Meme, SYFY, Wikipedia, PBS, investing websites) rather than the "scholarly sources" requested in prompts like B.4. The Grant report (B.4.2) cites Ranker, HISTORY.com, Wikipedia, PBS, and American Battlefield Trust — all legitimate popular sources, but none are scholarly (peer-reviewed academic works). This may reflect the search engine's coverage bias rather than the strategy's design, but the paper provides no way to distinguish these factors.

What evidence exists in the paper. The paper provides no characterization of the search backend: no API specification, no result count per query, no latency or coverage measurements, and no description of source filtering. The example reports cite 2–5 sources each, but it is unknown whether more relevant sources existed that the search engine failed to return, or whether the returned sources were the most credible available. The hallucination in B.3.2 is attributed to LM comprehension rather than search result quality, but this attribution is asserted rather than experimentally verified.

Mitigation status. The paper does not address search quality as a limitation. This is partly understandable for a system architecture paper — the search backend is a pluggable component, and the paper's contribution is the strategy execution framework, not the search engine. However, for a practitioner evaluating UDR's practical utility, the search backend is not a separable concern — it determines what information is available to the system, and without characterizing it, the demonstrated report quality cannot be assessed as an artifact of the UDR architecture versus an artifact of the search engine's performance. A minimal characterization (search engine used, results per query, basic latency numbers) would substantially improve the paper's practical grounding.

No Mid-Execution User Intervention or Dynamic Strategy Adaptation

The assumption or constraint. UDR's research strategy, once compiled to code and launched, executes to completion without user intervention except for a full stop (the "Stop research" button in Figure 3). The paper acknowledges this:

"the current implementation does not support mid-execution user intervention (beyond user stopping the workflow) or dynamic branching based on real-time user feedback. All decision logic must be encoded upfront in the research strategy" (Section 4)

All branching logic — which searches to perform next, when to stop iterating, how to detect that sufficient information has been gathered — must be specified in advance by the strategy author. The system cannot adapt its behavior based on intermediate findings in ways not anticipated by the strategy, nor can it ask the user for guidance (e.g., "I found conflicting information on this point — which source should I prioritize?").

The consequence. This limitation creates a fundamental tension in strategy design. Effective research is often exploratory — the researcher discovers what they don't know through the research process itself. Pre-specifying all decision logic means the strategy author must anticipate what kinds of intermediate findings will emerge, what dead ends might appear, and when the research should pivot. A strategy that is too rigid will produce shallow reports; a strategy that is too flexible (e.g., open-ended iteration) may run indefinitely or accumulate irrelevant information. The tradeoff between strategy complexity and robustness — already difficult when all logic is pre-specified — is exacerbated by the inability to incorporate real-time user judgment.

For the enterprise use cases the paper targets (P2), this is a significant limitation. Professional researchers in law, finance, and healthcare routinely make mid-research judgment calls: a search reveals an unexpected regulatory angle that merits deeper investigation, or a source's credibility is ambiguous and requires human evaluation. UDR's architecture assumes all such decisions can be codified in advance, which is unlikely to hold for genuinely novel or complex research tasks.

What evidence exists in the paper. Section 4 explicitly lists this as a limitation. The user interface (Figures 3 and 4) shows a "Stop research" button and a "Generate report" button that becomes active after stopping, confirming that the only user interaction during execution is early termination with partial results. The example strategies in Appendix A have no provisions for user consultation — they generate search phrases, search, accumulate results, and produce reports entirely autonomously.

Mitigation status. The paper acknowledges the limitation but does not address it technically. The notification mechanism (yield statements) provides a natural extension point: notifications could carry structured data (e.g., "conflicting sources found") and the system could pause execution until the user responds. The paper mentions this only as an absence ("does not support"), not as a design goal. The synchronous execution model and generator-based architecture would support this extension (a yield could wait for user input before resuming), but the paper does not explore it.

The Comment-Code Discipline's Reliability Is Not Quantified Across Strategy Complexity

The assumption or constraint. The paper's core reliability mechanism — the comment-code discipline that forces the LM to map each strategy step to a code block — is presented as having "all but eradicate[d]" the shortcut-taking, step-skipping, and spurious constraint-imposition observed in free-form code generation (Section 2.2, "Phase 1" and "Reliability"). This claim is based on the authors' qualitative development experience rather than measured data: no success rates, step-completion counts, or error categorization across multiple strategy generations are reported.

The consequence. A practitioner considering UDR for a production system needs to know: how often does the generated code actually implement the specified strategy faithfully, and how does this reliability scale with strategy length and complexity? The three example strategies range from ~10 steps (minimal) to a somewhat more complex iterative structure (intensive), but they are all relatively short. Real-world research strategies — especially for the specialized enterprise use cases UDR targets — could involve dozens of steps with nested conditionals, multiple validation passes, and complex state management. The paper provides no evidence that the comment-code discipline remains effective at those scales.

There is a specific failure mode the paper does not address: what happens when the LM generates syntactically valid but semantically incorrect code? The example strategies involve string parsing (splitting newline-separated outputs), variable management (accumulating context, clearing subcontext between iterations), and control flow (nested loops with correct iteration counts). A single off-by-one error in a loop, or an incorrect variable name in a context accumulation step, would produce a functionally broken strategy that passes basic validation (the code runs) but produces incorrect results (missing search results, wrong report content). The paper reports that code "conformance with execution requirements has been confirmed" before execution (Section 2.2, "Phase 2"), but this appears limited to "basic syntactic and execution checks" (Section 4) — which would not catch semantic errors.

What evidence exists in the paper. Only qualitative reports from the authors' development process. Section 2.2 ("Reliability") states that the generated code "rarely exhibits the failure modes encountered in our earlier prototypes, such as skipping strategy steps, misapplying search instructions, invoking the language model out of sequence, or introducing spurious checks and constraints not present in the original strategy." No quantitative reliability metrics are provided — not even a simple count like "in 50 strategy generations across 10 different strategies, X% produced code that passed validation and executed without errors on the first attempt." The four demonstrations in Appendix B show successful executions but represent an unknown sample from an unknown total number of attempts — the paper does not state whether these were first-attempt successes, or the result of multiple regeneration attempts with different seeds or prompt refinements.

Mitigation status. The paper does not quantify the reliability claim and does not propose methods for verifying semantic correctness of generated strategy code beyond basic execution checks. Section 4 acknowledges that "occasional semantic drift or hallucinated logic may still occur, particularly with ambiguous or underspecified strategies," but this acknowledgment does not characterize the frequency or severity of these occurrences. The reliability architecture (comment-code discipline + single-pass generation + pre-execution validation) is described as effective but not measured. For deployment in high-stakes enterprise settings (P2), this quantification gap is substantial — a system that silently produces incorrect behavior 5% of the time may be acceptable for casual research but unacceptable for financial or legal analysis.

7. Implications and Future Directions

How This Work Changes the Landscape

UDR does not introduce a new model architecture, a new training objective, or a new benchmark result. Its contribution is architectural rather than algorithmic — it demonstrates a design point in the space of agentic systems that has been largely overlooked in the rush toward increasing model autonomy. The paper's significance lies in making a credible case that the prevailing LM-as-orchestrator paradigm is not the only viable architecture for complex multi-step agentic workflows, and that inverting the relationship — making the LM a callable utility and delegating orchestration to generated code — can simultaneously solve several structural problems (transparency, model-agnosticity, user control over process) that LM-orchestrator systems inherit by design.

This is best understood as a design-space expansion rather than a paradigm shift. The paper is not arguing that all DRTs should be rebuilt on UDR's architecture, nor that LM-orchestrator systems are fundamentally flawed. Rather, it demonstrates that the design space for agentic systems is larger than current practice suggests, and that the specific point UDR occupies — strategy-as-data, LM-as-utility, state-in-variables — has attractive properties that no existing system provides. The magnitude of the contribution is that of opening a door that was previously assumed to be closed: prior to UDR, the default assumption in agentic system design was that any system flexible enough to handle arbitrary research strategies would require the LM to serve as the reasoning engine, planner, and decision-maker. UDR shows that a compiled-code architecture can achieve comparable flexibility with stronger guarantees of determinism, auditability, and model independence.

The reconciliation of contradictory design tensions. The paper resolves an implicit tension in agentic system design between expressiveness and control. Existing DRTs achieve control by hard-coding a fixed research strategy, sacrificing expressiveness entirely — the user cannot change how research is conducted. LM-orchestrator systems (the dominant paradigm in general-purpose DRTs) achieve expressiveness by letting the LM make methodological decisions, but sacrifice control — the user cannot audit or modify those decisions, and changing the model changes the behavior in opaque ways. UDR's architecture achieves both simultaneously: the user specifies the exact research methodology (control) in natural language (expressiveness), the system compiles it to deterministic code (auditability), and the LM serves only as a reasoning utility. This is a genuine resolution of a real design tension, not merely a different point on a continuum, because previous approaches could not achieve all three properties (user-specified strategy, deterministic execution, model-agnostic operation) within a single architecture.

Research directions that become more attractive. The paper's demonstration that arbitrary natural-language strategies can be compiled to reliable orchestration code — and its finding that the comment-code discipline substantially improves generation fidelity — makes compiled-code agent architectures a viable research direction that was previously underexplored. This extends well beyond deep research: any domain where users need to specify multi-step workflows with tool use (data analysis pipelines, code review procedures, document processing workflows, scientific literature reviews) could potentially benefit from a UDR-like architecture where strategy is a first-class input compiled to code rather than an implicit LM behavior.

More specifically, UDR makes the problem of strategy verification and optimization newly tractable. Because strategies are explicit, auditable code rather than implicit LM behaviors, they can be analyzed, tested, and improved through traditional software engineering methods. One could build a library of strategies, benchmark them against each other on standardized research prompts, identify common failure patterns (e.g., strategies that generate overly narrow search queries, strategies that aggregate context in ways that overwhelm the report-generation step), and iteratively refine them — all without retraining any model. This transforms research strategy design from a product-level design decision (locked into the system architecture) into a data-level engineering problem (strategies are artifacts that can be versioned, tested, and shared). The paper's recommendation R1 — providing a library of strategies rather than requiring from-scratch authorship — is the natural first step in this direction.

The finding that 8k tokens sufficed for all workflows regardless of complexity — if substantiated through further measurement — would have implications for the broader LLM systems community. The current trend toward million-token context windows assumes that more context is better and that the primary bottleneck for complex workflows is fitting everything into the LM's attention. UDR's architecture suggests an alternative hypothesis: for many structured workflows, selective context management through external state is not just more efficient but more reliable than context-window accumulation, because it avoids both the degradation in long-context LM performance and the opacity of what the LM attends to. If this finding generalizes, it would redirect research effort from building ever-larger context windows toward building better orchestration layers that manage state externally and call LMs with focused, scoped contexts.

Research directions that become less attractive. The paper's findings indirectly weaken the case for fully autonomous LM-orchestrated research agents as the universal solution for deep research. If UDR's architecture — with its deterministic control flow, small context windows, and model-agnostic design — can produce structured research reports with citations across diverse domains (as demonstrated in Appendix B), then the additional complexity and opacity of LM-orchestrated agents needs to be justified by measurable advantages in output quality, not assumed as the default. The burden of proof shifts: an LM-orchestrator DRT must now demonstrate that its autonomous decision-making produces better research outcomes than a well-designed user- or library-authored strategy executed by a compiled-code system, rather than relying on the implicit assumption that LM-level reasoning about research methodology is superior to explicit procedural specification. The paper does not make this argument polemically, but the architectural comparison is inherent in the system design — UDR deliberately excises the LM from orchestration, and if the resulting outputs are competitive (a question the paper leaves open), the case for LM-as-orchestrator weakens.

The paper also challenges the assumption that model capability and agent capability are inextricably linked. In current DRTs, improving the underlying model (e.g., upgrading from GPT-4 to GPT-4o) changes both the reasoning quality and the research behavior, making it impossible to attribute improvements to specific components. UDR's architecture makes these axes independent: one can upgrade the LM for reasoning tasks while keeping the strategy constant, or upgrade the strategy while keeping the LM constant. This suggests that holistic agent evaluation — where the entire system (model + strategy) is benchmarked as a unit — is less informative than component-level evaluation, where model reasoning quality and strategy design quality are assessed independently. Research programs that treat agent capability as a monolothic property to be maximized through model scaling become harder to justify when an alternative architecture separates the components and enables independent optimization.

Follow-Up Research This Work Enables

Cross-model code generation reliability benchmarking. The paper asserts that UDR "wraps around any language model" but tests exactly one (Llama 3.3 70B). A direct follow-up would measure the code generation success rate — the percentage of generated strategy functions that pass validation and execute without errors — across a panel of models spanning model families (Llama, GPT, Claude, Gemini, Qwen), model sizes (8B through 405B parameters), and capability tiers (base vs. instruction-tuned). The experiment would use a fixed set of 10–20 research strategies at varying complexity levels (similar to the three in Appendix A but expanded to include strategies with more steps, nested conditionals, and multi-phase validation), generate code once per model-strategy pair (with temperature 0 for reproducibility), and report: (a) pass@1 rate (syntactically valid, passes basic checks), (b) execution success rate (generator starts and yields notifications without runtime errors), and (c) manual audit of a subset for semantic correctness (does the generated code actually implement the specified strategy steps?). This would transform the model-agnosticity claim from architectural assertion to empirically bounded property, and would identify which models are practically usable as the strategy compiler in UDR. A negative result — e.g., models below 70B parameters consistently producing broken code — would establish a minimum capability threshold that the paper currently does not characterize.

Factual accuracy audit of UDR reports against cited sources. The paper acknowledges one hallucination in the stock movements report (B.3.2) but provides no systematic accuracy measurement. A critical follow-up would audit the factual accuracy of UDR-generated reports by checking a sample of claims against their cited sources. The experiment would use a set of research prompts where the correct answers are verifiable — the stock movements prompt (B.3) is ideal because opening/closing prices for specific dates are public and checkable — as well as prompts about historical events with documented facts, scientific topics with consensus answers, and biographical prompts with verifiable dates and events. For each report, a human annotator (or a secondary LM with access to the cited URLs) would check: (a) does the cited source actually contain the claimed information? (b) are there claims in the report that are not attributed to any source? (c) are there factual errors (not just interpretative differences) in the report? The experiment would compare UDR against at least one existing DRT (Gemini, Perplexity, or OpenAI Deep Research) on the same prompts, using the same annotation protocol, to establish whether UDR's citation accuracy is competitive. A negative result — e.g., UDR reports containing significantly more hallucinated claims than existing DRTs — would surface a quality gap that the current paper does not address but that is critical for the enterprise use cases UDR targets.

Strategy complexity scaling: at what step count does reliability degrade? The three example strategies in Appendix A range from approximately 10 to maybe 20 effective steps (the intensive strategy has nested loops). Real-world research strategies — for legal document review, financial due diligence, or systematic literature reviews — might have 50–100+ steps with branching logic, multiple validation passes, and complex state management. The paper provides no evidence that the comment-code discipline remains effective at those scales. A stress-test would generate strategies at increasing step counts (10, 20, 50, 100, 200 steps) — either hand-written or synthetically expanded from the example strategies — and measure: (a) code generation success rate, (b) strategy-step completion rate (what fraction of the specified steps are actually implemented in the generated code, as measured by counting comment-step pairs), (c) execution success rate, and (d) for the strategies that execute successfully, whether the output reports show evidence of the additional complexity (e.g., more nuanced search queries, more thorough cross-validation, longer and more detailed reports). This experiment would establish whether UDR has a practical ceiling on strategy complexity, which would determine its applicability to the high-value enterprise workflows it targets. A negative finding — e.g., reliability dropping sharply above 30–40 steps — would motivate research into hierarchical strategy compilation (compiling sub-strategies independently and composing them) or interactive strategy authoring where the user verifies segments as they are compiled.

Dynamic strategy adaptation through mid-execution user feedback. The paper acknowledges that UDR does not support mid-execution user intervention beyond full stop (Section 4), but the generator-based architecture with yield notifications is structurally well-suited for pause-and-resume interaction. A follow-up could extend UDR to support a "yield for input" pattern: at specified points in the strategy, the generated code yields a structured prompt to the user (e.g., "Found conflicting information on X from sources A and B. Which should be prioritized?" or "3 additional search directions identified: [1] ..., [2] ..., [3] ... Continue with which?"), pauses execution, and resumes when the user responds. The experiment would compare output quality (factual accuracy, comprehensiveness, user satisfaction) between fully autonomous execution and human-in-the-loop execution on research prompts that are genuinely ambiguous or exploratory — topics where the correct research strategy cannot be fully specified in advance because the user doesn't know what they'll find. This would test whether the paper's limitation (no dynamic adaptation) is a fundamental architectural constraint or merely an unimplemented feature, and whether human-in-the-loop interaction can compensate for the strategy authoring burden the paper identifies as tedious.

Verifier-based strategy evaluation and selection. The paper does not attempt to determine whether a given strategy is well-suited for a given prompt before executing it — users must select or author a strategy based on their own judgment. A natural extension would be to build a strategy evaluator that, given a research prompt and a candidate strategy (or a library of strategies), predicts which strategy will produce the highest-quality report before any search is performed. The evaluator could use features derived from the strategy structure (number of searches, number of LM calls, iteration depth, presence of cross-validation steps) and the prompt characteristics (domain, specificity, requested report length) to predict output quality, trained on a dataset of (strategy, prompt, quality score) triples generated by running multiple strategies on multiple prompts and having humans rate the resulting reports. This would address the paper's finding that strategy authoring is tedious — users could write a rough strategy and have the evaluator suggest improvements, or select from a library with the evaluator recommending the best match for their prompt. A negative result — strategy features not being predictive of output quality, or quality being dominated by search engine and LM factors outside the strategy's control — would clarify the limits of what strategy design can influence and redirect effort toward improving the underlying components (search, LM reasoning) rather than strategy optimization.

Comparison of variable-based state management against long-context accumulation for equivalent tasks. The paper claims that 8k tokens sufficed for all workflows and that variable-based state management is more efficient and reliable than context-window accumulation, but provides no comparative evidence. A controlled experiment would implement two versions of the same research task: one using UDR's variable-based state management (the LM is called with only the specific context needed for each subtask), and one using a long-context approach where all accumulated search results are appended to a single context and the LM is asked to produce the same outputs (search phrases, report) from that full context. Both versions would use the same underlying LM, the same search results, and the same output specifications. The experiment would measure: (a) token efficiency (total tokens processed across all LM calls), (b) factual accuracy of outputs (do the long-context version's reports contain more or fewer errors?), (c) latency (wall-clock time), and (d) context-size required for the long-context version to achieve comparable accuracy. This would directly test the paper's architectural hypothesis that selective, scoped context is more reliable than maximal context for structured multi-step workflows. A negative result — long-context accumulation matching or exceeding UDR's variable-based approach on accuracy at comparable cost — would challenge a core architectural motivation for UDR and suggest that the current trend toward giant context windows might be the simpler and equally effective path.

Practical Applications and Downstream Use Cases

Enterprise document research with auditable methodology. The paper's primary target — resolving P2 (specialized research strategies for high-value industries) — is the most direct application. In regulated industries like finance, legal, and healthcare, research outputs must often be accompanied by documentation of the research process: which databases were searched, in what order, with what query terms, and how conclusions were derived from retrieved documents. Current enterprise DRTs (NVIDIA AI-Q, SambaNova Deep Research) have fixed pipelines that cannot be adapted to specific regulatory requirements. UDR enables an organization to encode its compliance-mandated research procedure as a strategy — e.g., "search internal document database first; for any claims involving financial projections, cross-validate against SEC EDGAR filings; flag any sources older than 2 years; produce an audit trail showing which claim came from which source." Because the strategy is compiled to auditable code and all intermediate state is stored in named variables, the resulting research process is fully traceable — a regulator or internal auditor can inspect the generated code, the variable state at each step, and the mapping from report claims to source documents. The paper's finding that 8k tokens suffice regardless of workflow complexity means the system can handle large document collections without context-window constraints. The key enabler that makes this practical now (versus building a custom agentic system) is that the strategy can be written in natural language by a compliance officer or domain expert rather than requiring software engineering to modify the DRT's source code. The paper's recommendation R1 (providing a library of strategies) is critical here — an enterprise would deploy UDR with a library of pre-vetted, compliance-approved strategies for common research workflows, which users could select and minimally customize rather than authoring from scratch.

Competitive benchmarking of research strategies independent of model choice. The paper's model-agnostic architecture enables a use case that does not exist with current DRTs: strategy-only competition. A research organization could curate a benchmark of diverse research prompts (similar to the four in Appendix B but expanded to dozens across multiple domains) and invite strategy authors to submit natural-language strategies that UDR compiles and executes using a fixed, held-out LM. The quality of resulting reports could be evaluated through human judgment, factuality audits, or downstream task performance — and strategy authors could iterate and improve without ever touching the underlying model. This separates the problem of "what is the best way to research topic X" from "what is the best model for reasoning about X," allowing each to advance independently. The practical benefit is accelerating strategy innovation: a domain expert in pharmaceutical research could design a strategy optimized for drug discovery literature review and share it with colleagues, who could run it with their preferred model. The paper's finding that the comment-code discipline produces reliable strategy-to-code translation (even if not yet quantified) is the foundation that makes this feasible — submitted strategies must reliably compile to faithful code, or the competition degenerates into debugging LM code generation failures rather than evaluating strategy quality.

Cost-controlled batch research with per-task compute allocation. For organizations that need to perform research at scale — generating reports on hundreds or thousands of topics, monitoring news for competitive intelligence, or producing literature reviews for systematic reviews — UDR's strategy-as-input design enables fine-grained per-task compute budgeting. The paper's three example strategies represent different cost profiles: the minimal strategy makes ~2 LM calls and ~3 searches per report, the expansive strategy makes more calls (topic decomposition adds LM invocations), and the intensive strategy makes the most (iterative refinement with repeated LM queries). An organization could deploy all three strategies and route prompts to the appropriate one based on the prompt's estimated importance or complexity — low-priority monitoring prompts get the minimal strategy (cheap, fast), while high-stakes analysis prompts get the intensive strategy (thorough, expensive). The paper's architecture makes this straightforward because strategy is an explicit parameter — the same system runs all three strategies without modification. This is impossible with current DRTs, where the fixed strategy imposes a uniform cost per report regardless of the prompt's needs. The 8k-token-per-call finding is relevant here because it means per-report costs scale with the number and complexity of LM calls specified in the strategy, not with the total volume of research material accumulated, making cost prediction straightforward from the strategy specification alone. The primary barrier to this application is the current lack of strategy quality benchmarking — an organization would need to validate that each strategy produces reports of acceptable quality for its intended use case before deploying the routing system, which the paper's qualitative demonstrations do not yet support.

When to Prefer This Method

The paper does not present UDR as a direct alternative to any named system (e.g., "use UDR instead of Gemini Deep Research when...") or articulate a clear tradeoff against specific alternatives. It positions UDR as a different architectural paradigm — strategy-as-data, LM-as-utility — and directly addresses three problems (P1–P3) that existing DRTs cannot solve, but it does not claim that UDR produces better research reports than existing DRTs or identify conditions under which it would be preferred on output quality grounds. The paper's own recommendations (R1–R3 in Section 5) suggest the authors view UDR as a platform to be augmented with strategy libraries (R1) and extended with user control mechanisms (R2, R3) rather than as a drop-in replacement for existing DRTs in its current prototype form.

The closest the paper comes to a comparative positioning is its delineation of the three problems (P1–P3) that existing DRTs do not solve. This implies a conditional preference: prefer UDR over existing DRTs when (a) the user needs to enforce a specific research methodology that existing tools do not provide (P1), (b) the domain requires specialized document research strategies that consumer DRTs cannot express and enterprise DRTs have not implemented for that specific domain (P2), or (c) the user needs model-strategy independence — the ability to upgrade the underlying LM or switch between models without changing research behavior (P3). However, these preferences are based on capability (UDR can do something existing DRTs cannot) rather than quality (UDR does the same thing better). Because the paper provides no comparative quality measurements, a practitioner cannot determine whether choosing UDR for its flexibility comes at a cost in report accuracy, completeness, or citation quality relative to a fixed-strategy DRT optimized for that domain. The paper's own acknowledgment that strategy authoring is tedious and that factual accuracy is untested further complicates any simple decision rule — a practitioner who needs a custom research strategy might prefer UDR's flexibility but must accept the burden of strategy design and the uncertainty about output quality. A proper comparative decision framework would require the benchmarking experiments outlined in the follow-up research directions above.