MAI-UI Technical Report: Real-World Centric Foundation GUI Agents

ArXiv: 2512.22047

🎯 Pitch

MAI-UI’s online RL framework shows that simply scaling from 32 to 512 parallel training environments boosts AndroidWorld success rates by +5.2 points, while dynamically routing tasks between on-device and cloud models slashes cloud API calls by over 40% with no privacy loss. This 2B–235B family of GUI agents already surpasses Gemini-3-Pro and Gemini-2.5-Pro on ScreenSpot-Pro and AndroidWorld, respectively.

1. Executive Summary

This paper introduces MAI-UI, a family of foundation GUI agents spanning 2B to 235B-A22B parameters, that addresses four practical deployment challenges—agent–user interaction, MCP tool augmentation, device–cloud collaboration, and robustness to dynamic environments—through a unified methodology combining a self-evolving data pipeline, online RL with scaled parallel environments, and a native device–cloud collaboration system. On GUI grounding, MAI-UI-32B achieves 73.5% on ScreenSpot-Pro (surpassing Gemini-3-Pro and Seed1.8) and 91.3% on MMBench GUI L2; on mobile navigation, MAI-UI-235B-A22B sets a new state-of-the-art of 76.7% on AndroidWorld (exceeding UI-Tars-2, Gemini-2.5-Pro, and Seed1.8). The online RL framework yields significant gains from scaling parallel environments from 32 to 512 (+5.2 points) and increasing environment step budgets from 15 to 50 (+4.3 points), while the device–cloud collaboration system improves on-device performance by 33% and reduces cloud model calls by over 40%, establishing that test-time computation can be adaptively routed between local and cloud models based on task state and data sensitivity.

2. Context and Motivation

The Core Problem: GUI Agents Work in Labs, Not in the Real World

The fundamental problem this paper addresses is that Graphical User Interface (GUI) agents—AI systems that can perceive, reason about, and interact with graphical interfaces in response to natural language instructions—have made impressive progress in controlled benchmarks but remain fundamentally unready for reliable, robust, and secure deployment in practice. The paper identifies a specific gap between what existing systems can do and what real-world usage demands, and this gap is not a minor implementation detail—it represents a set of missing capabilities that, if unaddressed, prevent GUI agents from transitioning from research demonstrations to production systems that real users can trust.

This gap matters enormously because GUI agents represent a potential paradigm shift in human-computer interaction. The vision is compelling: instead of manually navigating complex interfaces—clicking through menus, remembering where settings are buried, copying data between apps—users could simply express their intent in natural language ("book a flight to Chicago next Tuesday morning, under $400, and add it to my calendar") and have an agent execute the full multi-step workflow. This would transform computing from tool-mediated (where the user must know how to use each tool) to goal-oriented (where the user specifies what they want and the agent figures out how). The paper explicitly frames this as a "revolution in digital interaction" (Section 1).

However, the gap between this vision and current reality is wide, and the paper argues that closing it requires solving four specific, interconnected challenges that prior work has largely left unaddressed.

The Four Challenges: What Makes Real-World Deployment Hard

The paper articulates these challenges in Section 1, and each one represents a design constraint that any production GUI agent must satisfy but that existing systems systematically fail to meet:

Challenge 1: Agent–User Interaction Is Missing

Existing GUI agent systems are typically optimized for end-to-end execution—given a complete, unambiguous instruction, execute it from start to finish without pausing. This works in benchmarks where tasks are carefully specified with all necessary parameters, but it breaks down catastrophically in real settings where user instructions are often ambiguous, incomplete, or underspecified.

The paper gives concrete examples of what this means in practice. A user might say "send the document to my colleague" without specifying which document, which colleague, or what method of sending. A benchmark-trained agent that cannot ask clarifying questions will either guess (and likely fail) or refuse the task. A real-world agent must proactively detect information gaps, ask targeted questions, and seek consent for sensitive operations (e.g., before making a purchase or sending an email to a distribution list). This capability—which the paper calls "agent–user interaction"—is not a nice-to-have feature; it is a fundamental requirement for aligning agent behavior with user intent when instructions are naturally imprecise.

The paper notes that this capability is "critical yet often neglected" in existing systems. Most prior GUI agent research treats user instructions as fixed, complete specifications and evaluates on whether the agent successfully executes the specified steps, not on whether it appropriately handles ambiguity. This creates a dangerous mismatch: models optimized for benchmark performance may confidently execute incorrect actions when faced with incomplete instructions, since they have been trained to always produce some action rather than to recognize when clarification is needed.

Challenge 2: UI-Only Operation Is Both Brittle and Limiting

Current GUI agents rely solely on UI manipulation—tapping, swiping, typing, and clicking on interface elements—to accomplish tasks. The paper identifies two distinct problems with this constraint:

Brittleness from long action sequences. Real-world tasks often require many sequential UI operations: to book a flight, an agent might need to open the airline app, navigate to the booking page, enter departure and arrival cities, select dates, choose a flight, enter passenger information, enter payment details, and confirm. Each step carries a non-zero probability of error—misidentifying a button, clicking at slightly the wrong coordinates, failing to scroll to the correct part of a page. Because each subsequent step depends on the previous one, errors propagate and compound. A 20-step sequence where each step has a 95% success rate has only a ~36% chance of complete success. This is the "brittleness to per-step errors" the paper describes.

Inherent capability ceiling. Some workflows are simply difficult or impossible to accomplish through UI manipulation alone on mobile devices. The paper gives a striking example: "via MCP tools, a mobile agent can manipulate GitHub repositories, bringing traditionally desktop-only workflows to mobile." GitHub's mobile web interface is cramped and limited; performing complex repository operations (viewing commit history with full metadata, comparing branches, managing pull requests) through UI taps is tedious at best and infeasible at worst. By integrating external tools through the Model Context Protocol (MCP)—a standardized interface for models to call APIs—the agent can bypass the UI entirely for operations that have programmatic equivalents.

The key insight here is that MCP integration provides structured shortcuts: calling a single API can compress what would be 10–15 fragile UI operations into one reliable function call. This simultaneously reduces brittleness (fewer steps = fewer opportunities for error) and expands the agent's capability boundary (it can now do things that the UI alone cannot support). The paper frames this as moving "beyond UI-only action" to a hybrid model where the agent intelligently chooses between GUI manipulation and API calls based on efficiency and feasibility.

Challenge 3: No Native Device–Cloud Collaboration Architecture

The paper identifies a structural problem in how GUI agents are deployed: they are typically either purely on-device (lightweight models that run locally on the phone or laptop) or purely cloud-based (large models accessed via API). Neither approach is adequate for realistic deployment, and the paper argues that the field lacks a native device–cloud collaboration capability that dynamically routes computation between local and cloud models.

Let me break down why each pole fails:

Cloud-only solutions have three critical drawbacks:

• Privacy risks: every screenshot of the user's screen—potentially containing emails, messages, financial information, passwords—must be transmitted to cloud servers. For many enterprise and personal use cases, this is a non-starter.
• Higher costs: cloud API calls incur per-request charges that scale with usage. For a GUI agent that might take 20–50 steps per task, this adds up quickly.
• Network dependency: the agent cannot function without connectivity. If the user is on a plane, in a tunnel, or in an area with poor coverage, the cloud agent is inaccessible.

On-device-only solutions face the opposite constraint:

• Limited model capacity: on-device models are constrained by the phone's memory, compute, and battery. They fundamentally cannot match the reasoning and perception capabilities of large cloud models (the paper compares their 2B on-device model to 235B cloud variants—a two-order-of-magnitude difference in parameters).
• Capability ceiling: as the paper's own results show (Table 6), the 2B on-device model achieves 49.1% on AndroidWorld, while the 235B cloud model achieves 76.7%. A 27.6-point gap on the same benchmark represents tasks that the on-device model simply cannot handle alone.

The paper's diagnosis is that neither mode alone fully meets real-world requirements, and that what's needed is a system where a capable but lightweight on-device model handles routine operations locally (preserving privacy, reducing cost, eliminating latency), while seamlessly handing off to a powerful cloud model when the task exceeds local capabilities (ensuring task completion). This handoff must be stateful (the cloud model must understand what has already happened), privacy-aware (sensitive data must not leave the device), and low-overhead (the switching mechanism must not add significant latency or complexity).

Crucially, the paper notes that "foundation GUI agents lack native device–cloud collaboration capability"—this is not a solved problem where existing systems just need better engineering. Building this collaboration requires designing both the model architecture and the runtime system to support bidirectional state transfer, monitoring of trajectory quality, and privacy-preserving routing decisions.

Challenge 4: Brittleness in Dynamic Environments from Static Training

This challenge gets at a fundamental limitation of how most GUI agents are trained. The standard approach is supervised fine-tuning on static, pre-collected trajectories: human annotators or existing agents perform tasks on specific app versions, screen layouts, and device configurations, and the recorded action sequences become training data. The model learns to imitate these specific trajectories.

The problem is that real-world GUIs are highly dynamic in ways that static datasets cannot capture:

• Layout variation across app versions: a button that was in the top-right corner in version 4.2 of an app might move to a bottom toolbar in version 5.0. A model trained only on version 4.2 screenshots will fail on version 5.0 because it has memorized absolute positions rather than developing a robust understanding of UI semantics.
• Device heterogeneity: the same app looks different on different screen sizes, aspect ratios, and resolutions. A trajectory recorded on a Pixel phone will not directly transfer to a Galaxy phone or a tablet.
• Unexpected interruptions: pop-ups, permission dialogs, notification banners, and system alerts can appear at any time and must be dismissed before the task can continue. The paper gives concrete examples in Figure 13: a notification permission request appears while the agent is creating a contact, and a file association dialog appears when opening a file. These interruptions were not present in the training data, so a model trained only on clean trajectories has no learned response to them.
• App state variability: whether a user is logged in or not, whether data has been cached, whether a trial has expired—all of these change the UI state in ways that training trajectories may not cover.

The paper's diagnosis is that agents trained on static data "overfit to specific interface patterns and struggle in out-of-domain scenarios" and "generalize poorly and remain brittle to real-world unpredictability." This is not merely a "more data" problem—no static dataset can exhaustively cover all possible pop-ups, layout variations, and device configurations. What's needed is training that exposes the model to dynamic environments during learning, so it develops robust strategies (e.g., "if an unexpected dialog appears, dismiss it and then resume the task") rather than memorizing brittle action sequences.

Where Existing Approaches Fall Short

The paper positions itself against a rich landscape of prior work, and its critique of existing approaches is specific and diagnostic rather than dismissive. Let me trace through the key limitations:

GUI grounding has advanced, but the SFT+RL paradigm is fragile. Prior work on GUI grounding—the fundamental capability of locating the correct UI element from a natural language instruction—has moved from pure supervised fine-tuning (SFT) approaches (JEDI, OS-Atlas, Aguvis, Uground, Aria-UI) toward reinforcement learning (RL) methods using GRPO (GUI-R1, GUI-Actor, GTA1). The RL-based methods achieved substantial gains because they allow the model to learn from its own mistakes rather than merely imitating human demonstrations. However, the paper identifies a critical problem with the pure RL paradigm: it is prone to policy collapse in the SFT+RL setting.

The paper explicitly cites Phi-Ground (Zhang et al., 2025a) as demonstrating that "SFT+RL framework is prone to policy collapse" (Section 4.1). What this means concretely: when you first fine-tune a model on grounding data (SFT) and then try to improve it with RL, the model often degenerates—it stops producing valid outputs, or it converges to a degenerate strategy like always predicting the center of the screen. The SFT phase, which teaches the model to output coordinates in the correct format, is necessary for stability, but the RL phase can undo this formatting knowledge if not carefully managed.

The paper positions its Instruction-as-Reasoning approach from the authors' prior work (UI-Ins, Chen et al., 2025) as solving this collapse problem. The key insight: instead of doing SFT with coordinate-only supervision (where the model just outputs "(x, y)"), the SFT phase teaches the model to generate diverse reasoning pathways before predicting coordinates—thinking about the element's appearance, function, location, and user intent before committing to a point. This "structured reasoning" pre-training makes the model more robust during RL because it has learned to produce coherent intermediate outputs rather than just jumping to coordinates. The paper argues this establishes "an effective example for the SFT+RL training paradigm" where previous attempts have failed.

GUI navigation has focused on either large cloud models or small on-device models, not their collaboration. The paper surveys the trajectory of GUI navigation research: early work used multi-component agent frameworks with separate planner and grounding modules (Mobile-Agent, Aria-UI), which were flexible but slow and complex. More recent work targets unified vision-language-action models that jointly learn grounding and navigation (UI-Tars, AgentCPM-GUI, UI-Venus), which are simpler and faster but require large models for strong reasoning. A growing line of work builds small on-device models (Ferret-UI Lite) for lower latency and improved privacy.

The paper's critique is subtle: each of these approaches is individually valid for its intended use case, but none of them provides a deployment-ready architecture that spans the full spectrum of real-world requirements. Large cloud models (UI-Tars-2, Gemini-2.5-Pro) achieve high success rates but raise privacy, cost, and connectivity concerns. Small on-device models (Ferret-UI Lite, 28% on AndroidWorld) address these concerns but cannot handle complex tasks. What's missing is a native collaboration system that dynamically routes between them—and this is what MAI-UI contributes.

Real-world capabilities—user interaction and tool use—are largely unevaluated in existing benchmarks. The paper makes a pointed observation about the evaluation landscape: existing benchmarks for mobile GUI agents (AndroidWorld, Mobile-Agent benchmark) "typically assume idealized interaction models where user instructions are perfectly clear and agents operate solely through GUI manipulation." This means the community has been optimizing for a simplified version of the problem that doesn't match production requirements. The paper adopts MOBILEWORLD (Kong et al., 2025), which explicitly includes tasks requiring agent–user interaction (45 tasks where the agent must detect ambiguity and ask clarifying questions) and MCP tool use (40 tasks where the agent must decide between GUI operations and API calls). Prior GUI agents were not designed or trained for these capabilities, so their performance on these tasks is near-zero (e.g., Qwen3-VL-32B: 6.7% on user interaction, 2.7% on MCP in Table 8). This gap—between benchmark performance on clean tasks and real performance on messy, interactive tasks—is precisely what the paper aims to close.

Online RL for GUI agents is underexplored due to infrastructure challenges. The paper acknowledges that online reinforcement learning—where the agent learns by interacting with a live environment and receiving rewards—has shown remarkable success in other domains (mathematical reasoning with DeepSeek-R1, visual reasoning with Vision-R1). But adapting this to GUI agents faces a severe infrastructure bottleneck: GUI environments are stateful and resource-intensive. Unlike mathematical reasoning, where you can verify a solution with a few lines of code, a GUI task requires running an actual Android emulator, maintaining app state across steps, and verifying task completion through screen inspection or backend access. Each training episode consumes significant memory and CPU resources, and scaling to thousands of parallel training episodes—which the RL literature shows is critical for performance—requires solving non-trivial systems engineering problems.

Prior work on GUI agent RL (like the RL fine-tuning in UI-Tars-2) did not scale to the level the paper argues is necessary. The paper's contribution here is showing that scaling parallel environments from 32 to 512 yields +5.2 points on AndroidWorld (Figure 12b), and that extending the interaction budget from 15 to 50 steps yields +4.3 points (Table 9). These are large effects—together, they account for most of the gap between SFT-only and RL-trained models—and they imply that previous RL attempts may have been limited not by algorithmic design but by insufficient environment scale and interaction depth.

How This Paper Positions Itself

The paper's positioning can be understood through three complementary framings:

1. It is a deployment-oriented integration paper, not a single-method paper. MAI-UI does not propose a single novel algorithm and evaluate it in isolation. Instead, it identifies a set of real-world deployment requirements and designs an integrated system that addresses all of them simultaneously. The components individually—the self-evolving data pipeline, the instruction-as-reasoning grounding method, the online RL framework, the MCP integration, the device–cloud collaboration system—each build on prior work. The novelty lies in (a) how these components are adapted and scaled to work together for GUI agents, and (b) the demonstration that addressing all four challenges jointly yields practical gains that addressing any single challenge in isolation would miss.

For instance, the device–cloud collaboration system requires both a capable local agent (which the 2B model training pipeline produces) and a capable cloud agent (which the 235B model provides)—neither alone would enable collaboration. The MCP integration requires extending the action space and data pipeline to include tool calls, which then feeds into the online RL training, which then improves the model's ability to decide when to use tools versus UI operations. These components are not independent modules; they are designed to compose.

2. It draws a direct parallel to the pretraining-inference compute tradeoff literature. While the paper does not explicitly frame itself this way in the related work section, the device–cloud collaboration system embodies a key principle: not all computation should happen at the same location or with the same model. This is conceptually similar to how the test-time compute scaling literature (which the paper does not cite) argues for adaptively allocating inference compute based on problem difficulty, except here the allocation is between local and cloud compute rather than between different search strategies. The paper shows that this adaptive routing yields concrete benefits: 42.7% of steps are executed locally, 40.5% of tasks are completed entirely on-device, and cloud model calls are reduced by over 40% (Figure 10). This is a cost, privacy, and latency win that neither pure-cloud nor pure-on-device approaches can achieve.

3. It pushes the field from "can the agent do this task?" to "can the agent do this task safely, privately, and efficiently in the real world?" The paper's four challenges are not about improving benchmark scores—though it does that too—but about addressing the deployment barriers that prevent GUI agents from being used in production. The emphasis on agent–user interaction acknowledges that users are not perfect instruction-writers. The emphasis on MCP integration acknowledges that UI manipulation is not always the best approach. The emphasis on device–cloud collaboration acknowledges that privacy and cost matter as much as capability. The emphasis on online RL in dynamic environments acknowledges that the real world is messier than any static dataset. Together, these form a comprehensive deployment thesis that goes beyond the typical "we trained a bigger model and got better numbers" narrative.

The paper is also forthright about what it does not solve. It acknowledges that the hardest tasks (difficulty bin 5 in the AndroidWorld analysis) remain challenging even for the largest models. It does not claim to have solved the privacy problem entirely—the privacy protection case study in Figure 11 is described as a "pilot study" with a "local privacy monitor" that is introduced as an additional component, not a fully integrated solution. And the MobileWorld benchmark results (41.7% overall for the 235B model, 37.3% for 32B) show that there is still a large gap between current performance and human-level reliability on realistic tasks. This candor strengthens the paper's credibility: it is claiming to have made substantial progress on practical deployment challenges, not to have solved GUI agents entirely.

3. Technical Approach

3.1 Reader Orientation

MAI-UI is a family of end-to-end neural models (not a pipeline of separate modules) that can look at a screenshot of a phone or desktop interface, read a natural language instruction, and either point to the correct UI element (grounding) or execute a sequence of actions (navigation) until the task is complete — while also knowing when to stop and ask the user a clarifying question, when to call an external API instead of tapping through the UI, and when to hand off from a small on-device model to a large cloud model for help.

The system solves the problem of deploying GUI agents in the real world by unifying four previously separate concerns — grounding, multi-step navigation, agent–user dialogue, and dynamic compute routing — into a single training and inference architecture. The "shape" of the solution is a multi-stage training pipeline (SFT → SFT → RL → online RL) applied to vision-language models at multiple scales, producing agents that can run entirely on-device, entirely in the cloud, or in a collaborative mode where a local agent monitors its own progress and escalates to a cloud agent only when it detects that it is stuck or about to make a privacy-sensitive mistake.

3.2 Big-Picture Architecture (Diagram in Words)

The MAI-UI system has five major components, arranged in both a training order (stages) and a runtime architecture (deployment):

1. GUI Grounding and Perception Pipeline — collects screenshots from real and virtualized environments, generates diverse natural-language instructions paired with element coordinates using the Instruction-as-Reasoning paradigm, and trains the base vision-language model to point to UI elements with explicit reasoning traces.

2. Self-Evolving Navigation Data Pipeline — generates large volumes of multi-step mobile navigation trajectories through a combination of human annotation, model-based rollouts, and iterative rejection sampling, where the model's own outputs are filtered and fed back as training data.

3. Extended Action Space Training — teaches the same model to issue ask_user (request missing information from the human) and mcp_call (invoke an external API tool) actions alongside standard GUI operations, using specially constructed trajectories where information is deliberately withheld or tasks are designed to benefit from API shortcuts.

4. Online Reinforcement Learning Framework — places the model in live Android emulators (up to 512 in parallel), lets it attempt tasks, evaluates success with hybrid rule-based and LLM-judge verifiers, and updates the policy using a customized GRPO algorithm with experience replay, repetition penalties, and asymmetric clipping.

5. Device–Cloud Collaboration System — at deployment time, a small on-device model serves as both the primary GUI agent and a trajectory monitor; when it detects that execution has deviated from the user's intent (and no sensitive data is on screen), it hands off to a large cloud model with an error summary for recovery.

Information flows through these components sequentially during training: the grounding pipeline produces a model that can localize elements → the navigation pipeline extends it to multi-step execution → online RL makes it robust to dynamic environments. At deployment, information flows bidirectionally between the local and cloud agents through a shared unified trajectory memory.

3.3 Roadmap for the Deep Dive

• First, the task formulation and action space (Section 3.4.1): what the model actually predicts and what actions it can take — this establishes the vocabulary for everything that follows.
• Second, the grounding and perception pipeline (Section 3.4.2): how the foundational "point to the right element" capability is built, including the Instruction-as-Reasoning paradigm that prevents RL policy collapse.
• Third, the self-evolving navigation data pipeline (Section 3.4.3): how multi-step trajectories are generated, filtered, and iteratively refined — this is the bridge from single-step grounding to multi-step navigation.
• Fourth, agent–user interaction and MCP augmentation (Section 3.4.4): how the model learns to talk to humans and call APIs — these are the capabilities that distinguish real-world agents from benchmark-only agents.
• Fifth, the online RL framework (Section 3.4.5): how the model learns from live environment interaction, including the infrastructure for scaling to 512 parallel emulators, the reward design, and the algorithmic modifications to standard GRPO.
• Sixth, the device–cloud collaboration system (Section 3.4.6): how the local and cloud agents work together at runtime, including the monitor training procedure and the privacy-preserving routing logic.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and integration paper whose core idea is that the real-world deployment barriers for GUI agents — lack of user interaction, UI-only brittleness, no device–cloud routing, and static-training fragility — can be addressed simultaneously through a unified multi-stage training and deployment architecture. The technical contribution is not a single algorithm but the orchestration of several techniques (instruction-as-reasoning, iterative rejection sampling, online RL at scale, and device–cloud monitoring) into a coherent whole that produces deployable agents at multiple scales.

3.4.1 Task Formulation and Action Space

The paper defines two categories of tasks that MAI-UI handles: GUI grounding and mobile GUI navigation. These are not separate models; they are capabilities of a single model trained jointly.

Grounding Task. GUI grounding is the problem of locating the UI element described by a natural language instruction within a screenshot. Formally:

Given a GUI screenshot and a natural language instruction $I$, the GUI agent model predicts a coordinate point $P = (x, y)$ that indicates the location of the target UI element.

This is framed as a coordinate regression problem but implemented as token generation: the model outputs the $(x, y)$ coordinates as text tokens in its response, typically preceded by reasoning text that explains why that element was chosen. The evaluation metric is point-in-box accuracy: a prediction is correct if the predicted coordinate falls within the ground-truth bounding box of the correct element (Equation 1).

Navigation Task. Mobile navigation is formulated as a Partially Observable Markov Decision Process (POMDP) with four components:

• $S$: the state space of the underlying mobile environment (app states, data, UI configurations).
• $O$: the observation space, which consists of a natural language instruction $I$ plus one or more screenshots of the current screen.
• $A$: the action space (defined below).
• $T: S \times A \rightarrow S$: the state transition function, which is the actual mobile environment — when the agent takes an action, the environment changes.

At each time step $t$, the agent predicts the next action as:

$a_t = \pi(I, o_t, h_t)$

where $I$ is the natural language instruction (constant throughout the task), $o_t \in O$ is the current observation (screenshot(s)), and $h_t = (a_1, o_1, \dots, a_{t-1}, o_{t-1})$ is the history context of all previous actions and observations. The policy $\pi$ is the MAI-UI model itself — it takes the instruction, the current screenshot, and the history, and outputs the next action.

Action Space. The paper defines a comprehensive action space for mobile GUI control, summarized in Table 1. The standard GUI actions include:

• click: tap at coordinates $(x, y)$
• long_press: long press at $(x, y)$
• type: enter specified text content
• swipe: swipe in a direction (up/down/left/right) at optional coordinates
• drag: drag from start to end coordinates
• system_button: press back, home, menu, or enter
• wait: pause briefly
• terminate: mark task as complete (success or fail)
• answer: provide a direct text answer (for question-answering scenarios)

The extended actions — which are central to the paper's real-world deployment thesis — are:

• ask_user: request user intervention with specified text (e.g., "What is the recipient's email address?"). This action pauses the autonomous execution and waits for human response.
• mcp_call: invoke an MCP tool with a tool name and corresponding arguments in JSON format. The tool executes on a server and returns structured output that the agent incorporates into subsequent steps.

Model Architecture. The paper uses Qwen3-VL (Bai et al., 2025a) as the backbone vision-language model across all scales. The MAI-UI family includes 2B, 8B, 32B, and 235B-A22B (a mixture-of-experts variant with 235B total parameters and 22B active) variants. All variants share the same architecture and are trained with the same multi-stage procedure, differing only in parameter count and (for the larger variants) additional real-world trajectory data.

Each model is trained to accept interleaved screenshot images and text (the instruction plus action history) and output text that includes both a reasoning/thinking section and a structured action in a predefined format. The thinking and action sections are distinguished by special XML-style tags, which enables format verification during training (the format reward in RL) and structured parsing at inference time.

3.4.2 GUI Grounding and Perception Training

Grounding — the ability to point to the correct UI element given a natural language instruction — is the foundational capability upon which all navigation is built. If the agent cannot reliably identify which button to press or which text field to type into, no amount of high-level planning will help. This section covers how the paper builds this capability.

Data Collection. The data pipeline (Figure 3) collects screenshots from two sources:

1. Open-source datasets such as JEDI (Xie et al., 2025a) and OS-Atlas (Wu et al., 2024), which provide pre-annotated screenshots with element bounding boxes.
2. Virtualized operating systems in containerized environments: the authors deploy actual GUI operating systems (both desktop and mobile) inside containers and use an MLLM-guided exploration strategy. At each step, the MLLM identifies valid actions from the current screen state, executes one, and captures the new screenshot. This produces a continuous stream of diverse, realistic GUI states.

After collecting screenshots, UI elements are localized using one of two methods: the accessibility tree (a11y tree — the structured representation of UI elements that operating systems provide for accessibility tools) or OmniParser V2 (Lu et al., 2024b), a vision-based UI parsing model. Both methods produce bounding boxes for individual UI elements.

Perception Data Generation. For each screenshot, the pipeline randomly selects one to three UI elements and prompts an MLLM to generate diverse tasks based on those elements, including question answering ("What is the title of this dialog?"), captioning ("Describe the layout of this settings page"), and state prediction ("What will happen if the user taps this button?"). These tasks are designed to develop the model's semantic, relational, and layout understanding of interfaces without requiring coordinate outputs — they teach the model what UI elements mean and how they relate to each other, which is prerequisite knowledge for grounding.

Grounding Data Generation: The Instruction-as-Reasoning Paradigm. This is where the paper's approach diverges significantly from prior work. The authors' prior work (UI-Ins, Chen et al., 2025) demonstrated two critical findings about grounding data:

• Instruction quality matters enormously: approximately 23.3% of instructions in open-source grounding datasets have quality issues (ambiguous references, incorrect element descriptions, inconsistent labeling) that actively harm performance when models are trained on them.
• Instruction diversity is as important as correctness: models trained on instructions from multiple human-like perspectives — describing the same element by its appearance, its function, its location, or the user's intent — significantly outperform models trained on a single perspective. Moreover, performance improves further when the model learns to select the appropriate perspective for each scenario rather than always using the same one.

The Instruction-as-Reasoning paradigm in MAI-UI operationalizes these findings as follows: for each UI element, an MLLM generates instructions from four distinct perspectives:

• Appearance: what the element looks like ("the blue button with rounded corners in the top-right corner").
• Function: what the element does ("the button that submits the form").
• Location: where the element is positioned relative to other elements ("the icon to the left of the search bar").
• Intent: why a user would want to interact with this element ("the option to share this document with team members").

The key design choice is that these diverse instructions are not merely used as data augmentation at training time — they are used as explicit reasoning pathways that the model learns to generate before predicting coordinates. Rather than the model seeing an instruction and directly outputting (x, y), it learns to first identify the most informative perspective given the instruction and the screenshot, reason through that perspective, and then predict coordinates. This injects structured reasoning into what is otherwise a purely coordinate-regression task.

Training Paradigm for Grounding. The grounding training proceeds in two stages:

Stage 1: Supervised Fine-Tuning (SFT). Using the instruction-as-reasoning data described above, the model is fine-tuned to generate both reasoning text and final coordinates. The loss is the standard next-token prediction loss (cross-entropy between the model's output distribution and the ground-truth token sequence). The model learns to output sequences of the form:

<thinking>
[Appearance perspective: The target element is a circular blue icon with a plus symbol, located in the floating action button area at the bottom-right of the screen...]
</thinking>
<answer>[x, y]</answer>

This SFT stage teaches two things simultaneously: (1) the format and conventions for structured reasoning plus coordinate output, and (2) a diverse repertoire of reasoning strategies that the model can draw upon.

Stage 2: Reinforcement Learning with GRPO. After SFT, the model undergoes RL using the Group Relative Policy Optimization (GRPO) algorithm (Guo et al., 2025a). GRPO is a variant of PPO that eliminates the need for a separate value function (critic) by using the average reward within a group of sampled outputs as the baseline. For each instruction, the model generates $G$ candidate responses, and the advantage for each response is computed as its reward normalized relative to the group.

The reward for grounding consists of two components:

Format Reward ($R_f$): a binary reward that checks whether the model's response contains the thinking content and final answer in their correct XML-style tags and whether the coordinates are extractable for evaluation. This reward ensures the model maintains proper output structure throughout RL training.

Point-in-Box Reward ($R_{acc}$): a binary reward defined as:

$R_{acc} = \begin{cases} 1 & \text{if } x_l \leq x_p \leq x_r \text{ and } y_l \leq y_p \leq y_r \\ 0 & \text{otherwise} \end{cases}$

where $(x_p, y_p)$ is the predicted coordinate and $(x_l, y_l, x_r, y_r)$ defines the ground-truth bounding box with top-left corner $(x_l, y_l)$ and bottom-right corner $(x_r, y_r)$.

What it computes: a simple binary indicator of whether the model's predicted point falls inside the correct UI element's bounding box. No partial credit is given — if the point is even one pixel outside the box, the reward is zero.

Why this form: the paper notes that "dense reward formulations yield similar performance," so they opted for this simple binary scheme. The point-in-box metric is the standard evaluation metric used by all GUI grounding benchmarks (ScreenSpot-Pro, UI-Vision, MMBench-GUI), so optimizing it directly makes the training objective identical to the evaluation metric. This avoids the common RL pitfall of optimizing a proxy reward that is not perfectly correlated with the true evaluation criterion.

How Instruction-as-Reasoning Prevents Policy Collapse. The paper explicitly describes a problem that prior work encountered: when models are trained with SFT on coordinate-only supervision and then fine-tuned with RL, they often experience policy collapse — the model degenerates to producing invalid outputs (e.g., always predicting the center of the screen, or generating unparseable coordinate formats). The Instruction-as-Reasoning approach prevents this because:

1. The SFT stage teaches the model to generate diverse, structured reasoning before predicting coordinates. This creates a richer output distribution where the model has multiple "strategies" (the different reasoning perspectives) rather than a single mapping from instruction to coordinates.
2. During RL, the model can explore within this space of reasoning strategies — trying appearance-based reasoning on one rollout, function-based reasoning on another — rather than making random perturbations to coordinate predictions. This structured exploration space is more likely to contain successful trajectories.
3. The format reward maintains the structural constraints (thinking tags, answer tags), preventing the model from drifting into unparseable output formats even as it explores.

The paper frames this as addressing the core limitation of pure RL paradigms: "a key limitation of a pure RL paradigm is that it overlooks the substantial benefit offered by an initial SFT stage." The Instruction-as-Reasoning SFT provides that benefit in a way that is specifically designed to be compatible with subsequent RL — the reasoning pathways developed during SFT serve as exploration scaffolds during RL.

Zoom-In Strategy for High-Resolution Scenarios. During inference (not training), the model can optionally use a two-pass zoom-in procedure for complex, high-resolution interfaces like professional CAD software or dense spreadsheets:

1. First pass: the model processes the full-resolution screenshot and predicts a coarse coordinate.
2. Crop: a window centered on the predicted point is cropped, with width and height each equal to half the original image dimensions. This window is then resized back to the original resolution, effectively zooming in on the region of interest.
3. Second pass: the model processes the zoomed-in region and outputs the precise refined coordinate.

This strategy is optional and is applied only when the initial prediction is on a high-resolution interface where fine-grained localization is challenging. The paper reports results both with and without zoom-in (Tables 2, 3, 4, 5), showing consistent improvements of 3–5 percentage points on challenging benchmarks.

3.4.3 Mobile GUI Navigation Training: Self-Evolving Data Pipeline

While grounding is the foundational capability, navigation is the end goal — executing multi-step sequences of actions to complete user instructions. This section covers the data pipeline that produces training trajectories for navigation, which is the most data-hungry component of the system.

The navigation training pipeline (Figure 4) operates in three major phases: Navigation Task Generation, Trajectory Synthesis, and Iterative Rejection Sampling. The key design principle is that the pipeline is self-evolving: the model and the training data improve each other through multiple rounds of generation, filtering, and fine-tuning.

Phase 1: Navigation Task Generation. Task instructions — the natural language goals that the agent must accomplish — come from three sources:

1. Application manuals: common usage scenarios are parsed from app documentation and distilled into intent-level task descriptions. For example, a calendar app manual might describe "create a recurring weekly meeting" or "share an event with contacts."
2. Expert-designed tasks: human annotators formulate realistic and diverse mobile navigation goals aligned with commonly used scenarios. These are intentionally varied in complexity, ranging from single-app operations ("turn on dark mode in Settings") to cross-app workflows ("find the address from the latest email and navigate to it in Maps").
3. Open-source datasets: existing navigation datasets are filtered by task complexity and reachability — the paper ensures that tasks are actually feasible given the applications available in their environment.

This multi-source strategy is designed to maximize both diversity (different app domains, different task types) and quality (the filtering ensures that tasks are executable and well-specified).

Phase 2: Trajectory Synthesis. Once a large set of task instructions exists, the pipeline generates execution trajectories — the step-by-step sequences of actions, screenshots, and observations that accomplish each task. This phase has several sub-components:

Seed Task Expansion. Starting from the seed tasks produced in Phase 1, an MLLM is prompted to generate novel task variants. The paper categorizes this expansion into two levels of diversity:

• L1 expansion: adjust critical parameters of the original task goal — date/time ranges, numeric thresholds, sorting/filter criteria, etc. For example, "find emails from last week" might be expanded to "find emails from last month" or "find emails from yesterday with attachments."
• L2 expansion: replace the core objects involved in the task while remaining constrained to the same scenario and set of applications. For example, "send the latest photo to Mom" might be expanded to "send the second-most-recent document to your manager" or "share the oldest screenshot with the family group chat."

This expansion serves two purposes: it increases dataset size (more training examples) and, more importantly, it increases instruction diversity, which the grounding analysis showed is critical for robust performance.

Model Synthesis and Human Annotation. The expanded tasks are executed through two parallel pipelines:

1. Human annotation: annotators manually perform each task on an Android emulator, recording both screenshots and ground-truth action sequences at each step. This produces high-quality trajectories but is slow and expensive.
2. Model-based synthesis: multiple existing GUI agents are used to automatically produce valid action sequences for navigation tasks. Because multiple valid execution paths often exist for a given task (e.g., you can navigate to Settings through the gear icon or through the app menu), using multiple agents broadens trajectory coverage and "enhance[s] dataset robustness."

The paper explicitly notes that "for a given task goal, multiple valid execution paths often exist. By combining these complementary sources, we significantly broaden trajectory coverage and enhance dataset robustness." This is a crucial design insight: training on multiple valid paths to the same goal teaches the model that there isn't a single "correct" sequence of actions, which makes it more robust when the UI layout differs from what it saw during training.

Fine-Grained Correctness Judgment. After trajectories are generated, they undergo quality assessment through two independent validation pipelines:

1. Manual quality check (for human-annotated trajectories): a second annotator verifies alignment between the action sequence, screenshots, and the original task goal. Inconsistent or ambiguous demonstrations are either corrected or discarded.
2. Fine-grained correctness judgment (for model-generated trajectories): an MLLM-as-a-judge module analyzes the task instruction, action history, and screenshots to assess correctness at both the trajectory level and the step level.

The fine-grained judgment approach is one of the paper's most important data-efficiency innovations. The key observation is:

"many generated trajectories fail to fully complete the intended goal. However, failed trajectories often contain a substantial prefix of correct actions, with errors typically occurring only at intermediate or later steps. Recognizing that not all steps in a failed trajectory are erroneous, we adopt a fine-grained judging approach to identify and retain useful sub-trajectories."

This means the judge performs two evaluations:

• Overall Trajectory Judgment: assess end-to-end success, prioritizing visual evidence from screenshots over textual claims generated by the GUI agent (since agents can hallucinate success). If the trajectory is fully successful, it is kept in its entirety.
• Erroneous Trajectory Reuse: for failed trajectories, identify the longest prefix of correct actions before the first deviation. This prefix — a valid partial trajectory — is retained as training data, while the erroneous suffix is discarded.

This approach "reduce[s] data waste and enable[s] the model to learn from partial successes." In practice, it means that a trajectory that succeeds for 8 steps and then fails on step 9 can still contribute 8 correct examples to the training set, rather than being entirely discarded as a failure. This is critical because model-generated trajectories have high failure rates, especially early in training; without partial-trajectory reuse, the data yield would be prohibitively low.

Phase 3: Iterative Rejection Sampling. The final phase is an iterative self-improvement loop that jointly refines both the model and the training data distribution. Let $M^{(t)}$ denote the model after the $t$-th round of fine-tuning, and let $I_{\text{expansion}}$ be the set of diverse task instructions from the seed task expansion phase.

In round $t+1$:

1. Rollout: the current model $M^{(t)}$ is deployed as the rollout policy to generate new trajectories on $I_{\text{expansion}}$:

$D^{(t+1)}_{RS} = \{\text{Rollout}(M^{(t)}, i) \mid i \in I_{\text{expansion}}\}$

where $D^{(t+1)}_{RS}$ is the set of all generated trajectories and $\text{Rollout}(M^{(t)}, i)$ represents executing the model on task instruction $i$ in the mobile environment.

2. Filtering: each trajectory is filtered through the fine-grained correctness judgment module to retain only high-quality or partially correct segments.

3. Data mixing: the training set for the next iteration is constructed by combining the newly generated rejection-sampled data with novel trajectories synthesized from the Trajectory Synthesis phase:

$D^{(t+1)} = \alpha \cdot D^{(t+1)}_{RS} \cup \beta \cdot D_{\text{synthesis}}$

where $D_{\text{synthesis}}$ denotes the trajectories generated from human annotation and agent rollouts (Phase 2), and $\alpha$ and $\beta$ are mixing proportions (the exact values are not specified in the paper, but the principle is that both sources contribute to each iteration).

4. Fine-tuning: the model $M^{(t+1)}$ is fine-tuned on $D^{(t+1)}$, completing the iteration.

Why this iterative process works better than one-shot training:

• Rejection-sampled data closes the gap between pass@1 and pass@N: the model can sometimes succeed on a task if it tries multiple times (pass@N > 0) even if its most likely output is wrong (pass@1 = 0). By rolling out the model, keeping the successful trajectories, and training on them, the model learns to make its pass@1 behavior more like its pass@N behavior. This is the standard motivation for rejection sampling fine-tuning.
• Novel trajectories injected each iteration raise the pass@N ceiling: if the model were only trained on its own filtered outputs, it would converge to a local optimum — it would get better at what it already does but would never learn fundamentally new strategies. By continuously injecting fresh trajectories from human annotation and other agents, the pipeline ensures that the model is exposed to new solution patterns that it wouldn't discover on its own.
• The data distribution aligns with the model's evolving capabilities: early in training, the model is weak and would fail on complex tasks; training on those tasks would be inefficient because the trajectories would be mostly erroneous (yielding only short correct prefixes). By using the current model to generate data, the pipeline naturally focuses on tasks that are at the edge of the model's current capability, which is the most informative region for learning.

The paper does not specify the exact number of iterations, but the conceptual framework is that this loop continues until the performance improvements from additional rounds diminish.

Supervised Fine-Tuning for Navigation. After the self-evolving data pipeline produces a large corpus of high-quality trajectories, the model undergoes SFT on this data (mixed with a small portion of grounding data to prevent catastrophic forgetting of the grounding capability). This SFT stage produces the base navigation model that will later be improved by online RL.

The training uses standard next-token prediction loss on the trajectory sequences, where the model learns to predict each action given the instruction, the current screenshot, and the history of previous actions and screenshots. The model is trained to output not just the action but also a reasoning/thinking segment before each action, following the same Instruction-as-Reasoning pattern established in grounding training.

3.4.4 Agent–User Interaction and MCP Tool Use

These two capabilities — knowing when to ask the user for missing information and knowing when to use an external API instead of GUI manipulation — are what the paper identifies as critical for real-world deployment but absent from most existing GUI agents. They are trained by augmenting the self-evolving data pipeline with specialized trajectories, which are then mixed into the SFT data.

Agent–User Interaction Training. The core challenge is that the model must learn to detect information gaps and issue the ask_user action rather than guessing or failing. The training procedure works as follows:

1. Task construction with deliberately omitted information. Tasks are designed such that critical parameters are missing from the user instruction. For example, a task might be "send the document to my colleague" without specifying which document, which colleague, or which communication channel.

2. During annotation or rollout, when the annotator or agent reaches a step that requires the missing information, it issues an ask_user action with a natural-language query requesting the specific missing detail (e.g., "Which document would you like me to send? I can see three PDF files in your recent documents.").

3. Synthetic user agent response. The query is routed to a synthetic user agent — an LLM that is conditioned on hidden context containing the missing information (e.g., the hidden context specifies that the colleague is "Mike" and the channel is "email"). The synthetic user returns a concise, context-appropriate reply that provides the missing information: "Please send the Q4 report to Mike via email."

4. Trajectory continuation. The query–response pair is logged in the action history, and the annotation or rollout continues, incorporating the returned information to complete the task. This produces a complete trajectory that includes both the ask_user and the subsequent actions using the user's response.

5. Both single-turn and multi-turn interactions are generated. Some tasks require only one clarification; others require multiple rounds (e.g., first asking which document, then asking which recipient, then asking whether to include a cover message).

The synthetic user agent is a critical design choice. Using a real human for every training trajectory would be prohibitively expensive and slow. Using a fixed set of pre-written responses would limit diversity. The LLM-based synthetic user can generate diverse, contextually appropriate responses while maintaining consistency with the hidden ground truth. This enables generating large volumes of interaction-training data without human bottleneck.

MCP Augmentation Training. MCP (Model Context Protocol) is a standardized interface for models to call external tools and APIs. Training the model to use MCP tools involves:

1. Task design for MCP-beneficial scenarios. Tasks are designed that either require or substantially benefit from external MCP tools. The paper mentions three tool categories as examples: Amap (mapping and directions), GitHub (repository operations), and Stockstart (financial data).

2. During trajectory annotation or rollout, the annotator or agent can issue mcp_call with a tool name and arguments in JSON format. For example: mcp_call(name="amap_maps_direction_driving", arguments={"origin": "120.103631,30.278430", "destination": "120.022842,30.277608"}).

3. MCP server execution. The MCP server receives the call, executes it (e.g., queries the Amap API for driving directions), and returns structured output (e.g., JSON with distance, duration, route steps).

4. Trajectory recording. The tool schema (function signature), arguments, results, and the subsequent UI actions are all recorded in the trajectory. Only trajectories that demonstrate correct MCP tool selection and usage are retained.

5. The model learns both to recognize when an MCP call would be beneficial (rather than continuing with UI operations) and how to invoke the correct tool with correct arguments based on the task context and the information available on screen.

The paper presents two case studies (Figure 8) that illustrate the two core benefits of MCP augmentation:

• Compressing multi-step UI operations into a few API calls: in Figure 8a, comparing driving times between two apartment addresses and a destination would normally require switching between SMS (to read the addresses) and a maps app (to compute routes), copying and pasting addresses, and running two separate route searches. With Amap MCP calls, the agent simply queries the API directly with the coordinates, compressing what might be 15–20 UI actions into 2–3 tool calls.
• Enabling desktop-only workflows on mobile: in Figure 8b, querying GitHub commit history and formatting it for email is straightforward on a desktop browser but cumbersome on a mobile interface. The MCP GitHub API call retrieves structured commit metadata directly, bypassing the limitations of mobile web interfaces.

The model is trained on these MCP-augmented trajectories alongside standard GUI-only trajectories, so it learns to make context-dependent decisions about whether UI manipulation or API calls are more appropriate for a given subtask.

3.4.5 Online Reinforcement Learning in Dynamic Environments

The SFT-trained navigation model can execute tasks it has seen in training but remains brittle to unexpected situations — pop-ups, permission dialogs, layout variations — that were not captured in the static trajectory data. Online RL addresses this by placing the model in live, dynamic environments and letting it learn from both successes and failures.

Environment Infrastructure. The paper builds a containerized mobile environment system designed specifically for RL training. The key components:

• Containerized Android emulator: the entire GUI environment is encapsulated within a Docker image, comprising a rooted Android Virtual Device (AVD), self-hosted backend services, and a dedicated REST API server for orchestration. This design ensures:

  • Consistency: the unified containerization eliminates external dependencies and guarantees behavioral consistency across heterogeneous host systems — every training run sees the same environment behavior regardless of which physical server it runs on.
  • Generalizability: the environment integrates over 35 applications spanning system utilities and open-source software (Mattermost for enterprise communication, Mastodon for social media, Mall4Uni for e-commerce). Self-hosting these applications provides full backend access, enabling precise manipulation of initial task states and deterministic verification of execution outcomes — something that would be impossible with live third-party services.
  • RL-native design: the environment exposes standard RL primitives through the containerized API: reset (initialize a task to a known starting state), step (execute an action and return the new observation), get_observation (capture the current screenshot), evaluate (check whether the task is complete and whether it succeeded), and close (release resources).

• AVD snapshot mechanism: for reproducible task initialization, the system uses Android Virtual Device snapshots. Each task has a predefined starting state saved as a snapshot; calling reset restores the device to that exact state, ensuring that every training episode begins from the same conditions.

• Centralized Environment Manager: to scale environments across distributed infrastructure, the paper introduces a manager that coordinates container instances across multiple physical machines. This manager handles:

  • Efficient resource utilization: through automatic container reuse, environments are reset and reassigned upon rollout completion instead of being destroyed and recreated, which would add significant overhead.
  • Cross-machine orchestration: a unified REST API provides transparent access to distributed resources across heterogeneous hosts, so the training code doesn't need to know which physical machine each emulator is running on.
  • Fault tolerance: automatic detection and recovery mechanisms handle container failures, with failover protocols that seamlessly replace compromised instances from a standby pool — critical for continuous online RL training that may run for days or weeks.

The paper reports that "by coordinating just 10 standard Alibaba Cloud ECS servers (ecs.ebmg5s.24xlarge), the Manager supports up to 512 concurrent environment instances for parallel rollout execution."

Long-Horizon RL Training Framework. Training RL agents for GUI tasks with 50-step horizons faces two interconnected technical challenges:

1. Synchronous rollout inefficiency: in standard RL training, the model generates actions (inference), the environment processes them (which may take seconds for GUI operations), and then the model generates the next action. If inference and environment interaction are sequential, the GPU sits idle while the environment processes actions, and the environment sits idle while the GPU computes.

2. Memory constraints from ultra-long trajectories: a 50-step trajectory can contain millions of tokens (each screenshot is encoded into hundreds or thousands of visual tokens, and 50 screenshots plus action text adds up quickly). These long sequences exceed single-GPU memory capacity, preventing end-to-end gradient computation.

The paper addresses these challenges with two optimizations built on top of the verl framework (Sheng et al., 2024):

Asynchronous Rollout for Multi-Turn Efficiency. The paper implements a custom agent loop (Figure 6) that decouples inference and environment interaction:

• The agent loop asynchronously dispatches inference requests to a group of GPU servers hosting the latest policy model. While one set of actions is being generated, another set can be executing in the environment.
• The loop incorporates asynchronous environment interaction with session management, maintaining backup sessions for seamless failover — if one environment container crashes, the loop can immediately switch to a backup without losing the trajectory so far.
• On the server side, load balancing distributes inference requests across available GPUs, and prefill caching accelerates generation efficiency in multi-turn settings by reusing the KV cache of previous turns (since the instruction and earlier screenshots in the history haven't changed).

Hybrid Parallelism for Ultra-Long Sequences. To enable end-to-end training on trajectories with millions of tokens, the paper leverages Megatron's hybrid multi-dimensional parallelism:

• Tensor parallelism (TP): splits individual weight matrices across GPUs, so each GPU stores and computes only a fraction of each layer.
• Pipeline parallelism (PP): splits the model into sequential stages, each assigned to a different GPU, so different micro-batches can be processed simultaneously at different pipeline stages.
• Context parallelism (CP): splits the sequence itself (the trajectory tokens) across GPUs along the sequence dimension, so no single GPU needs to store the entire sequence in memory.

By combining these three parallelism strategies, the system can shard each long rollout trajectory across many GPUs while keeping per-GPU memory bounded, enabling training on trajectories that would be impossible to fit on a single device.

Additionally, the paper resizes images to half their original resolution (from 1080p to approximately 720p equivalent). This significantly reduces the number of visual tokens per screenshot (approximately 50.1% speedup per step) without compromising model performance — the paper reports that 720p "achieves performance comparable to 1080p."

Task and Verifier Design. Effective RL requires a well-structured task distribution and reliable reward signals.

Task distribution and automatic curriculum. The paper curates a set of over 35 applications spanning simple single-app operations to complex multi-app workflows. Tasks are dynamically stratified into four difficulty levels based on the current policy's pass@K success rate:

• Frontier tasks (0–25% SR): tasks the model mostly fails at — these push the boundaries of current capability.
• Exploration tasks (25–50% SR): tasks the model sometimes succeeds on — these drive skill development.
• Near-mastery tasks (50–75% SR): tasks the model approaches proficiency on.
• Exploitation tasks (75–100% SR): tasks the model already performs well on — these reinforce learned behaviors and prevent forgetting.

Building on this stratification, an automatic curriculum progressively adjusts task sampling throughout training. Early stages emphasize simpler tasks to establish foundational skills, while the distribution gradually shifts toward challenging tasks as success rates improve. This adaptive strategy "prevents training collapse from excessive difficulty while ensuring continuous learning signals, effectively addressing the exploration-exploitation tradeoff."

Hybrid verification. The paper develops a hybrid approach to reward computation, tailored to task characteristics:

• Deterministic tasks with clear success criteria use rule-based verifiers with root-level AVD access. For example, a task to "create a new contact named Emilia Gonzalez with number +14240925675" can be verified by directly checking the contacts database on the Android device to confirm that a contact with that name and number exists. This is precise and requires no human judgment.
• Complex tasks where rule-based verification is labor-intensive (e.g., "summarize the key points from the team chat and email them to the manager") use an MLLM-as-a-Judge framework that evaluates execution trajectories against task objectives by examining the final screenshots and action history.

The paper reports that this hybrid approach achieves 83% agreement with human annotations, enabling reliable large-scale verification without manual bottlenecks. The 17% disagreement rate is a limitation, but the paper argues this is acceptable given the scalability benefits.

Training Algorithm and Reward Design. The paper uses a customized version of GRPO (Group Relative Policy Optimization) extended with several features to improve exploration and stability for GUI tasks.

The GRPO objective is:

$J_{\text{GRPO}}(\theta) = \mathbb{E}_{(q) \sim D, \{o_i\}_{i=1}^G \sim \pi_{\theta_{\text{old}}}(\cdot|q)} \left[ \frac{1}{\sum_{c=1}^G |o_c|} \sum_{i=1}^G \sum_{t=1}^{|o_i|} \min \left( r_{i,t}(\theta) \hat{A}_{i,t}, \text{clip}(r_{i,t}(\theta), 1 - \varepsilon_{\text{low}}, 1 + \varepsilon_{\text{high}}) \hat{A}_{i,t} \right) \right]$

where:

• $q$ is a task instruction sampled from the task distribution $D$.
• $\{o_i\}_{i=1}^G$ are $G$ output trajectories (rollouts) sampled from the old policy $\pi_{\theta_{\text{old}}}$ for task $q$. The paper uses $G = 16$.
• $|o_i|$ is the number of tokens in trajectory $i$.
• $r_{i,t}(\theta) = \frac{\pi_\theta(o_{i,t} | q, o_{i,<t})}{\pi_{\theta_{\text{old}}}(o_{i,t} | q, o_{i,<t})}$ is the importance sampling ratio — how much more (or less) likely the new policy is to generate token $t$ of trajectory $i$ compared to the old policy. This corrects for the fact that the trajectories were generated by the old policy, not the current one.
• $\hat{A}_{i,t} = \frac{R_i - \text{mean}(\{R_i\}_{i=1}^G)}{\text{std}(\{R_i\}_{i=1}^G)}$ is the normalized advantage — how much better (or worse) trajectory $i$'s total reward $R_i$ is compared to the average reward of all $G$ trajectories in the group, normalized by the standard deviation.

What it computes: the GRPO objective maximizes the expected reward of the policy, but with two safeguards: (1) the importance sampling correction ensures unbiased gradient estimates even though trajectories come from the old policy, and (2) the clipping operation $\min(r_{i,t} \hat{A}_{i,t}, \text{clip}(r_{i,t}, \dots) \hat{A}_{i,t})$ prevents the policy from changing too much in a single update — if the importance ratio $r_{i,t}$ exceeds the clipping bounds $[1 - \varepsilon_{\text{low}}, 1 + \varepsilon_{\text{high}}]$, the gradient is zeroed for that token, preventing destructive large updates.

Why this form: GRPO is chosen over standard PPO because it eliminates the need for a separately trained value function (critic). In standard PPO, the advantage $\hat{A}_{i,t}$ is computed using a learned value function $V(s_t)$ that estimates expected future reward. Training this value function adds complexity, requires additional memory, and introduces another source of estimation error. GRPO replaces the value function with the group mean: $\text{mean}(\{R_i\})$ serves as the baseline. This works because, with $G = 16$ independent rollouts for the same task, the average reward across the group is a reasonable estimate of the expected reward under the current policy, especially when rewards are binary (success/failure).

Reward design. The reward signal $R_i$ for each trajectory comprises two components:

1. Task completion reward: a binary indicator ($0$ or $1$) of successful execution, determined by either the rule-based verifier or the MLLM-as-a-Judge. This is the primary signal.

2. Action-level repetition penalty: to discourage unproductive looping behavior (where the agent gets stuck repeating the same action or a cycle of actions), the paper penalizes recurring action sequences. The penalty applies to single repeated actions (e.g., clicking the same button 10 times in a row) and to cyclic patterns of 3–5 actions (e.g., repeatedly alternating between two screens without making progress). Actions with identical types but different parameters are not penalized — for example, clicking at $(100, 200)$ and then clicking at $(300, 400)$ are both "click" actions but with different targets, so they are not considered repetition. This enables flexible execution while preventing non-progressive behavioral loops.

Additional algorithmic features:

Clip Higher (asymmetric clipping). Following DAPO (Yu et al., 2025), the paper uses an asymmetric clipping strategy where the upper bound ($\varepsilon_{\text{high}} = 0.3$) is larger than the lower bound ($\varepsilon_{\text{low}} = 0.2$). This means the policy is allowed to increase the probability of successful actions more aggressively than it is allowed to decrease the probability of unsuccessful ones — the narrower lower bound provides stronger protection against destructive updates that would eliminate potentially useful behaviors, while the wider upper bound encourages rapid adoption of successful strategies. This asymmetry is specifically designed to encourage exploration in a domain where most actions in a failed trajectory are still correct (per the fine-grained correctness judgment insight).

Experience Replay. The paper maintains a replay buffer of successful trajectories collected during training. When a rollout group (the set of 16 trajectories sampled for a given task) contains no successful completions, the group is augmented with randomly sampled trajectories from the buffer. The buffer is continuously updated with newly successful experiences, retaining only the most recent 8 trajectories per task to maintain near on-policy learning (older trajectories from earlier policy versions are discarded so the buffer doesn't contain stale data).

This mechanism ensures continuous learning signals even during challenging exploration phases: if the model consistently fails on a difficult task, the group would otherwise contain only failure trajectories with zero reward, producing no gradient signal. By injecting past successful trajectories, the replay buffer provides positive examples that the model can learn from, stabilizing training and accelerating convergence. The "near on-policy" constraint (keeping only 8 most recent per task) ensures the injected trajectories are reasonably representative of what the current policy should be able to achieve.

Experimental Analysis of Online RL. The paper provides several quantitative results on the effectiveness of online RL (Table 9, Figure 12):

• Performance gains across all scales: online RL consistently improves over SFT baselines — +4.0 points for 2B (45.1% → 49.1%), +6.0 points for 8B (64.7% → 70.7%), and +3.5 points for 32B (69.8% → 73.3%). These correspond to relative improvements of 8.9%, 9.3%, and 5.0%.

• Comparison with standard GRPO: standard GRPO applied after SFT yields only +1.8 points on AndroidWorld (64.7% → 66.5%). The enhanced version with data curriculum, repetition penalty, and experience replay achieves +6.0 points (64.7% → 70.7%). This means the algorithmic enhancements contribute +4.2 points beyond what vanilla GRPO provides — the modifications are not minor.

• Effect of interaction budget: extending the maximum environment interaction budget per trajectory from 15 to 30 to 50 steps yields progressive improvements (+1.7, +3.8, and +6.0 points over SFT baseline). A larger budget enables more extensive rollouts and provides richer exploration opportunities — the model can attempt longer tasks and learn from errors that occur later in trajectories.

• Scaling parallel environments: increasing parallel environments from 32 to 512 significantly accelerates learning and improves final performance (65.5% → 70.7%, Figure 12b). Training with fewer environments exhibits early saturation, indicating that limited environments constrain policy improvement by reducing the diversity of experiences the model encounters.

The paper also provides qualitative case studies (Figures 13, 14) showing that RL-trained models handle unexpected permission dialogs and pop-ups that would derail SFT-only models, and can recover from navigating to the wrong application.

3.4.6 Device–Cloud Collaboration System

The final major component is a runtime architecture that enables a small on-device model and a large cloud model to collaborate on task execution, with the goal of getting the privacy/cost/latency benefits of on-device processing while retaining the capability of cloud models for difficult cases.

System Architecture (Figure 7). The system consists of three modules:

Local Agent. The Local Agent runs on the user's device and serves dual roles:

• As a GUI agent: it perceives the current screenshot, generates reasoning, and produces actions to execute the task step by step. This is the primary execution path — most steps are handled locally.
• As a trajectory monitor: it periodically evaluates whether the trajectory so far remains aligned with the user instruction. The monitor checks indicators such as action execution failure (the device reported an error), repeated actions without progress, incorrect inputs (e.g., typing the wrong text based on misunderstanding the instruction), or general task deviations (navigating to the wrong application or screen).

When the monitor detects a deviation and the current context does not contain privacy-sensitive data, it triggers a switch to the Cloud Agent. Critically, the monitor also generates an error summary that explains why the switch was triggered — e.g., "The agent repeatedly attempted to click the 'Login' button without entering a password. The task requires logging into the TaoDian app with user 'mashu' and password [redacted]."

Cloud Agent. The Cloud Agent is a high-capacity model (e.g., MAI-UI-32B or 235B) running on cloud infrastructure. It is invoked only when the monitor detects trajectory deviation. In addition to the standard GUI agent inputs (user instruction, current screenshot, action history), it receives the error summary from the monitor. Given the trajectory history and error summary, the Cloud Agent executes subsequent steps, leveraging its higher reasoning and perception capacity to recover from the deviation and complete the task.

Local Unified Trajectory Memory. On the device, a unified history persists that records the task instruction, historical screenshots, and the model's past outputs (including both thoughts and actions). This memory serves as the single source of truth for both agents: when the Cloud Agent takes over, it receives this complete history, so it has full context for what has already happened. The memory module also projects the unified history into the action spaces expected by the device and cloud models — since these models may have slightly different output formats, the memory normalizes the representations so either model can resume from any state without ambiguity.

Execution Loop. The collaboration proceeds as follows:

1. The user provides a task instruction to the Local Agent.
2. At each step, the Local Agent observes the current screenshot, decides an action, and executes it. The environmental observations and model outputs are written to the Local Unified Trajectory Memory.
3. Every few steps, the Local Agent (in its role as monitor) assesses alignment between the user instruction and the trajectory so far.
4. If alignment is met, the loop continues on device (step 2).
5. If deviation is detected and no sensitive data is involved (checked by an additional privacy detection module), the system packages the trajectory history and error summary and sends them to the Cloud Agent.
6. The Cloud Agent receives the history and error summary, generates the next actions, and sends them back to the device for execution. From this point, execution continues on the cloud model until task completion (or until a decision is made to hand back to the local model — though the paper doesn't discuss the handback direction explicitly).
7. If deviation is detected but sensitive data is present, the system blocks the cloud switch and continues on-device execution.

Training the Local Agent for Monitoring. The Local Agent is trained on two data sources jointly:

1. Standard GUI agent data: the same perception, grounding, and navigation training data used for the SFT stages, ensuring the model retains full GUI agent capability.
2. Monitor data: examples that include alignment reasoning (the model's analysis of whether the trajectory matches the instruction), alignment decisions (a binary or categorical judgment of deviation), and error summaries (the natural-language explanation of what went wrong). This data is constructed by taking trajectories — both successful and failed — from the data pipeline and annotating them with monitor-style judgments.

This multi-task training recipe teaches the on-device model to execute and monitor simultaneously without requiring separate models or fragile prompt engineering. The paper contrasts this with prior work (e.g., LightAgent, Jiang & Huang, 2025) which relied on prompt engineering for monitoring — the paper argues this is insufficient because "the monitor must handle varied and complex cases, and prompt engineering alone is unlikely to deliver reliable monitoring."

Experimental Results for Device–Cloud Collaboration (Figure 10). The paper evaluates the system on AndroidWorld using MAI-UI-2B as the local agent and MAI-UI-32B as the cloud agent:

• Performance improvement: the device–cloud collaboration (DCC) system achieves a relative improvement of 33.4% over the on-device model alone (from 49.1% to 65.5% on AndroidWorld). This means that even weak on-device models can be made practically useful through cloud augmentation.
• Comparison with random switching: the DCC system surpasses a random switch (RS) baseline — which calls the cloud model at the same frequency but without monitor-guided decisions about when to switch — by 7.7 percentage points. This demonstrates that the monitor is making intelligent decisions about when escalation is needed, not just adding cloud compute indiscriminately.
• Efficiency gains: compared to cloud-only serving, the DCC system reduces cloud model calls by 42.7%, substantially lowering serving cost and latency. Over 40% of tasks are completed entirely on-device, meaning the cloud model is never invoked for those tasks.
• Impact of error summaries: an ablation that removes the error summary at switching time shows a performance drop of 6.9 points (from 65.5% to 58.6%), demonstrating that the monitor-generated error summary is crucial for the cloud agent's trajectory recovery process — the cloud model needs to know why the task went off-track to effectively correct course.
• Comparison with larger pure-cloud models: the DCC system (2B local + 32B cloud) achieves 65.5%, surpassing several larger pure-cloud baselines including UI-Tars-1.5 (57.8%), Seed1.5-VL (62.1%), and Qwen3-VL-32B (57.3%). This demonstrates that intelligent collaboration can outperform simply using a larger model for everything.

Privacy Preservation (Figure 11). The paper presents a pilot case study showing how the collaboration system protects sensitive data. In a task requiring the agent to log into an app with a password:

1. The Local Agent initially navigates correctly but then deviates — it repeatedly taps the "Login" button without first entering the password (steps 2–4 in Figure 11).
2. The trajectory monitor detects the misalignment and proposes switching to the cloud agent.
3. However, the privacy detection module (an additional component introduced for this case study) detects that the current screen contains password credentials and blocks the cloud switch.
4. The Local Agent is forced to continue on-device execution, and it ultimately corrects its own trajectory — it issues an ask_user action to request the password, receives the user's response, enters the password, and completes the login.

Crucially, no privacy-sensitive content (the password) is ever transmitted to the cloud. This demonstrates that the system can respect privacy constraints even when the on-device model is struggling — the monitor identifies the problem, but the privacy module prevents the naive solution of sending everything to the cloud.

4. Key Insights and Innovations

Innovation 1: Real-World Deployment as a First-Class Design Constraint Rather Than an Afterthought

The paper's most distinctive intellectual move is not any single algorithmic contribution but its reframing of what it means to "solve" GUI agents. Prior work in this space has overwhelmingly optimized for benchmark performance on idealized tasks — static screenshots with unambiguous instructions, no pop-ups, no missing information, no privacy constraints. The implicit assumption has been that if you can get a model to score well on ScreenSpot or AndroidWorld, you've made progress toward practical utility. MAI-UI rejects this framing entirely and instead treats the real-world deployment requirements themselves as the primary design specification, with benchmark performance as a downstream consequence.

This is a fundamental shift, not an incremental refinement. To see why, consider the typical GUI agent paper's structure: propose a new grounding method or navigation strategy, evaluate on standard benchmarks, claim improvement. The deployment barriers — user ambiguity, UI brittleness from long action sequences, privacy concerns, dynamic environment robustness — are either ignored entirely or mentioned as "future work." MAI-UI inverts this: it identifies four specific deployment blockers (Section 1) and then designs the training pipeline, action space, and runtime architecture to address all four simultaneously. The benchmark results (73.5% on ScreenSpot-Pro, 76.7% on AndroidWorld) are presented as validation of this deployment-first approach, not as the primary contribution.

This reframing matters because it changes the objective function of GUI agent research. Under the benchmark-optimization paradigm, the rational allocation of effort is to push scores higher on existing evaluations. Under the deployment-requirement paradigm, the rational allocation is to identify capability gaps that prevent real-world use and design training and systems to fill those gaps — some of which (like MCP-augmented tasks on MobileWorld, where prior end-to-end models score 5.4%) are not even measured by standard benchmarks. Table 8 makes this concrete: previous state-of-the-art end-to-end models achieve 5.5% (GUI-Owl-32B) and 10.4% (UI-Venus-72B) on MobileWorld overall, while MAI-UI-235B reaches 41.7%. The gap between AndroidWorld performance (where prior models do well) and MobileWorld performance (where they collapse) exposes the inadequacy of benchmark-optimization as a research strategy.

The device–cloud collaboration system (Section 2.4, Figure 10) is the purest expression of this philosophy. Rather than asking "how can we make the on-device model better at AndroidWorld?" the paper asks "what architecture would give users the best combination of privacy, cost, latency, and capability in real usage?" The answer — a local agent that monitors its own progress and escalates to cloud only when necessary — is not an optimization of any existing metric; it's a system design derived from deployment constraints that happens to also improve measured performance (33% relative gain for the on-device model through collaboration).

Innovation 2: The Instruction-as-Reasoning Paradigm as a Solution to SFT+RL Policy Collapse

The paper identifies and solves a specific failure mode in GUI grounding that prior work had documented but not adequately addressed: policy collapse in the SFT+RL training paradigm. The problem is worth understanding because it reveals a deeper tension in how vision-language models learn structured outputs.

Prior work established two competing approaches to GUI grounding: pure SFT (JEDI, OS-Atlas, Aguvis, Uground) which is stable but plateaus in performance, and pure RL (GUI-R1, GUI-Actor, GTA1) which achieves higher scores but requires careful training recipes to avoid degeneracy. The obvious synthesis — SFT followed by RL — should combine the stability of SFT with the performance of RL, but in practice it often produces policy collapse: the model forgets how to output valid coordinates, degenerates to predicting the screen center, or produces unparseable responses. Phi-Ground (Zhang et al., 2025a) explicitly documented this problem but did not fully resolve it; the field lacked a principled understanding of why SFT+RL collapses and how to prevent it.

MAI-UI's Instruction-as-Reasoning approach (from the authors' prior UI-Ins work) provides a diagnosis and a solution. The diagnosis is that SFT with coordinate-only supervision teaches the model a brittle input-output mapping: given an instruction and screenshot, output (x, y). There is no intermediate structure that the RL phase can explore or modify — the only "knob" is the coordinate prediction itself, and RL perturbations to this mapping easily break the formatting constraints that SFT carefully taught (outputs must be parseable coordinates in valid XML tags). The solution is to make the SFT phase teach not just what to output but how to reason about the output: the model learns to generate diverse reasoning pathways (appearance, function, location, intent) before predicting coordinates. This creates a richer, more structured output distribution where RL exploration operates on the selection and composition of reasoning strategies rather than on raw coordinate perturbation.

The significance of this innovation extends beyond GUI grounding. The tension between SFT stability and RL exploration is a general problem in post-training vision-language models. The Instruction-as-Reasoning approach suggests a general principle: SFT should teach structured intermediate outputs that serve as exploration scaffolds for subsequent RL, rather than teaching only the final output format. The reasoning pathways are not merely a performance booster; they are a stabilization mechanism that makes the RL optimization landscape more benign by providing valid exploration directions that preserve output format constraints.

The evidence for this claim comes from the paper's grounding results across multiple benchmarks. On UI-Vision (Table 3), MAI-UI-32B achieves 47.1% without zoom-in, compared to 36.8% for UI-Venus-72B (the previous best) — a +10.3 point gain that cannot be explained by model scale alone since the baseline has more than twice the parameters. On MMBench-GUI L2 (Table 4), the improvement is even more dramatic: 91.3% for MAI-UI-32B versus 85.3% for Qwen3-VL-32B (the base model without MAI-UI training) — a +6.0 point gain from the same architecture. These consistent improvements across diverse grounding scenarios (professional software, desktop apps, web interfaces, mobile) suggest that the Instruction-as-Reasoning approach provides a general training recipe, not just a benchmark-specific trick.

The paper also reports an emergent capability that the training paradigm was not explicitly designed to produce: the model "can strategically select appropriate reasoning perspectives given different contexts and compose multiple perspectives into a cohesive one" and "can also generate novel analytical angles beyond the four trained perspectives" (Section 3.6). This emergent generalization suggests that the SFT phase taught a generalizable reasoning skill rather than merely four memorized templates.

Innovation 3: Scaling Online RL for GUI Agents Through Infrastructure, Not Just Algorithms

The paper's third major contribution is showing that online RL for GUI agents works dramatically better at scale, and building the infrastructure to achieve that scale. Prior work on GUI agent RL (e.g., UI-Tars-2, AgentCPM-GUI) operated with tens of parallel environments and limited interaction budgets — not because smaller scale was optimal, but because scaling GUI environments is genuinely hard. Each environment is a full Android emulator consuming gigabytes of memory, and synchronizing hundreds of them across distributed infrastructure while maintaining consistent task initialization is a systems engineering challenge that the RL literature rarely addresses in papers.

MAI-UI's key finding is that scale is not just helpful for GUI agent RL — it may be necessary to realize most of the benefit. Figure 12b shows that increasing parallel environments from 32 to 512 yields +5.2 points on AndroidWorld, and the learning curves for lower environment counts "exhibit early saturation, indicating that limited environments constrain policy improvement." Table 9 shows that extending the maximum environment step budget from 15 to 50 yields +4.3 points, and that the enhanced training recipe (curriculum, repetition penalty, experience replay) adds +4.2 points over standard GRPO. Together, these scaling factors account for most of the +6.0 point RL gain for the 8B model — meaning that if you simply applied standard GRPO with 32 environments and 15-step budgets, you would get only +1.8 points, barely distinguishable from noise.

This finding has significant implications for the field. It suggests that prior negative or modest results from GUI agent RL may have been limited by infrastructure, not algorithmic design. The paper's contribution is not a novel RL algorithm — GRPO is established, the asymmetric clipping is from DAPO, the replay buffer is standard — but rather the demonstration that these known techniques, when applied at sufficient scale, produce substantial gains that previous work missed. The infrastructure contribution (containerized Android environments, centralized environment manager with fault tolerance, asynchronous rollout for GPU efficiency, hybrid parallelism for long trajectories) is described in enough detail (Section 2.3.3) to serve as a blueprint for other groups attempting GUI agent RL.

The conceptual insight is that GUI environments are analogous to the massively parallelized simulators that enabled breakthroughs in other RL domains (Atari, MuJoCo, Minecraft). The difference is that GUI emulators are stateful and resource-intensive rather than fast and lightweight, so the engineering challenge is different — it's about efficient container orchestration rather than fast step execution — but the scaling principle is the same: diverse experiences from many parallel environments prevent overfitting to specific environment dynamics and enable learning of robust strategies that generalize.

A subtle but important corollary: the paper's automatic curriculum (which stratifies tasks by the current policy's success rate and progressively shifts toward harder tasks) addresses a specific failure mode of large-scale RL that is easy to overlook. With 512 parallel environments generating thousands of trajectories per training step, if all tasks are uniformly sampled, the model will see overwhelmingly many examples of tasks it cannot complete (producing zero-reward trajectories with no learning signal) and tasks it has already mastered (producing saturated learning signals). The curriculum ensures that the task distribution remains in the "learning zone" where rewards are informative, which is critical for making the scale actually translate to improved performance rather than wasted compute.

Innovation 4: Fine-Grained Correctness Judgment as a Data Efficiency Breakthrough for Trajectory Learning

The paper's most important data-related innovation is the fine-grained correctness judgment approach in the trajectory synthesis pipeline (Section 2.3.1), which addresses a fundamental inefficiency in how GUI agent training data is typically generated.

The standard approach to building navigation datasets for GUI agents is to collect full execution trajectories and either keep them (if successful) or discard them (if failed). This creates a severe data efficiency problem: model-based trajectory generation (which is the only scalable approach, since human annotation is expensive) produces high failure rates, especially for complex tasks and early in the iterative training process. If you discard all failed trajectories, you throw away the majority of generated data; if you train on them, you teach the model to make mistakes.

The paper's insight is that failed trajectories are not uniformly wrong — they typically contain a substantial prefix of correct actions before the first error. The fine-grained judgment module (an MLLM-as-a-judge that evaluates correctness at each step) identifies the longest correct prefix and retains it as training data while discarding only the erroneous suffix. This transforms what would be wasted compute (generating trajectories that are entirely discarded) into productive training data generation, because even failed rollouts contribute partial-trajectory examples.

This matters for two reasons beyond the obvious data efficiency gain. First, it changes the economics of iterative self-improvement. In the iterative rejection sampling loop (Phase 3, Section 2.3.1), the model generates trajectories on a large set of tasks, and only high-quality examples are retained for the next training round. Without fine-grained judgment, this loop would produce very little training data per iteration because most generated trajectories fail. The partial-trajectory reuse makes the loop viable by dramatically increasing the yield per generation cycle.

Second, it provides a form of curriculum learning that emerges naturally from the data rather than being explicitly designed. Partial trajectories from failed attempts are inherently "simpler" than full trajectories — they consist of the early, typically easier steps of tasks before the model encounters the difficult decision that causes failure. Training on these partial trajectories naturally exposes the model to progressively more complex action sequences as its capability improves and its rollouts succeed for longer before failing.

The paper does not provide an explicit ablation quantifying how much fine-grained judgment contributes relative to discarding all failed trajectories, but the qualitative case is strong. The practical implication is that trajectory-level binary success/failure judgments are information-poor compared to step-level correctness judgments, and that investing in fine-grained evaluation — whether through MLLM judges or other means — is a high-leverage way to improve data efficiency in GUI agent training. This principle likely generalizes beyond GUI agents to any sequential decision-making domain where partial success is common and correctness can be evaluated at intermediate steps.

Innovation 5: The Device–Cloud Collaboration Architecture as a New Deployment Paradigm for AI Agents

The paper's device–cloud collaboration system (Section 2.4) introduces a deployment architecture that redefines the relationship between local and cloud models. Prior approaches treat these as mutually exclusive options — you either deploy a small model on-device (accepting capability limits for privacy and cost benefits) or you use a large cloud model (accepting privacy and cost costs for capability). MAI-UI proposes a third option: runtime collaboration with intelligent handoff based on task state.

What makes this genuinely novel rather than an obvious engineering optimization is the monitor-guided switching mechanism. A naive collaboration system might switch based on simple heuristics (task complexity estimates, predefined rules about which app categories require cloud assistance). MAI-UI's monitor is qualitatively different: it evaluates whether the current trajectory remains aligned with the user's intent, detecting not just task difficulty but execution quality — whether the agent is stuck in a loop, has navigated to the wrong screen, or has made an incorrect input. This means the system escalates not when tasks look hard but when the local agent is actually failing, which is a more precise and efficient criterion.

The error summary generation is the conceptual linchpin. Without it, a cloud handoff provides the cloud model with the same inputs the local model had (screenshots, action history) but no additional information about what went wrong. The cloud model then needs to independently diagnose the problem before recovering. The error summary provides this diagnosis, making the handoff more like a targeted intervention than a restart. The ablation showing a +6.9 point gain from including error summaries (Figure 10, left) demonstrates that this is not a minor feature — it accounts for a substantial fraction of the collaboration benefit.

The privacy protection case study (Figure 11) adds an important dimension: the monitor-plus-privacy-module combination creates a runtime system that can detect both task failure and privacy risk and make opposing recommendations (escalate to cloud vs. block escalation), resolving the tension through a priority rule (privacy trumps capability). This is a concrete instantiation of a principle that the broader AI deployment literature has discussed abstractly — that deployment systems need to manage multiple, potentially conflicting constraints — but rarely demonstrated in an end-to-end GUI agent system.

The broader significance is that this architecture suggests a general template for deploying AI agents in sensitive contexts: pair a capable-but-limited local model with a powerful cloud model, train the local model to monitor its own execution quality, and design the handoff protocol to include diagnostic context rather than just raw state. This template applies beyond GUI agents to any domain where tasks have variable difficulty and data has variable sensitivity — code generation, document processing, medical decision support — making it a potentially reusable architectural pattern rather than a domain-specific hack.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper evaluates on a broad spectrum of benchmarks spanning three categories: (1) Grounding: ScreenSpot-Pro (professional high-resolution layouts), UI-Vision (diverse apps with spatial/functional/reasoning queries), MMBench-GUI L2 (hierarchical-instruction following across Windows, MacOS, Linux, iOS, Android, Web), OSWorld-G and OSWorld-G-Refine (complex desktop scenarios with text matching, element recognition, layout understanding, and fine-grained manipulation), and ScreenSpot-V2 (mobile, desktop, and web interfaces); (2) Mobile-use navigation (offline): Android Control (planning and action execution) and GUI Odyssey (cross-app navigation); (3) Mobile-use navigation (online): AndroidWorld (116 tasks across 20 apps in a live Android emulator) and MOBILEWORLD (201 tasks across 15+ apps, including agent–user interaction and MCP tool use subsets). All grounding benchmarks use standard released test splits; AndroidWorld and MobileWorld provide live evaluation environments rather than static test sets.

• Base model(s). All MAI-UI variants use Qwen3-VL (Bai et al., 2025a) as the vision-language backbone, spanning 2B, 8B, 32B, and 235B-A22B (MoE with 22B active) parameters. The paper argues Qwen3-VL is a strong representative foundation model for multimodal tasks, and the full spectrum of sizes enables deployment across on-device and cloud scenarios while isolating the contribution of the MAI-UI training methodology (same architecture, different data and training procedures).

• Metrics. The primary metrics are: (a) Point-in-box accuracy for grounding benchmarks — a prediction is correct if the predicted (x, y) coordinate falls within the ground-truth bounding box of the target element (Equation 1). Results are averaged across all test examples and reported as percentages. (b) Success rate (SR) for online navigation benchmarks — a trajectory is successful if the agent completes the task before exhausting the step budget, verified by either rule-based checkers with backend access or MLLM-as-a-Judge evaluation. (c) Exact match accuracy for offline navigation benchmarks (Android Control). (d) Success rate per capability subset for MobileWorld — breaking out GUI-only tasks, user-interaction tasks, and MCP-tool tasks separately.

• Baselines. The paper compares against an extensive set of models at comparable scales. For grounding: GPT-4o (OpenAI, 2024), Claude (various versions), Gemini-3-Pro (DeepMind, 2025b), Seed1.8 (Bytedance Seed, 2025a), Qwen3-VL baselines at 2B/8B/32B (Bai et al., 2025a), InfiGUI-3B/7B (Liu et al., 2025b), Ferret-UI Lite (Yang et al., 2025b), UI-TARS-7B/72B (Qin et al., 2025a), Phi-Ground (Zhang et al., 2025a), GUI-Actor-7B (Wu et al., 2025), SE-GUI-7B (Yuan et al., 2025), GUI-G2-7B (Tang et al., 2025), OpenCUA-7B/32B (Wang et al., 2025b), GTA1-7B/32B (Yang et al., 2025a), UI-Venus-7B/72B (Gu et al., 2025c), GUI-Owl-7B/32B (Ye et al., 2025), UGround-v1-7B/72B (Gou et al., 2025), InternVL3-78B (Zhu et al., 2025), SeeClick (Cheng et al., 2024), and OS-Atlas-7B (Wu et al., 2024). For online navigation: Qwen3-VL at various scales, ScaleCUA-3B (Liu et al., 2025e), Ferret-UI Lite-3B, UI-Tars-7B/72B/1.5/2, UI-Venus-7B/72B, GUI-Owl-7B, Step-GUI-8B (Yan et al., 2025), Seed1.5-VL/Seed1.8, Gemini-2.5-Pro (DeepMind, 2025a), and Doubao-1.5-UI-TARS. For MobileWorld, the paper also compares against agentic frameworks using Claude-4.5-Sonnet, Gemini-3-Pro, or GPT-5 as planners with UI-Ins-7B (Chen et al., 2025) as executor.

• Generation budget / compute accounting. For grounding benchmarks, models make a single prediction per instruction (no budget scaling). For online navigation, the primary compute metric is the number of environment steps (actions executed before termination or timeout) and the number of parallel environments used during RL training (32–512). The paper does not directly compare inference FLOPs across models of different scales; instead, it groups results by model size and compares within comparable parameter counts. The zoom-in strategy for grounding uses 2 inference passes (coarse + fine) and is reported separately.

• Cross-validation / statistical protocol. The paper uses two-fold cross-validation for compute-optimal strategy selection on the navigation data pipeline (Section 2.3.1), ensuring that strategy selection and evaluation use disjoint data. For grounding, standard test set evaluation is used without cross-validation. For online navigation benchmarks (AndroidWorld, MobileWorld), each task is evaluated once per model with a fixed step budget, and success rate is computed as the fraction of completed tasks. The paper does not report confidence intervals or statistical significance tests for the benchmark comparisons, which is a limitation discussed below.

Main Quantitative Results

Grounding Performance

Headline results. MAI-UI establishes new state-of-the-art across all five grounding benchmarks at every model scale. On ScreenSpot-Pro (Table 2), MAI-UI-32B achieves 67.9% average accuracy, a +4.1 point absolute gain over the previous best GTA1-32B (63.6%). With the zoom-in strategy, MAI-UI-32B reaches 73.5%, surpassing Gemini-3-Pro (72.7%) and Seed1.8 (73.1%). At the 8B scale, MAI-UI-8B achieves 65.8% (70.9% with zoom-in), outperforming all 7B baselines including GUI-Owl-7B (54.9%) by +10.9 points. At the 2B scale, MAI-UI-2B achieves 57.4% (62.8% with zoom-in), which notably exceeds larger models like GUI-Owl-32B (58.0%) and UI-Venus-72B (61.9%).

UI-Vision (Table 3). MAI-UI-32B achieves 47.1% accuracy (49.2% with zoom-in), a +12.4 point absolute gain over UI-Venus-72B (36.8%), the previous best open-source model. The improvement is largest on the Functional category (+14.8 points: 57.1% vs. 42.3%) and spatial reasoning (+3.2 points: 26.9% vs. 23.7%). The 8B variant (40.7%, 42.4% with zoom-in) outperforms all prior models regardless of size except UI-Venus-72B. Notably, proprietary models perform poorly on this benchmark: GPT-4o achieves only 1.4% and Claude-3.7-Sonnet achieves 8.3%.

MMBench-GUI L2 (Table 4). MAI-UI-32B achieves 91.3% average accuracy, a +7.9 point gain over GTA1-32B (83.4% — note: the paper states "+7.9 points over the prior best GTA1" but Table 4 shows 91.3% vs. 83.4%, a +7.9 point gap). The improvement is particularly strong on the Advanced (high-level, goal-oriented) subtasks across all platforms: for example, on Android-Advanced, MAI-UI-32B achieves 93.2% vs. 88.5% for Qwen3-VL-32B (the base model) and 87.0% for GUI-Owl-32B. At the 8B scale, MAI-UI-8B (88.8%) outperforms all baselines including much larger models like InternVL3-78B (72.2%) and UI-TARS-DPO-72B (74.3%). The 2B variant (82.6%) surpasses several 7B baselines including GUI-Actor-7B (76.5%) and GTA1-7B (78.5%).

OSWorld-G and OSWorld-G-Refine (Tables 5, 11). On OSWorld-G, MAI-UI-32B achieves 67.6% (70.9% with zoom-in), exceeding UI-Venus-72B (70.4%) and GTA1-32B (65.2%). The improvements are balanced across categories: Text Matching 78.5%, Element Recognition 75.2%, Layout Understanding 78.3%, and Fine-grained Manipulation 62.4% (all with zoom-in). On OSWorld-G-Refine (Table 11), which reduces instruction ambiguity, MAI-UI-32B reaches 73.9% (75.0% with zoom-in), exceeding GTA1-32B (72.2%) and OpenCUA-32B (70.2%). At the 8B scale on OSWorld-G-Refine, MAI-UI-8B with zoom-in (72.9%) surpasses the 32B open-source baseline GTA1-32B (72.2%), demonstrating cross-scale capability gains.

ScreenSpot-V2 (Table 10). MAI-UI-32B achieves 96.5% average accuracy, setting a new state-of-the-art. Performance is strong across all domains: Mobile-Text 99.0%, Desktop-Text 99.5%, Web-Text 97.4%, Mobile-Icon 92.9%, Desktop-Icon 93.6%, Web-Icon 94.6%. The 8B variant (95.2%) outperforms UI-Venus-72B (95.3% — roughly tied) and GTA1-32B (95.2% — tied). The 2B variant (92.5%) outperforms many 7B models including GUI-Owl-7B (92.8% — slightly behind) and OpenCUA-7B (92.3%).

Mobile-Use Navigation: Offline Benchmarks

Headline results. MAI-UI produces competitive results on Android Control (high-level instructions) and sets a new state-of-the-art on GUI Odyssey. As shown in Table 7:

• Android Control (high-level): MAI-UI-32B achieves 75.5% exact match, competitive with UI-Venus-72B (77.2%) and exceeding OS-Atlas-7B (56.5%) and Aguvis-7B (54.2%). The 8B variant (69.1%) is competitive with AgentCPM-GUI-8B (69.2%). The 2B variant achieves 67.3%, outperforming several larger models including OS-Atlas-7B (56.5%).

• GUI Odyssey: MAI-UI-32B achieves 83.4%, substantially outperforming the previous best OdysseyAgent-7B (73.7%) by +9.7 points, UI-Venus-72B (72.4%) by +11.0 points, and Aguvis-7B (13.5%) by a massive margin. The 8B variant (80.1%) also exceeds all prior methods. The 2B variant achieves 72.6%, competitive with much larger models.

Mobile-Use Navigation: Online Benchmarks

AndroidWorld (Table 6). MAI-UI establishes new state-of-the-art across all model scales:

• MAI-UI-235B-A22B: 76.7% success rate, surpassing UI-Tars-2 (73.3%) by +3.4 points, Gemini-2.5-Pro (69.7%) by +7.0 points, and Seed1.8 (70.7%) by +6.0 points. This is the highest reported result on AndroidWorld for an end-to-end model.

• MAI-UI-32B: 73.3%, matching UI-Tars-2 (73.3%) at the 230B scale and surpassing larger baselines including UI-Venus-72B (65.9%) by +7.4 points, Gemini-2.5-Pro (69.7%) by +3.6 points, and UI-TARS-1.5 (64.2%) by +9.1 points.

• MAI-UI-8B: 70.7%, exceeding Step-GUI-8B (67.7%) by +3.0 points and GUI-Owl-7B (66.4%) by +4.3 points. This surpasses even the 235B Qwen3-VL base model (63.7%) and Seed1.5-VL (62.1%).

• MAI-UI-2B: 49.1%, a +21.1 point absolute gain over the strongest on-device baseline Ferret-UI Lite-3B (28.0%) — a 75.4% relative improvement. It also outperforms ScaleCUA-3B (23.7%), UI-Tars-7B (33.0%), and UI-Tars-1.5-7B (30.0%), despite having fewer parameters than all of these.

The critical pattern is that MAI-UI models at smaller scales consistently outperform much larger baseline models: the 8B variant exceeds multiple 72B+ baselines, and the 2B variant surpasses several 7B models. This suggests the training methodology (self-evolving data pipeline + online RL) provides efficiency gains beyond what parameter scaling alone achieves.

MobileWorld (Table 8). This is the most challenging and realistic benchmark, including tasks requiring agent–user interaction and MCP tool use:

• Overall: MAI-UI-235B-A22B achieves 41.7% success rate, MAI-UI-32B attains 37.3%, and MAI-UI-8B scores 24.9%. These represent gains of +20.8, +16.4, and +4.0 points over the best end-to-end baseline (Doubao-1.5-UI-TARS, 20.9%). Prior end-to-end models performed very poorly: GUI-Owl-32B (5.5%), UI-Venus-72B (10.4%), Qwen3-VL-235B-A22B base model (9.5%). The MAI-UI-235B-A22B result is competitive with agentic frameworks that use GPT-5 (51.7%) or Gemini-3-Pro (46.3%) as planners with an external grounding model — despite MAI-UI being a single end-to-end model.

• User-interaction subset (45 tasks): MAI-UI-235B-A22B achieves 51.1%, surpassing the best end-to-end baseline (Doubao-1.5-UI-TARS, 32.4%) by +18.7 points, and notably exceeding the Gemini-3-Pro + UI-Ins-7B agentic framework (24.4%) by +26.7 points.

• MCP subset (40 tasks): MAI-UI-235B-A22B achieves 37.5%, compared to only 5.4% for the best prior end-to-end model (Doubao-1.5-UI-TARS scored on this subset implies 5.4% from the overall metric; Qwen3-VL-235B base scores 5.4%). This +32.1 point gain directly reflects the paper's MCP augmentation training.

• GUI-only subset (116 tasks): MAI-UI-235B-A22B achieves 39.7%, outperforming all end-to-end baselines.

The pattern across difficulty levels is stark: the gap between MAI-UI and prior end-to-end models is largest on the most challenging, real-world-oriented capabilities (user interaction and MCP), precisely the areas where the paper's proposed methodology makes novel contributions. On the GUI-only subset (which more closely resembles AndroidWorld-style tasks), the improvements are smaller but still substantial.

Online RL Performance Analysis

SFT vs. RL comparison (Table 9). Online RL consistently improves over SFT-only models across all scales:

• 2B: 45.1% → 49.1% (+4.0 points, +8.9% relative)
• 8B: 64.7% → 70.7% (+6.0 points, +9.3% relative)
• 32B: 69.8% → 73.3% (+3.5 points, +5.0% relative)

The gains are robust across model sizes, though proportionally smaller for the largest model (which already achieves high SFT-only performance, leaving less room for improvement on the 116-task AndroidWorld benchmark).

Algorithm ablations (Table 9). On the 8B model:

• Standard GRPO (max_env_steps=50): 64.7% → 66.5% (+1.8 points)
• MAI-UI's enhanced GRPO (max_env_steps=15): 64.7% → 66.4% (+1.7 points)
• Enhanced GRPO (max_env_steps=30): 64.7% → 68.5% (+3.8 points)
• Enhanced GRPO (max_env_steps=50): 64.7% → 70.7% (+6.0 points)

The key finding: at equal step budgets (50), the enhanced GRPO with data curriculum, repetition penalty, and experience replay achieves +6.0 points, while standard GRPO achieves only +1.8 — the algorithmic enhancements contribute +4.2 points beyond what vanilla GRPO provides. The interaction budget is also critical: extending from 15 to 50 steps yields +4.3 points, suggesting that deep exploration of long-horizon tasks is essential for effective RL.

Scaling parallel environments (Figure 12b). Increasing parallel environments from 32 to 512 significantly accelerates learning and improves final AndroidWorld performance: 65.5% → 70.7% (+5.2 points). The curves for 32 and 64 environments show early saturation, while 128 and 512 environments continue improving. This provides evidence that exploration diversity — experiencing many different environment states, pop-ups, and edge cases — is a bottleneck for GUI agent RL, and that scaling infrastructure to support hundreds of parallel environments is necessary to realize the full benefit.

Training reward trend (Figure 12a). The reward metric (EMA with span=6) steadily increases throughout training from approximately 0.60 to 0.80, demonstrating that the model progressively improves at completing tasks in the training environment.

Device–Cloud Collaboration Results

Performance improvement (Figure 10, left). The device–cloud collaboration (DCC) system using MAI-UI-2B (local) + MAI-UI-32B (cloud) achieves 65.5% on AndroidWorld. This represents:

• +16.4 points over the local agent alone (49.1%), a 33.4% relative improvement.
• +7.7 points over the random switch baseline (57.8%), which calls cloud at the same frequency but without monitor-guided switching.
• Competitive or superior performance to several pure-cloud baselines: UI-Tars-1.5 (57.8%), Qwen3-VL-32B (57.3%), Seed1.5-VL (62.1%).

Efficiency gains (Figure 10, right). The DCC system executes 42.7% of steps locally on-device (reducing cloud API calls by the same proportion compared to cloud-only serving). 40.5% of tasks are completed entirely on-device with zero cloud calls. This translates to substantial cost savings, reduced latency (since local inference is faster than cloud API round-trips), and improved privacy (fewer screenshots transmitted).

Error summary ablation (Figure 10, left). Removing the monitor-generated error summary at switching time (DCC w/o ES) reduces performance to 58.6%, a -6.9 point drop from the full DCC system (65.5%). This demonstrates that the error summary is not a minor feature — it provides critical diagnostic context that enables the cloud model to effectively recover from trajectory deviations.

Ablation Studies and Robustness Checks

Asymmetric clipping + experience replay vs. standard GRPO: Enhanced GRPO (with data curriculum, repetition penalty, experience replay, and asymmetric clipping ε_low=0.2, ε_high=0.3) achieves +6.0 points over SFT baseline (64.7% → 70.7%), while standard GRPO achieves +1.8 points (64.7% → 66.5%) at the same step budget of 50. The +4.2 point difference demonstrates that these algorithmic modifications are essential, not incidental (Table 9).

Environment step budget: Increasing max_env_steps from 15 to 30 to 50 yields progressive gains: +1.7, +3.8, and +6.0 points respectively (Table 9). The non-linear scaling (the jump from 30 to 50 is larger than from 15 to 30) suggests that longer-horizon tasks provide disproportionately valuable learning signals, likely because they expose the model to more diverse failure modes and recovery opportunities.

Image resolution for online RL: The paper reports (Section 3.5 text) that 720p resolution achieves performance comparable to 1080p while providing ~50.1% speedup per step during training. Conversely, 540p resolution substantially degrades model performance due to insufficient detail for fine-grained UI element perception. This suggests a resolution threshold effect: below ~720p, visual information loss harms grounding accuracy; above ~720p, the additional pixels provide diminishing returns for UI tasks.

Zoom-in strategy for grounding: Across benchmarks, the two-pass zoom-in strategy consistently improves performance: ScreenSpot-Pro (+5.6 points for 32B: 67.9% → 73.5%), UI-Vision (+2.1 points for 32B: 47.1% → 49.2%), OSWorld-G (+3.3 points for 32B: 67.6% → 70.9%), OSWorld-G-Refine (+1.1 points for 32B: 73.9% → 75.0%). The gains are larger on benchmarks with high-resolution professional software (ScreenSpot-Pro) and smaller on benchmarks already at high accuracy (OSWorld-G-Refine, where ceiling effects may apply). The zoom-in strategy is a test-time only intervention, so it represents inference-time compute that the baseline models could also apply, making this a somewhat asymmetric comparison.

Cloud agent error summary (Figure 10): As discussed above, removing the error summary at device–cloud handoff reduces success by -6.9 points on AndroidWorld, demonstrating the critical role of diagnostic context in trajectory recovery.

Number of parallel environments (Figure 12b): Increasing from 32 to 512 parallel environments yields +5.2 points on AndroidWorld for the 8B model. The curve shows diminishing returns between 128 and 512 environments (the gap is smaller than between 32 and 128), suggesting that the benefits of further scaling beyond 512 may be limited without improvements in other aspects of the training pipeline.

Instruction-as-Reasoning for grounding (Section 3.6): While not presented as a formal ablation table, the text describes that the reasoning-perspective approach mitigates policy collapse that occurs in pure RL grounding paradigms. The effectiveness is demonstrated indirectly through the consistent grounding improvements across all five benchmarks at all model scales, but the paper does not include a direct ablation comparing coordinate-only SFT+RL vs. instruction-as-reasoning SFT+RL. This is a missing experiment that would have strengthened the claim.

PRM aggregation strategy and PRM vs. ORM: These ablations are absent from the current paper. The paper does not use a Process Reward Model or Outcome Reward Model for navigation — it uses task-completion rewards from rule-based verifiers or MLLM-as-a-Judge. This is a design choice, not a missing ablation, but it means the paper cannot compare against approaches that use learned verifiers for reward shaping.

Critical Assessment

Claim 1: "MAI-UI establishes new state-of-the-art across GUI grounding and mobile navigation." The grounding claim is strongly supported by the experimental results. On ScreenSpot-Pro, UI-Vision, MMBench-GUI L2, and OSWorld-G, MAI-UI models at each scale consistently outperform all comparable baselines, often by substantial margins (+4 to +12 points). The inclusion of proprietary models (Gemini-3-Pro, Seed1.8) as baselines and the demonstration that open-source MAI-UI models surpass them on select benchmarks (ScreenSpot-Pro: 73.5% vs. 73.1% for Seed1.8) strengthens credibility.

However, there are caveats:

• The zoom-in strategy creates an asymmetric advantage: MAI-UI with zoom-in is compared against baseline models (including proprietary ones) that may not use the same inference-time augmentation. The paper does not report zoom-in results for baselines, so it's unclear whether the gains are from better grounding or from the inference-time trick. The non-zoom-in results are still state-of-the-art in most cases, but the margins are narrower (e.g., ScreenSpot-Pro 67.9% vs. GTA1-32B 63.6% — a +4.3 point lead, not +9.9).

• The strongest baseline (GTA1-32B) is recent and may use similar techniques: GTA1 (Yang et al., 2025a) also targets GUI test-time scaling, so the comparison is between two contemporaneous systems with potentially overlapping methodologies. The paper does not analyze why MAI-UI outperforms GTA1 — is it the data, the RL recipe, or the instruction-as-reasoning SFT?

• Single model family: all MAI-UI variants use Qwen3-VL as the backbone. The improvements could be partially attributed to Qwen3-VL being a particularly strong base model rather than to the MAI-UI methodology. The paper does compare against Qwen3-VL baselines at each scale (Table 2, Table 4, Table 6), showing consistent gains over the untrained base model, which partially addresses this concern. However, the choice of Qwen3-VL over other vision-language backbones (InternVL3, LLaVA) is not justified or ablated.

Claim 2: "Online RL yields significant gains from scaling parallel environments (+5.2 points) and increasing step budget (+4.3 points)." Supported with specific figures and ablations. The evidence is clean: Figure 12b shows monotonic improvement with environment count, and Table 9 shows progressive gains with step budget. The critical insight — that scale is not just helpful but necessary to realize most of RL's benefit — is genuinely novel and well-supported.

However:

• These gains are shown only on the 8B model on AndroidWorld: the paper does not present parallel-environment scaling curves for 2B or 32B models, so we don't know whether the scaling behavior generalizes across model sizes. The 32B model gains only +3.5 points from RL overall (vs. +6.0 for 8B), which could indicate that larger models benefit less from environment scaling — or it could indicate ceiling effects on the 116-task AndroidWorld benchmark.
• The interaction budget scaling conflates two effects: increasing max_env_steps from 15 to 50 both (a) allows the model to attempt tasks it couldn't previously complete within the step limit, and (b) provides richer exploration signals. The paper does not disentangle which effect dominates.
• Cost of scaling is not discussed: 512 parallel Android emulators on 10 cloud servers is a substantial infrastructure investment. The paper does not provide wall-clock training time, total GPU-hours, or cost estimates for the RL stage, making it difficult for other researchers to assess feasibility.

Claim 3: "The device–cloud collaboration system improves on-device performance by 33% and reduces cloud model calls by over 40%." The numbers are clearly reported in Figure 10. However:

• The comparison is against the 2B local agent alone (49.1%): this is the weakest possible baseline for the collaboration system. A stronger comparison would be against a hybrid system that uses simple heuristics (e.g., always escalate when confidence is below a threshold, or always escalate after N failed attempts). The random switch baseline (57.8%) is a step in this direction, but the paper does not compare against more sophisticated task-difficulty-based routing schemes.
• The privacy protection case study (Figure 11) is a pilot, not a systematic evaluation: no quantitative metrics on privacy preservation (e.g., fraction of sensitive data leaked, false positive/negative rates for privacy blocking) are reported. The paper acknowledges this limitation by describing it as a "pilot study," but the system-level claim about privacy-aware routing relies heavily on this unquantified capability.
• The collaboration uses MAI-UI-32B as cloud and MAI-UI-2B as local: both are MAI-UI models trained with the same methodology. The gains may not transfer if the local model and cloud model come from different families or are trained with different data distributions, which is a realistic deployment scenario (e.g., an on-device Qwen model collaborating with a cloud Gemini model).

Claim 4: "MAI-UI addresses the four deployment challenges (agent–user interaction, MCP tools, device–cloud collaboration, dynamic environment robustness)." This claim is partially supported:

• Agent–user interaction: quantitative results on MobileWorld User-Int. subset show MAI-UI-235B achieving 51.1%, a +18.7 point gain over the best end-to-end baseline. This is strong evidence that the training pipeline successfully teaches the model when and how to ask clarifying questions. The case study in Figure 9 provides qualitative validation.
• MCP tool use: MobileWorld MCP subset shows 37.5%, a +32.1 point gain. However, 37.5% is still a low absolute success rate — the model fails on 62.5% of MCP tasks. The case studies in Figure 8 demonstrate capability but also show relatively simple tool invocations (calling a single API with straightforward arguments). The paper does not evaluate on more complex MCP scenarios requiring chained tool calls or error handling when tool calls fail.
• Device–cloud collaboration: supported with quantitative AndroidWorld results (65.5% with DCC) and the privacy case study, but as noted above, the privacy evaluation is qualitative.
• Dynamic environment robustness: the online RL results and the qualitative case studies (Figures 13, 14) showing recovery from pop-ups and wrong-app navigation are compelling but anecdotal. The paper does not report systematic metrics on robustness to specific perturbation types (e.g., pop-up frequency, layout variation, app version changes). A targeted robustness benchmark — where the same tasks are evaluated with and without injected perturbations — would provide stronger evidence.

Missing experiments that would strengthen the paper:

1. Ablation on the fine-grained correctness judgment contribution: what fraction of training data comes from partial-trajectory reuse vs. fully successful trajectories? How much does removing partial-trajectory reuse degrade performance?
2. Direct ablation of Instruction-as-Reasoning: compare coordinate-only SFT+RL vs. instruction-as-reasoning SFT+RL on grounding to quantify the anti-collapse benefit.
3. Cross-model-family validation: train MAI-UI on a non-Qwen backbone (e.g., InternVL3) to test whether gains transfer.
4. Cost analysis for online RL: GPU-hours, wall-clock time, and infrastructure cost for the 512-environment RL training.
5. Systematic robustness evaluation: benchmark with injected pop-ups, layout shifts, and app version changes to quantify the robustness gains from online RL.
6. MCP failure analysis: what types of MCP tasks does the 37.5% success rate represent? Are failures due to incorrect tool selection, wrong arguments, or failure to integrate tool outputs?
7. Privacy-preservation metrics: quantitative evaluation of the privacy monitor's false positive rate (blocking cloud when it would be safe) and false negative rate (allowing cloud access to sensitive data).

Overall assessment. The experiments credibly demonstrate that MAI-UI achieves state-of-the-art grounding and navigation performance across standard benchmarks, that online RL at scale provides meaningful gains over SFT-only training, and that the targeted training for user interaction and MCP tool use produces capabilities absent from prior end-to-end models. The device–cloud collaboration results are promising but preliminary. The paper's central contribution — that addressing real-world deployment challenges through integrated training and systems design yields practical benefits — is supported by the results, with the caveat that the "real-world" evaluation (MobileWorld) still shows substantial room for improvement (41.7% overall success), and the privacy and robustness evaluations are more qualitative than quantitative. The paper is strongest as a demonstration of what careful data pipeline design, scaled RL infrastructure, and deployment-aware systems architecture can achieve; it is weakest in isolating which specific design choices are responsible for the gains and in quantifying performance on the specific deployment blockers it identifies as motivation.

6. Limitations and Trade-offs

The Hardest Tasks Remain Essentially Unsolved

The constraint. The paper's approach yields near-zero improvement on the most difficult navigation tasks, where the base model's initial capability is minimal. On AndroidWorld, the paper groups tasks into difficulty levels based on the current policy's pass@K success rate (Section 3.5), with "frontier tasks" at 0–25% success rate representing the hardest problems. The MobileWorld benchmark — which the paper explicitly describes as "a more challenging and more realistic online benchmark" (Section 3.2.2) — exposes this limitation starkly: MAI-UI-235B-A22B achieves only 41.7% overall success, despite being the strongest model in the lineup. On the MCP tool-use subset specifically, success is only 37.5%, meaning the agent fails on nearly two-thirds of tasks requiring API tool integration.

The paper acknowledges this implicitly when discussing that prior GUI agents were not designed for these capabilities and that previous end-to-end models score only 5.4% on MCP tasks (Table 8). In the device–cloud collaboration analysis, even the largest cloud model (MAI-UI-32B) achieves only 73.3% on AndroidWorld — meaning more than a quarter of tasks in that less-challenging benchmark still fail despite being handled by the strongest available model.

The consequence. Test-time compute and infrastructure scaling cannot compensate for fundamental capability gaps. If the base model cannot recognize when an MCP call is appropriate, cannot compose the correct tool arguments from screen context, or cannot plan a complex multi-app workflow, neither additional online RL training nor device–cloud handoff will create this capability from nothing. The 37.5% MCP success rate and 41.7% MobileWorld overall success mean that in a real deployment, the agent would fail on the majority of tasks requiring these extended capabilities — severely limiting practical utility. A user encountering a 58.3% failure rate on complex tasks would likely abandon the system, regardless of its strong grounding benchmark scores.

Evidence in the paper. Table 8 (MobileWorld) shows the 41.7% overall success ceiling. The SFT vs. RL comparison in Table 9 shows that RL adds only +3.5 points for the 32B model (69.8% → 73.3%) — the proportional gain shrinks as the model scales up, suggesting diminishing returns from RL as the capability ceiling approaches. The MCP subset result (37.5%) and the user-interaction subset (51.1%) are the most direct evidence of these ceiling effects. The paper provides no analysis of why MCP tasks fail — whether due to incorrect tool selection, argument errors, integration failures, or fundamental task complexity — which makes it impossible to determine whether the failures represent fixable errors or capability gaps that require fundamentally different approaches.

Mitigation status. The paper does not specifically address how to push beyond this ceiling. The device–cloud collaboration system helps by routing harder tasks to larger models, but even the largest cloud model hits limits. The online RL framework improves robustness within the model's capability range but does not fundamentally expand it. The paper acknowledges the gap implicitly through the MobileWorld benchmark design but does not propose solutions for the hardest-task regime beyond what is already implemented.

Difficulty Estimation Cost Is Unaccounted for in the Deployment Efficiency Claims

The constraint. The device–cloud collaboration system depends on the Local Agent's ability to monitor its own trajectory quality and decide when to escalate to the cloud. The paper trains the local model (MAI-UI-2B) jointly on standard GUI agent data and monitor-specific data, teaching it to detect trajectory deviations and generate error summaries (Section 2.4.2). However, the paper does not report the cost of this monitoring in terms of inference overhead, latency, or compute.

At every few steps, the local agent must perform the monitor role: evaluate whether the trajectory remains aligned with the user instruction, assess alignment indicators (action execution failure, repeated actions without progress, incorrect inputs, task deviations), and if deviation is detected, generate an error summary. The paper states that the monitor is "explicitly trained to judge whether the trajectory so far remains aligned with the user instruction across diverse apps, layouts, and tasks" (Section 2.4.2), which implies a non-trivial inference burden. Furthermore, the privacy-preserving case study (Figure 11) introduces an "additional local privacy monitor" that blocks cloud switches when sensitive content is present — this is another inference pass, run on-device, that is not accounted for in any cost calculation.

The consequence. Two unaccounted costs undermine the efficiency claims:

1. Monitoring inference overhead: If the monitor runs every k steps and requires a full forward pass (processing the entire trajectory history plus the current screenshot), the on-device inference cost is higher than the paper's implicit assumption of "one inference per action." For a 50-step task with monitoring every 3 steps, that would mean roughly 17 additional monitoring passes — a 34% increase in on-device inference.

2. False positive/negative tradeoff in monitoring: the monitor's accuracy in detecting deviations determines how often the system makes unnecessary cloud calls (false positives — escalating when the local agent would have recovered on its own) or misses genuine failures (false negatives — the agent silently continues on a wrong trajectory). The paper provides no precision/recall metrics for the monitor, making it impossible to assess whether the reported 33% performance improvement from DCC is robust or would degrade significantly with a less well-trained monitor.

3. Latency cost of monitoring: on-device inference for a 2B model is fast but not free. Running both the agent role and the monitor role at each check interval increases per-step latency, potentially negating some of the latency benefits of on-device execution. The paper reports that DCC reduces cloud model calls by 42.7% but does not report end-to-end latency comparisons.

Evidence in the paper. The paper acknowledges that trajectory monitoring requires explicit training: "In practice, the monitor must handle varied and complex cases, and prompt engineering alone is unlikely to deliver reliable monitoring" (Section 2.4.2). However, it does not report: (a) the frequency at which the monitor runs (Section 2.4.1 says "every few steps" without specifying the exact interval), (b) the additional inference cost per monitoring check, (c) the monitor's accuracy metrics on held-out deviation detection tasks, or (d) end-to-end latency comparisons between DCC and pure-cloud or pure-on-device baselines.

Mitigation status. The paper does not address these costs. The efficiency claims — "reduces cloud model calls by 42.7%" and "over 40% of tasks completed entirely on-device" — are computed against cloud-only serving as the baseline, which implicitly assumes monitoring is free. Joint training of the agent and monitor roles (Section 2.4.2) is a design choice that reduces model storage overhead (one model instead of two) but does not reduce inference overhead (the model still performs additional computation for monitoring). The paper does not discuss whether monitoring could be made cheaper through smaller dedicated monitor heads, sparse checking schedules, or confidence-based triggering.

Online RL Scaling Results Are Reported Only on a Single Model Size and Benchmark

The constraint. The paper's strongest experimental claim about online RL — that scaling parallel environments from 32 to 512 yields +5.2 points on AndroidWorld, and that increasing step budgets from 15 to 50 yields +4.3 points — is demonstrated only on the 8B model (Table 9, Figure 12b). The paper does not report parallel-environment scaling curves for the 2B, 32B, or 235B models. The RL ablation comparing standard GRPO versus enhanced GRPO is also shown only for the 8B model (Table 9).

Furthermore, all online RL experiments are evaluated on a single benchmark: AndroidWorld (116 tasks across 20 apps). The paper does not report online RL training results on MobileWorld or any other online benchmark.

The consequence. Three uncertainties arise:

1. The scaling behavior may not generalize across model sizes. The 32B model gains only +3.5 points from RL overall (69.8% → 73.3%), compared to +6.0 for the 8B model. This could mean that larger models benefit less from environment scaling because they already have stronger priors and need less diverse exploration. Alternatively, it could mean that the 32B model hits a ceiling on the 116-task AndroidWorld benchmark and would show larger gains on a harder benchmark. Without scaling curves for 32B, we cannot distinguish these explanations. If the former is true, the infrastructure investment for 512 environments may be unjustified for larger models; if the latter, the paper's headline scaling finding is benchmark-specific.

2. The specific scaling curves may be infrastructure-dependent. Training with 512 AVD instances requires "just 10 standard Alibaba Cloud ECS servers (ecs.ebmg5s.24xlarge)" (Section 2.3.3). The optimal environment count may depend on GPU-to-environment ratios, network bandwidth between inference servers and environment containers, and the specific task distribution. Another organization running on different hardware might see different scaling behavior. The paper provides no guidance on how to determine the optimal environment count for a given infrastructure setup.

3. The relative importance of algorithmic enhancements vs. environment scaling is not isolated. The +6.0 point gain for the 8B model comes from the combination of enhanced GRPO (curriculum + repetition penalty + experience replay + asymmetric clipping) at 50 max_env_steps and 512 environments. We cannot determine how much of the gain comes from each factor independently — e.g., would enhanced GRPO with 32 environments still outperform standard GRPO with 512 environments? This makes it difficult for practitioners to prioritize their investment (better algorithms vs. more hardware).

Evidence in the paper. The 8B-focused analysis is shown in Table 9 and Figure 12. The 32B RL gain is reported as a single number (69.8% → 73.3%) without any scaling analysis. The 2B RL gain (45.1% → 49.1%) is similarly a single pre/post comparison. The paper does not discuss why only the 8B model was used for the detailed scaling analysis.

Mitigation status. The paper acknowledges that "scaling parallel environments to enhance exploration diversity is critical for overcoming performance bottlenecks in GUI agent RL training" (Section 3.5), but does not discuss the lack of cross-model and cross-benchmark validation for this claim. The "MAI-UI-8B-RL" label in Figure 12 makes the scope explicit, but the paper does not caution readers against overgeneralizing the 8B results to other model sizes.

No Systematic Evaluation of Robustness to Specific Perturbation Types

The constraint. The paper argues that online RL improves robustness to dynamic environments by exposing the model to "pop-ups or permission dialogs [that] can appear unexpectedly" and "layouts [that] vary across app versions and devices" (Section 1). However, the evaluation of this robustness is entirely qualitative: two case studies (Figures 13 and 14) showing the agent dismissing a notification permission dialog and recovering from navigating to the wrong app. The paper does not report systematic metrics on robustness to specific perturbation types.

The consequence. The paper cannot answer questions that a practitioner would need to know for deployment confidence:

• What is the success rate degradation when pop-ups appear? If a task has a 70% success rate in a clean environment, does it drop to 60% or 20% when a notification dialog appears? The case study in Figure 13 shows the RL-trained model succeeds, but it does not report the failure rate of the SFT-only baseline on the same task with the same pop-up.
• How does robustness vary by perturbation type? Permission dialogs, network error pop-ups, app update notifications, and keyboard auto-show/hide events are all different types of interruptions. The agent may handle some well and others poorly. Without per-type metrics, a developer cannot anticipate which scenarios are risky.
• Does online RL improve robustness uniformly or only for perturbations similar to those seen during training? The training environment uses 35+ self-hosted applications (Section 2.3.3). If the RL training never exposes the model to, say, in-app purchase dialogs, does the robustness generalize to those? The paper provides no out-of-distribution perturbation evaluation.
• What is the robustness-latency tradeoff? The RL-trained model may be more robust because it acquires cautious strategies (e.g., always checking for and dismissing unexpected dialogs before proceeding). This increases reliability but also increases the average number of steps per task. The paper does not report whether the RL-trained model takes more steps on average than the SFT-only model.

Evidence in the paper. The qualitative case studies in Figures 13 and 14 are the only robustness-specific evidence. The AndroidWorld benchmark includes dynamic elements (since it's a live emulator), and the performance improvement from RL on this benchmark (+4.0 to +6.0 points across model sizes in Table 9) could be interpreted as an aggregate robustness signal. However, AndroidWorld does not systematically inject or vary perturbations — it includes whatever dynamic behaviors arise naturally from the apps — so this is a weak proxy for robustness evaluation. The paper does not report AndroidWorld results stratified by whether tasks encountered unexpected dialogs or interruptions.

Mitigation status. The paper does not frame this as a limitation or propose a systematic robustness evaluation. The MobileWorld benchmark (Section 2.5) is described as "more challenging and realistic" but its additional challenge comes from user interaction and MCP requirements, not from systematically varied UI perturbations. A targeted evaluation — such as running the same tasks with and without injected pop-ups and measuring the performance delta — would have substantially strengthened the robustness claims. The paper's acknowledgment that "without exposure to dynamic environments in training, agents generalize poorly and remain brittle to real-world unpredictability" (Section 1) is a motivation for online RL, but the evaluation of whether RL actually solves this problem remains anecdotal.

The Instruction-as-Reasoning Anti-Collapse Claim Lacks a Direct Ablation

The constraint. The paper claims that Instruction-as-Reasoning — training the model to generate diverse reasoning perspectives (appearance, function, location, intent) before predicting coordinates — prevents policy collapse in the SFT+RL grounding paradigm, a problem that prior work (Phi-Ground, Zhang et al., 2025a) documented but did not solve (Section 3.6). Specifically, the paper states: "Instruction-as-Reasoning stabilizes RL by pretraining the model to generate diverse reasoning pathways, which enhances exploratory behavior and stabilizes policy optimization in the RL phase."

However, the paper does not include a controlled experiment comparing Instruction-as-Reasoning SFT+RL against coordinate-only SFT+RL. The grounding results in Tables 2–5 compare MAI-UI against other models (with different architectures, data, and training procedures) rather than against an ablated version of MAI-UI without the reasoning-perspective training. The paper's own prior work (UI-Ins, Chen et al., 2025) is cited as the source of the Instruction-as-Reasoning approach, but the current paper does not replicate or extend the ablations from that prior work.

The consequence. Three alternative explanations for MAI-UI's strong grounding performance cannot be ruled out:

1. The grounding gains may come from data quantity and quality rather than the reasoning structure. The data pipeline (Section 2.2.1) collects screenshots from containerized environments, generates diverse instructions with MLLM prompting, and filters out low-quality examples (noting that 23.3% of open-source grounding instructions have quality issues). A coordinate-only model trained on this same carefully curated data might achieve comparable performance — the reasoning text might be epiphenomenal rather than causal.
2. The RL stabilization may come from other factors. The paper uses GRPO with format reward (rewarding correct XML-style tag structure) and point-in-box reward. The format reward alone might be sufficient to prevent policy collapse by penalizing unparseable outputs. Without an ablation, we cannot determine whether the reasoning pathways are necessary or whether the format reward plus the high-quality SFT data would achieve similar stability.
3. The emergent multi-perspective capability may be post-hoc rationalization. The paper claims the model "can strategically select appropriate reasoning perspectives given different contexts" and "can also generate novel analytical angles beyond the four trained perspectives" (Section 3.6). Without controlled experiments comparing models trained with and without the reasoning structure, this claim is based on qualitative observation of model outputs, which is susceptible to confirmation bias (noticing when the model's reasoning happens to align with the trained perspectives).

Evidence in the paper. Section 3.6 describes the benefits of Instruction-as-Reasoning but does not present a formal ablation. The grounding benchmark results (Tables 2–5) show MAI-UI outperforming baselines, but the gap between MAI-UI and coordinate-only models could be attributed to any combination of: better data, better base model (Qwen3-VL), the RL phase, the Instruction-as-Reasoning structure, or the zoom-in strategy. The closest thing to an ablation is the comparison against the Qwen3-VL base models at each scale (e.g., Qwen3-VL-32B achieves 54.9% on ScreenSpot-Pro vs. MAI-UI-32B at 67.9%), but this is a comparison against a model with no grounding-specific training at all, not against a model trained on the same data with coordinate-only SFT+RL.

Mitigation status. The paper references the authors' prior work (UI-Ins, Chen et al., 2025) as the source of the Instruction-as-Reasoning paradigm and its experimental validation. However, a reader evaluating MAI-UI as a standalone contribution cannot assess the strength of this claim without cross-referencing the prior paper. The current paper would be strengthened by even a single ablation on one grounding benchmark (e.g., ScreenSpot-Pro) comparing Instruction-as-Reasoning SFT+RL against coordinate-only SFT+RL with otherwise identical data and training.

The MobileWorld Evaluation Reveals a Large Absolute Performance Gap Despite State-of-the-Art Relative Gains

The constraint. While MAI-UI achieves substantial relative improvements over prior end-to-end models on MobileWorld (+20.8 points overall for the 235B model, +32.1 points on the MCP subset, +18.7 points on the user-interaction subset per Table 8), the absolute success rates remain low: 41.7% overall, 37.5% on MCP, 51.1% on user interaction. This means that even the best MAI-UI model fails on the majority of MobileWorld tasks.

The paper explicitly positions MobileWorld as bridging "the gap for more challenging and realistic online assessment" (Section 3.2.2) and notes that it "comprises evaluations beyond pure GUI operations." The benchmark thus represents the paper's closest approximation to real-world deployment conditions — and in those conditions, the agent is unsuccessful more often than not.

The consequence. The headline claim — that MAI-UI addresses "four key challenges to realistic deployment" (Section 1) — is partially undermined by the low absolute MobileWorld scores. Specifically:

• The MCP augmentation training produces capability, but not reliability. 37.5% MCP success means the agent correctly decides to use a tool and correctly formulates the tool call for only about one in three MCP-appropriate tasks. In a real deployment, this failure rate would be unacceptable for any task where tool use is the expected or required approach. The paper does not break down whether MCP failures are due to the agent not recognizing that a tool is available, selecting the wrong tool, providing incorrect arguments, or failing to integrate the tool's output — any of these failure modes would require different fixes.

• The user-interaction training produces a reasonable asking capability (51.1%) but a substantial failure rate for a capability that is binary in nature. Either the agent recognizes that information is missing and asks, or it proceeds with incomplete information and likely fails. A 48.9% failure rate on user-interaction tasks means that in roughly half of cases where the user's instruction is ambiguous, the agent does not ask for clarification — it either guesses or fails silently.

• The gap between AndroidWorld (76.7%) and MobileWorld (41.7%) is ~35 points for the same model (MAI-UI-235B-A22B). This gap quantifies how much harder real-world-oriented tasks are compared to standard mobile navigation tasks. A practitioner evaluating MAI-UI for deployment would need to estimate where their use case falls on the AndroidWorld-to-MobileWorld spectrum — and most real-world use cases likely fall closer to MobileWorld (ambiguous instructions, benefit from API tools) than to AndroidWorld (well-specified tasks, pure UI operation).

Evidence in the paper. All MobileWorld results are in Table 8. The GUI-only subset of MobileWorld (39.7% for the 235B model) provides a partial control: even without user-interaction or MCP requirements, MobileWorld tasks are substantially harder than AndroidWorld tasks (39.7% vs. 76.7%), likely due to longer task horizons, cross-app workflows, and more complex app interactions. This suggests that the low overall score is not solely due to the novel extended capabilities — the base navigation difficulty is also higher.

Mitigation status. The paper is transparent about the MobileWorld results and does not attempt to spin them as more positive than they are. However, it also does not analyze the failure modes or discuss how the remaining performance gap might be closed. The agentic framework baselines (GPT-5 + UI-Ins at 51.7%, Gemini-3-Pro + UI-Ins at 46.3%) establish an upper bound that MAI-UI as an end-to-end model is approaching but not yet matching. The paper does not discuss whether further scaling of model size, RL training, or data would close this gap, or whether fundamentally different approaches (e.g., explicit planning modules, better error recovery strategies, more sophisticated tool-use training) would be required.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper shifts the evaluation and development of GUI agents from benchmark-centric optimization toward deployment-requirement-driven design. The core reframing is not methodological — the individual techniques (GRPO, instruction-as-reasoning, iterative rejection sampling, device–cloud routing) all build on prior work — but architectural: MAI-UI treats real-world deployment constraints (user ambiguity, UI brittleness, privacy concerns, dynamic environment robustness) as first-class design specifications rather than post-hoc considerations. This is a substantial shift in how the field defines progress.

The magnitude of this shift is best understood by contrasting MAI-UI with the typical GUI agent paper. Prior work asks: "Given a fixed benchmark (ScreenSpot, AndroidWorld), can we improve the score?" MAI-UI asks: "What capabilities must a GUI agent possess to be deployed in practice, and what training and system architecture produces those capabilities?" The benchmarks become validation of the deployment thesis rather than the thesis itself. This inversion matters because it changes the objective function of research: rather than incrementally improving grounding accuracy on static screenshots, the community is now incentivized to identify and close capability gaps that prevent real-world use — gaps that existing benchmarks may not even measure (e.g., MCP tool use, where prior end-to-end models scored 5.4% on MobileWorld versus MAI-UI's 37.5%).

The paper also resolves a specific contradiction in the GUI grounding literature. Prior work established two competing paradigms — pure SFT (stable but plateauing) and pure RL (higher performance but prone to collapse) — with the natural synthesis (SFT+RL) documented as fragile by Phi-Ground (Zhang et al., 2025a). MAI-UI's Instruction-as-Reasoning approach provides both a diagnosis and a solution: SFT with coordinate-only supervision teaches a brittle input-output mapping that RL perturbations easily break; SFT that teaches structured reasoning pathways creates an exploration space where RL can select and compose strategies without violating output format constraints. This converts a "don't do SFT+RL" finding into a "here's how to do SFT+RL correctly" recipe, resolving the tension rather than merely documenting it.

More broadly, the paper establishes online RL at scale as a viable and necessary training stage for GUI agents. Prior RL efforts in this domain operated at modest scale (tens of environments, limited step budgets) and produced modest gains. MAI-UI's demonstration that scaling from 32 to 512 parallel environments yields +5.2 points, and that increasing step budgets from 15 to 50 yields +4.3 points, changes the default expectation: online RL for GUI agents is not a minor fine-tuning step but a core training component whose benefits are largely unrealized at small scale. This makes infrastructure investment in containerized GUI environments a high-leverage research activity — more environments directly translate to better agents — which was not obvious before this paper.

The device–cloud collaboration architecture introduces a new deployment paradigm that blurs the false dichotomy between on-device and cloud models. Prior work treated these as mutually exclusive choices with fixed tradeoffs (privacy vs. capability). MAI-UI's monitor-guided handoff system demonstrates that a small local model augmented with strategic cloud escalation can approach the performance of much larger pure-cloud models while preserving privacy for over 40% of tasks and reducing cloud calls by ~43%. This creates a new design point in the deployment space that was previously unavailable: privacy-aware, cost-efficient routing that adapts per-task rather than per-deployment.

Research directions that become more attractive after this work:

• Infrastructure for scaled GUI agent RL: the paper shows that environment count is a first-order determinant of RL performance. Investment in faster, more lightweight GUI emulators, better container orchestration, and more efficient environment reset mechanisms now has clear ROI.
• Learned verifiers for GUI task completion: the paper uses hybrid rule-based and MLLM-as-a-Judge verification (83% agreement with humans). Improving verifier accuracy — especially for the 17% of cases where the MLLM judge disagrees with humans — is a bottleneck that directly limits RL training quality.
• Fine-grained trajectory evaluation: the paper's partial-trajectory reuse demonstrates that step-level correctness signals are far more data-efficient than trajectory-level binary success. Applying this insight systematically — through better step-level verifiers, automatic error localization, or learned progress estimators — could dramatically improve data yield from model-based rollouts.
• Benchmarks that measure deployment-readiness rather than raw capability: MobileWorld's inclusion of user-interaction and MCP subsets exposes large performance gaps (41.7% overall) that standard benchmarks hide. The field needs more benchmarks designed around deployment scenarios, including privacy-preservation metrics, latency constraints, and robustness to specific perturbation types.

Research directions that become less attractive:

• Pure benchmark optimization on static grounding datasets: MAI-UI achieves 73.5% on ScreenSpot-Pro and 91.3% on MMBench GUI L2, suggesting diminishing returns from further incremental improvements on these benchmarks. The larger gap is on deployment-oriented metrics (MobileWorld 41.7%), redirecting effort toward capability expansion rather than benchmarking.
• Pure on-device or pure-cloud solutions: the device–cloud collaboration results (33% improvement over on-device alone, >40% cloud call reduction) suggest that hybrid architectures dominate either extreme for most deployment scenarios. Research on isolated on-device models without considering cloud fallback is now less practically relevant.
• Complex multi-component agent frameworks without end-to-end training: MAI-UI as a single end-to-end model (41.7% on MobileWorld) is competitive with agentic frameworks using GPT-5 (51.7%) or Gemini-3-Pro (46.3%) as planners. The gap is narrowing, suggesting that end-to-end training with scaled data and RL may eventually match or exceed pipeline approaches while being simpler to deploy.

Follow-Up Research This Work Enables

Cheap, online difficulty estimation for adaptive device–cloud routing. The current device–cloud collaboration system monitors trajectory alignment "every few steps" (Section 2.4.1), but the paper does not report the inference cost of monitoring or the optimal checking frequency. A natural follow-up would train a lightweight difficulty predictor — a small classifier head on top of the 2B model's intermediate representations — that estimates, from the first few screenshots alone, whether a task is likely to require cloud intervention. This predictor could be trained on the DCC system's own logs: for each task, the ground-truth label is whether the monitor triggered a cloud switch and whether that switch successfully recovered the trajectory. The key metrics would be: (a) cloud call reduction compared to the current periodic-checking approach, (b) success rate compared to always-cloud and always-local baselines, and (c) inference overhead of the predictor relative to the full monitoring pass. A negative result — showing that early screenshots contain insufficient signal for difficulty prediction — would imply that runtime monitoring is fundamentally necessary and that investment should go into making monitoring cheaper rather than replacing it.

Systematic robustness evaluation with controlled perturbation injection. The paper demonstrates qualitative robustness to pop-ups and permission dialogs (Figures 13, 14) but provides no quantitative metrics. A targeted evaluation would take the AndroidWorld benchmark and systematically inject perturbation types during task execution: notification pop-ups at random steps, permission dialogs, network error messages, keyboard appearance/disappearance, and app version changes (where UI elements shift position). For each perturbation type and each model variant (SFT-only vs. RL-trained at different environment counts), the experiment would measure: (a) success rate degradation relative to clean execution, (b) average recovery steps (how many extra actions the model takes to dismiss the perturbation and resume), and (c) whether robustness to one perturbation type transfers to others. The hypothesis from the paper's qualitative results is that RL-trained models, especially those trained with 512 environments, should show smaller degradation than SFT-only models. A finding that RL robustness is perturbation-specific (e.g., handles pop-ups but not layout shifts) would indicate that the current RL environment diversity is insufficient and that perturbation injection during training is necessary.

Isolating the contribution of partial-trajectory reuse to data efficiency. The fine-grained correctness judgment approach (Section 2.3.1) retains the longest correct prefix from failed trajectories, but the paper does not quantify how much training data this contributes or ablate its impact. A controlled experiment would train navigation models at a fixed scale (e.g., 8B) with identical hyperparameters but vary the data filtering strategy: (a) keep only fully successful trajectories (discard all failures), (b) keep full trajectories including failed ones (train on erroneous actions), (c) keep only the correct prefix of failed trajectories (the MAI-UI approach), and (d) keep the correct prefix plus a fixed number of subsequent steps (to test whether exposure to the failure point helps). The key metrics are: final AndroidWorld success rate, number of training trajectories required to reach a given performance threshold, and whether models trained on partial trajectories exhibit different failure modes (e.g., better early-step accuracy but worse recovery behavior). A finding that partial-trajectory reuse is the primary driver of data efficiency would validate the paper's central data pipeline innovation; a finding that it provides only marginal benefit over discarding failures would shift focus to improving trajectory generation quality rather than salvage.

Cross-model-family validation of the MAI-UI training recipe. All results use Qwen3-VL as the backbone. To test whether the training methodology generalizes, replicate the full pipeline (grounding SFT + navigation SFT + online RL) on a different vision-language backbone — for example, InternVL3-8B (Zhu et al., 2025) or a comparable LLaVA variant. The experiment would compare: (a) the base model's untrained grounding and navigation performance, (b) performance after MAI-UI SFT stages, and (c) performance after online RL. The key question is whether the relative gains (e.g., +6.0 points from RL on 8B, +10.9 points on ScreenSpot-Pro grounding) transfer proportionally or are Qwen3-VL-specific. A positive result would establish MAI-UI as a model-agnostic training recipe; a negative result (e.g., the grounding gains disappear on a different architecture) would indicate that the Instruction-as-Reasoning paradigm interacts with specific architectural properties of Qwen3-VL and needs modification for other backbones.

MCP tool-use failure analysis and targeted improvement. The 37.5% MCP success rate on MobileWorld (Table 8) represents the largest capability gap relative to the agentic framework baselines (48.6–51.6%). A diagnostic study would categorize all MCP task failures into: (a) the agent did not recognize that a tool was available or appropriate (used GUI instead), (b) the agent selected the wrong tool, (c) the agent provided incorrect or incomplete arguments, (d) the tool executed successfully but the agent failed to integrate the output into subsequent steps, or (e) the task was fundamentally infeasible with available tools. This categorization would be done by human annotation of a subset of failed trajectories. The results would directly inform follow-up work: if failures are predominantly type (a), the fix is better tool-selection training data; if type (c), the fix is better argument-generation training with more diverse tool schemas; if type (d), the fix is better output-parsing and integration training. A finding that failures are evenly distributed across categories would suggest that the MCP training data needs to be scaled in both quantity and diversity, rather than targeted at a specific weakness.

Scaling online RL to larger models and harder benchmarks. The paper's RL scaling analysis (Figure 12b, Table 9) is restricted to the 8B model on AndroidWorld. A natural extension would replicate the parallel-environment scaling experiment for the 32B model on MobileWorld (rather than AndroidWorld), which is a harder benchmark where the 32B model has more room for improvement (37.3% MobileWorld vs. 73.3% AndroidWorld). The experiment would train MAI-UI-32B with RL at environment counts of 32, 128, 256, and 512 on MobileWorld tasks, measuring: (a) whether the scaling benefits observed for 8B on AndroidWorld (+5.2 points from 32 to 512 environments) transfer to larger models on harder benchmarks, (b) whether the larger model requires more or fewer environments to saturate, and (c) whether MobileWorld's user-interaction and MCP subsets benefit differently from environment scaling than the GUI-only subset. A finding that the 32B model saturates at fewer environments than the 8B model would suggest that larger models have stronger inductive biases and need less exploration diversity; a finding that MobileWorld benefits more from scaling than AndroidWorld would validate the paper's claim that harder tasks require rich exploration and justify the infrastructure investment for production-scale training.

Practical Applications and Downstream Use Cases

Privacy-sensitive enterprise mobile assistants. In enterprise settings where employees handle confidential data (emails, documents, customer information) on mobile devices, a pure-cloud GUI agent is often unacceptable due to data governance requirements. MAI-UI's device–cloud collaboration system enables a deployment where the 2B on-device model handles routine tasks locally (checking calendar, drafting messages, finding files) while escalating to the 32B or 235B cloud model only for complex cross-app workflows — and critically, the privacy monitor blocks cloud escalation when sensitive content is on screen (as demonstrated in the pilot study, Figure 11). The paper's numbers suggest that ~40% of AndroidWorld tasks complete entirely on-device with the 2B model, and the DCC system achieves 65.5% overall (Figure 10), providing a concrete performance estimate for an enterprise deployment. The ~43% reduction in cloud API calls directly translates to lower per-user operating costs.

Accessibility tools for users with motor or visual impairments. GUI agents that can execute complex multi-step tasks from natural language instructions represent a potentially transformative accessibility technology. A user with limited manual dexterity could say "find the latest email from my doctor, extract the appointment time, and add it to my calendar" rather than performing dozens of precise taps and swipes. The critical requirement for accessibility applications is reliability under unpredictable conditions — the agent cannot fail silently or require the user to diagnose what went wrong. MAI-UI's online RL training (which improves robustness to pop-ups, dialogs, and unexpected UI states) and its agent–user interaction capability (which allows the agent to ask clarifying questions rather than guessing when instructions are ambiguous) directly address these reliability requirements. The 51.1% user-interaction success rate on MobileWorld (Table 8) is insufficient for standalone deployment in high-stakes accessibility contexts, but the architecture supports a human-in-the-loop mode where the user confirms critical actions, making the current capability level practical as an assistive rather than fully autonomous tool.

Automated testing and QA for mobile applications. Mobile app testing requires executing diverse user workflows across many device configurations, OS versions, and app states — precisely the kind of dynamic, variable environment that MAI-UI's RL training pipeline is designed to handle. A QA team could specify test scenarios in natural language ("create an account, add an item to cart, apply a discount code, proceed to checkout, verify the total") and deploy MAI-UI agents across hundreds of parallel emulator instances to execute these scenarios at scale. The online RL framework's support for 512 parallel environments (Section 2.3.3) and the containerized AVD infrastructure provide a blueprint for this deployment. The key advantage over scripted UI testing is robustness to UI changes: when an app update moves a button or changes a flow, a scripted test breaks, but an RL-trained MAI-UI agent may adapt (as suggested by the robustness case studies in Figures 13 and 14). The paper does not evaluate MAI-UI on app-testing-specific metrics (bug detection rate, false positive rate, test coverage), so a pilot deployment would need to establish these baselines.

Self-improving data generation for GUI agent training. The iterative rejection sampling pipeline (Section 2.3.1, Phase 3) is a general mechanism for improving any GUI agent through self-play: deploy the current model to generate trajectories on novel tasks, filter for correctness, and retrain. This pipeline could be applied to continuously improve a deployed MAI-UI agent using real user interactions (with appropriate privacy safeguards). As users issue instructions and the agent executes them, successful trajectories (where the user does not intervene or correct the agent) become training data for the next model iteration; failed trajectories (where the user takes over or the task is abandoned) are discarded or mined for partial successes. The paper's demonstration that iterative rejection sampling "makes both the model and the training corpus self-evolving" suggests that this flywheel can produce compounding improvements over time — each deployment cycle generates data that improves the next model, which generates better data, and so on. The key requirement is reliable automatic verification of task success, which the paper's hybrid verifier approach (rule-based + MLLM-as-a-Judge, 83% agreement with humans) partially addresses but which would need improvement for fully autonomous self-improvement without human annotation in the loop.