MemGUI-RL

MemGUI-RL: Reinforcement Learning for Proactive Context Management in Long-Horizon Mobile GUI Agents

Anonymous ICLR 2027 submission

Long-horizon mobile agents must decide not only where to tap but also what to keep. MemGUI-RL post-trains policy-managed memory with FARPO (Folding-Aware Reward-Decoupled Policy Optimization): each verifier component is standardised within its prompt group before weighting, and the span-to-step exposure of the training prompts is set explicitly. The resulting MemGUI-8B-RL is the best open-data 8B model on both MemGUI-Bench and the out-of-distribution MobileWorld.

Code Repo MemGUI-8B-RL Training Data Evaluation Logs

Core Idea

Two properties of policy-managed memory break the naive GRPO recipe.

A Context-as-Action (ConAct) policy folds its own history, updates its UI memory and emits the next GUI action in one response. Annotation-based verifiers score that response with several components, and the components behave very differently: (1) aggregating them before normalisation lets the component with the largest within-group spread decide the update, and (2) span-level history folds are only 22.7% of the annotated folds, so natural sampling collapses the policy to step-only folding.

Why GRPO needs adapting for policy-managed memory, and what FARPO changes
Why GRPO needs adapting. (a) Applied naively to ConAct, GRPO normalises the weighted sum of heterogeneous verifier rewards with one shared standard deviation, and natural sampling drives the deep-fold rate on held-out states from 17.7% to 0.2%. (b) FARPO standardises every component before weighting and turns span exposure into a sampling ratio ρ.

Results

MemGUI-8B-RL: the best open-data 8B model on both long-horizon benchmarks.

Main results of MemGUI-8B-RL on MemGUI-Bench and MobileWorld, and the 2x2 ablation
Main results. (a) MemGUI-Bench Pass@3 against end-to-end models up to 32B. (b) Out-of-distribution MobileWorld GUI-only success among open-data models. (c) The 2×2 ablation from the same SFT checkpoint: reward decoupling and folding-aware sampling are both needed.
43.0%MemGUI-Bench Pass@3

Up from 35.9% for MemGUI-8B-SFT (+19.8% relative) and more than double the Qwen3-VL-8B-Instruct backbone (20.3%).

35.0%Information-retention rate

IRR rises from 30.2% (+15.9% relative); the largest gains land on cross-application tasks.

19.7%MobileWorld GUI-only SR

Out-of-distribution transfer without any MobileWorld rollout in training (SFT: 17.9%).

ρA bimodal footprint

Folding exposure is a hyper-parameter: below ρ≈2 the policy folds step by step, above it span folds dominate.

Method

FARPO fixes which candidates are compared and which decisions the policy sees.

Overview of FARPO
Overview. (a) The inherited ConAct loop: one policy emits the UI action and the folding command that maintains its structured memory. (b) Folding-aware sampling keeps all span folds and subsamples step folds, replacing the natural 22.7% span share with a chosen prior. (c) Reward-decoupled group advantage: the factorised verifier scores G = 5 sampled responses, each component is standardised within the group (constant components gated out), weighted with w, broadcast to tokens, batch-whitened and optimised with a dual-clipped PPO surrogate and a KL penalty to the SFT policy.
Learning signal, offline memory metrics and failure labels

Reward-decoupled group advantages

Following GDPO, every verifier component is centred and scaled within its prompt group before the preference weights are applied, so a one-standard-deviation change in the format component and in the folding component move the advantage by the same amount. Zero-variance components are gated out instead of adding noise, and batch whitening keeps the response ordering.

Folding exposure controls a bimodal footprint

Folding-aware sampling

Span folds are rare and step folds are already solved by the SFT policy, so most prompt groups carry no learning signal under natural sampling. FARPO keeps every span example and draws mρ = min{Nstep, ⌈Nspan/ρ⌉} step examples, which lifts the share of informative groups from 25–33% to 62–77% and turns folding exposure into an explicit, bimodal hyper-parameter.

Case study on MemGUI-Bench task 091
MemGUI-Bench task 091 (Amazon → Wikipedia → Messages, eight facts). Only the FARPO policy stores the facts with explicit memory entries, compresses 71 steps into two folded lines and delivers the message.

Case Study

Span folds and explicit memory survive an 80-step task.

The base model loops between "I have the price, now I need the rating" and "I have the rating, now I need the price" for 35 steps. The SFT policy and the GRPO recipe fold every step separately and exhaust their step budgets before Messages is opened. Reward decoupling without folding-aware sampling never leaves the search box. MemGUI-8B-RL folds each sub-task into one line, adds four memory entries and sends one SMS with all eight facts.

Open Release

Everything needed to reproduce the post-training.

CodeFARPO trainer (reward-decoupled estimator, folding-aware sampler, ConAct verifier) and the launch scripts of every reported run. ModelMemGUI-8B-RL, the FARPO (ρ = 9) checkpoint post-trained from MemGUI-8B-SFT. Training dataAnnotated MemGUI-3K states in the RL conversation format, with the train / held-out split used in the paper. Evaluation logsMemGUI-Bench and MobileWorld trajectories and judge outputs for every model row in the paper.

MemGUI-8B-SFT, MemGUI-3K and the ConAct interface are inherited from the MemGUI-Agent release; MemGUI-Bench and MobileWorld are used with their public harnesses.