MemGUI-Agent - A long-horizon mobile GUI agent jointly launched by Zhejiang University and Kuaishou

What is MemGUI-Agent

MemGUI-Agent is a long-horizon mobile GUI agent jointly launched by Zhejiang University and Kuaishou, designed specifically for cross-app, multi-step, long-chain mobile automation tasks. Traditional ReAct-style agents suffer from key information being diluted in long-horizon tasks due to the linear expansion of historical records. Through the ConAct protocol, MemGUI-Agent enables the model to simultaneously decide at each step how to operate the UI and how to organize memory, achieving compact and structured context management.

Main features of MemGUI-Agent

• Long-horizon task execution: Supports complex automation workflows across apps and pages, handling an average of over 30 consecutive operations.
• Active context management: Through the ConAct protocol, the agent autonomously decides when to compress history, when to write UI facts, and how to describe the current step.
• Structured memory operations: Supports memory actions such as memory_add, memory_update, and memory_delete, persistently preserving key UI facts such as prices, phone numbers, and specifications.
• End-to-end model training: Provides the LoRA fine-tuned model MemGUI-8B-SFT based on Qwen3-VL-8B, capable of long-horizon tasks with only 8B parameters.
• Full-chain open source: Code, the MemGUI-3K dataset, the training/evaluation pipeline, and model weights are all open.

Technical principles of MemGUI-Agent

• ConAct (Context-as-Action) core framework: MemGUI-Agent elevates context management from a traditional external rule module or post-processing summarizer to a first-class action on par with UI clicks, swipes, and inputs. In each step of reasoning, the model actively decides what history to compress, what UI facts to remember, and how to describe the current step, making context updates an executable and trainable policy output.
• Three structured state fields: The agent splits the context into three complementary fields that work together。Folded Action History is responsible for compressing completed subtask segments into reusable summaries to avoid linear expansion of history; Folded UI State specifically persists key facts across screens, ensuring information can still be precisely retrieved after dozens of steps; Recent Step Record records the screen observation, action intent, executed action, and result of the most recent step, providing reliable local material for subsequent history folding and memory writing.
• Five-segment structured output and memory operations: In each step, the model outputs five structured segments: thinking, folding, tool_call, ui_observation, and action_intent. Among them, tool_call can be a regular UI operation or an explicit memory operation such as memory_add, memory_update, or memory_delete.

Follow WeChat and reply “open source” to join theAI Open Source Project Exchange Group

How to use MemGUI-Agent

• Clone the open-source code: Pull the complete code and training/evaluation pipeline from the GitHub repository github.com/kwai/MemGUI-Agent .
• Download the dataset: Obtain the MemGUI-3K dataset on HuggingFace.
• Load model weights: Download MemGUI-8B-SFT from HuggingFace.
• Configure the Android environment: Connect a device or start an emulator, and deploy the agent execution environment.
• Enter the task instruction: Describe long-horizon goals in natural language, and the Agent automatically plans and executes.

Core Advantages of MemGUI-Agent

• Significantly Reduced Context Cost: In long-horizon regions of 150 steps, ConAct saves approximately 1,500 input tokens on average compared with ReAct, suppressing linear growth.
• Precise Retention of Key Information: By persistently storing complete facts through UI State, it avoids information dilution and hallucination in long-horizon tasks.
• Small Model, Big Capability: After training with MemGUI-3K, an 8B model surpasses most 32B and even 235B baselines on long-horizon benchmarks, enabling efficient deployment.
• Excellent Out-of-Distribution Generalization: MemGUI-8B-SFT achieves a 17.9% success rate on MobileWorld, an improvement of 8.5 percentage points over the baseline, demonstrating strong cross-scenario transfer capability.
• Fully Reproducible Pipeline: From data construction and model training to the evaluation protocol, everything is fully open source, lowering the barrier to research.

Project Address of MemGUI-Agent

• Project Website：https://memgui-agent.github.io/
• HuggingFace Model Repository：https://huggingface.co/collections/lgy0404/memgui-agent
• arXiv Technical Paper：https://arxiv.org/pdf/2606.19926

Comparison of MemGUI-Agent with Similar Competing Products

Application Scenarios of MemGUI-Agent

• Cross-App Information Integration: Automatically collect information from e-commerce, maps, and social apps and aggregate it into note-taking software.
• Long Form Filling: Automatically complete complex registration or reimbursement processes based on multi-source information (email, contacts, calendar).
• Contact Management and Updating: Extract new phone numbers/emails from social media posts, automatically update contacts, and send text messages.
• Product Price Comparison and Recording: Browse multiple e-commerce platforms, record prices and specification parameters, and generate comparison documents.
• Automated Testing and RPA: Execute long-chain UI regression tests or enterprise-level process automation on mobile devices.