MemHarness - An agent memory reconstruction framework launched by Shanghai AI Lab and others

What is MemHarness

MemHarness is an LLM Agent memory reconstruction framework launched by Shanghai AI Lab in collaboration with universities such as Zhejiang University, Fudan University, and Shanghai Jiao Tong University. Existing memory-augmented Agents often inject retrieved historical experience directly into the context; if old experience does not match the current state, it instead leads to negative transfer. Inspired by the human memory reconstruction mechanism, MemHarness inserts an explicit critique and reconstruction step between retrieval and action, retaining, rewriting, or discarding historical experience in light of the current context, and is trained end-to-end via GRPO without additional manual annotation.

Main features of MemHarness

• Memory retrieval: The Agent generates a query based on the current observation and recalls the top-k relevant historical experiences and their source states from the Milvus vector memory store.
• Critique and reconstruction: A unified policy model compares the memory source state with the current state, critiques its applicability, rewrites the experience into state-aligned guidance, or deems it inapplicable and discards it.
• Action generation: Generates executable environment actions based on the reconstructed guidance or autonomous reasoning.
• Experience write-back and pruning: After each round of interaction, the trajectory is summarized into experience and written to the memory store, semantic deduplication is performed, and low-value memories are periodically pruned based on experience utility.
• End-to-end training: Jointly optimizes retrieval, reconstruction, and action generation via GRPO, without the need for independent annotated data for the reconstruction stage.

Technical principles of MemHarness

• Five-stage decision process:MemHarness decomposes memory-guided decision-making into five consecutive stages: environment observation, experience retrieval, memory critique, contextual memory reconstruction, and action generation, simulating the human cognitive process of retrieval—evaluation—reconstruction and replacing the static paradigm in which traditional Agents directly replay memories.
• Contextual memory reconstruction mechanism:The policy model concatenates the task description, current history, retrieved experience, and its source state into a reconstruction context, identifies differences by comparing the historical source state with the current state, retains transferable knowledge, rewrites mismatched content, or outputs <EMPTY> marker to reject that memory and fall back to autonomous reasoning.
• Unified policy model architecture: The three stages of retrieval query generation, memory critique and reconstruction, and executable action generation share the same policy model parameters, eliminating the need to separately train a value network or supervised data for the reconstruction module, so that memory utilization and decision reasoning are tightly coupled within the same model.
• GRPO end-to-end training:Since reconstruction guidance has no ground-truth annotation, MemHarness uses GRPO to perform end-to-end optimization of the full retrieval—reconstruction—action chain; the reward consists of sparse task outcomes and format rewards, and group-normalized advantages assign trajectory-level credit to all tokens, simultaneously training thinking, reconstruction, and action generation capabilities.

Follow WeChat and reply "open source" to join theAI open source project communication group

How to use MemHarness

• Clone the repository and create the environment: Execute git clone https://github.com/KnowledgeXLab/MemHarness.git and create a Conda environment for python==3.12 , then install vLLM, Flash Attention 2, and MemHarness itself in order.
• Deploy the embedding service: Via vllm serve BAAI/bge-m3 --port 8001 Start the BGE-M3 embedding model to provide vector encoding services for the Milvus memory bank.
• Install the target environment: Install the ALFWorld or WebShop interactive environment separately according to task requirements, and download the corresponding game files and pretrained detectors.
• Cold-start SFT (optional): Run scripts/build_*_coldstart_data.py to build cold-start data, and execute scripts/cold_start_sft.sh to align the model with the interaction format and memory summary format.
• GRPO end-to-end training: Run run_scripts/train_alfworld.sh or run_scripts/train_webshop.sh. The framework will automatically start the memory vector database, perform Agent retrieval, experience write-back, and pruning, and complete GRPO training.

MemHarness's core advantages

• Reconstruction is better than replay: Explicitly insert critique and reconstruction steps to transform static memory fragments into context-sensitive state-aligned guidance, avoiding negative transfer.
• Small models surpass large models: At the 7B parameter scale, it surpasses Gemini-2.5-Pro by 23.1% and 39.7% on ALFWorld and WebShop, respectively.
• Strong OOD robustness: In out-of-distribution scenarios, the average success rate reaches 85.9%, significantly higher than the 76.3% of original memory replay.
• Implicit reasoning enhancement: Even when memory is disabled at test time, the reconstruction training objective still improves the base policy success rate from 76.4% to 83.0%, fundamentally enhancing the Agent's intrinsic reasoning ability.
• No additional annotation required: The reconstruction capability naturally emerges through GRPO end-to-end training and does not rely on manually written reconstruction supervision data.

MemHarness's project address

• GitHub repository：https://github.com/KnowledgeXLab/MemHarness
• HuggingFace model library：https://huggingface.co/KnowledgeXLab/MemHarness
• arXiv technical paper：https://arxiv.org/pdf/2607.28272

Comparison of MemHarness with similar competitors

MemHarness's application scenarios

• Embodied intelligent household chores: In home environments such as ALFWorld, the Agent can reconstruct past object-fetching and cleaning experiences to adapt to different room layouts and complete complex household tasks.
• Autonomous online shopping: On e-commerce platforms such as WebShop, reconstruct historical shopping experiences into current product search and filtering strategies to improve the success rate of goal-oriented shopping.
• Intelligent customer service dialogue: After retrieving similar historical tickets, the customer service Agent reconstructs solutions to avoid directly applying old replies and giving irrelevant answers.
• Code-assisted development: The programming agent reconstructs past bug-fix experience, adapts it to the current code context, and provides precise fix suggestions rather than copying older solutions.
• Research experiment planning: The experiment agent reconstructs historical experiment parameters and results into optimal configuration suggestions for the current experimental conditions, reducing trial-and-error costs.