HyperEyes - A Parallel Multimodal Search Agent Jointly Launched by Xiaohongshu and Cambridge

What is HyperEyes

HyperEyes is a parallel multimodal search agent jointly launched by Xiaohongshu and the University of Cambridge. It pioneers the UGS paradigm, integrating visual grounding and retrieval into a single atomic action to achieve single-turn multi-entity concurrent search. HyperEyes adopts a dual-granularity efficiency-aware reinforcement learning framework. Across 6 benchmarks, the 30B version surpasses the strongest open-source models of the same scale by 9.9% in accuracy and reduces the number of tool-calling rounds by 5.3 times, achieving Pareto optimality in both accuracy and efficiency.

Main Features of HyperEyes

• Parallel Multimodal Search: Concurrently grounds and retrieves multiple entities in an image within a single turn, replacing the traditional serial crop-then-search pipeline.
• Unified Grounded Search: Embeds visual grounding boxes directly into the retrieval action parameters, with a single function call carrying multiple target boxes.
• Progressive Rejection Sampling: Purifies efficient trajectories under increasing round budgets, constructing 30,000 zero-redundancy parallel seed data.
• Dual-Granularity Efficiency-Aware RL: TRACE dynamically tightens the trajectory-level efficiency bar, while OPD injects token-level correction signals on failed trajectories.
• IMEB Benchmark Evaluation: Releases a 300-instance multi-entity visual evaluation set, paired with the CAS score to jointly quantify accuracy and search efficiency.
• Multi-Tool Collaboration: Integrates image search and text search tools, supporting concurrent acquisition of visual evidence and textual evidence.

Technical Principles of HyperEyes

• UGS Action Space Reconstruction: Transforms visual grounding from an independent preceding step into an embedded parameter of the retrieval action, physically opening up a single-turn multi-target concurrent pathway.
• Parallel Data Synthesis Pipeline: Synthesizes multi-entity queries by splicing multiple types of images, constructs multi-constraint intersection problems based on random walks on graphs, and eliminates shortcut solutions.
• TRACE Dynamic Reference Reward: Uses the current best trajectory as a dynamic bar, giving rewards only when the model is more efficient than the bar, automatically tightening the standard each round.
• OPD Asymmetric Policy Distillation: Activates a 235B teacher model when a trajectory's final answer is wrong, providing dense token-level supervision for failed trajectories and protecting efficient concurrent instincts.
• Joint Optimization Objective: GRPO combines trajectory-level efficiency rewards with token-level distillation loss to simultaneously optimize the accuracy and efficiency of the policy network.
• CAS Cost-Aware Scoring: Uses the formula Acc²×100/(N_tok+2N_tool+1) to unify accuracy, token consumption, and tool rounds into an efficiency metric.

How to Use HyperEyes

• Obtain Open-Source Resources: Visit the GitHub repository https://github.com/DeepExperience/HyperEyes 。
• Prepare the Base Model: Download and deploy Qwen3-VL-30B or 235B as the vision-language model backbone, ensuring GPU memory meets inference requirements.
• Configure External Retrieval Tools: Integrate image search and text search APIs (such as Bing Image Search, Google Custom Search) as external tools for the Agent to execute UGS parallel search.
• Input Multi-Entity Queries: Upload complex images containing multiple entities and input natural language questions. HyperEyes will automatically execute UGS unified grounded search, concurrently grounding and retrieving all targets in a single turn.
• View Parallel Search Results: The model returns structured retrieval results and the final answer, including visual evidence and textual evidence for each entity.
• Evaluating search efficiency: Quantitatively evaluate the combined efficiency of accuracy, token consumption, and tool invocation rounds through the CAS (Cost-Aware Score) metric

Core advantages of HyperEyes

• Efficiency leap: The 30B version averages only 2.2 tool invocation rounds, 1/5 that of the strongest open-source model of the same scale, achieving a 5.3x efficiency improvement.
• Leading accuracy: Surpasses the strongest open-source model of the same scale by 9.9% on 6 benchmarks, and the 235B version approaches Gemini-3.1-Pro with a 1.1% gap.
• Robust to noise: The parallel strategy avoids over-retrieval hallucinations, improving accuracy by 3.7%-5.8% in mixed true-false evidence tests.
• Eliminating error cascades: UGS eliminates serial pruning dependencies, so earlier localization bias no longer contaminates subsequent search results.
• Pareto optimal: Comprehensively dominant on the joint accuracy-efficiency frontier, with a CAS score 7.6 times that of the second-best open-source model.
• Full-stack efficiency reshaping: Performs底层 reconstruction from action space, data synthesis to RL training, systematically solving the serial dilemma.

HyperEyes project address

• GitHub repository：https://github.com/DeepExperience/HyperEyes
• arXiv technical paper：https://arxiv.org/abs/2605.07177

Comparison of HyperEyes with similar competitors

Application scenarios of HyperEyes

• Multi-person visual reasoning: Identify the identity, occupation, and historical events of specific people in group photos, and concurrently retrieve biographical information about multiple people.
• E-commerce product comparison: Concurrently search complex scene images containing multiple products to obtain price, brand, and review information.
• Cross-modal knowledge question answering: Answer complex queries involving relationships among multiple objects, landmarks, and artworks in images.
• News fact-checking: Concurrently retrieve news images containing multiple people and scenes to verify event authenticity and background information.
• Academic chart parsing: Perform parallel localization and content retrieval on paper screenshots containing multiple charts, formulas, and citations.
• Social media moderation: Concurrently search multiple entities in complex images to quickly determine violating or misleading content.