This skill allows you to systematically evaluate the performance of autonomous agents. Unlike traditional software, agents are non-deterministic and can reach a goal through multiple valid paths. Evaluation should focus on the outcomes (Outcomes) and the efficiency of the process.
Mental Model: Evaluation is not binary (pass/fail); it is a multidimensional measurement of the probability of success and the quality of reasoning.
🚩 Fragility and Autonomy
- Fragility: Low. These principles are robust and fundamental.
- Freedom: High in the design of rubrics specific to each project.
🚀 When to Activate
- When building test frameworks for agents.
- To validate whether a change in context engineering improves or worsens performance.
- When comparing different models (e.g. Flash vs Pro) for a specific task.
- To establish "Quality Gates" in production pipelines.
🧠 Core Concepts
1. The Three Pillars of Performance (95% of Variance)
Studies show that three factors explain almost all of an agent's performance:
- Token Budget (80%): More tokens (within the signal limit) usually allow for deeper exploration.
- Tool Calls (10%): The ability to interact with and observe the environment is critical.
- Model Choice (5%): Superior models (e.g. Gemini 3 Ultra) multiply the efficiency of the other two factors.
2. Evaluation Challenges
- Non-Determinism: Two identical executions can produce different but equally valid paths.
- Context-dependent failures: An agent may fail only when the context is very long or noisy.
- Composite Dimensions: Quality includes factual accuracy, tone, brevity, and tool efficiency.
🛠️ Implementation in Gemini Elite Core
Pattern: Multidimensional Rubric
Define criteria with specific weights according to their importance to the project.
| Dimension | Weight | Description |
|---|---|---|
| Factual Accuracy | 0.35 | The facts match the source of truth. |
| Completeness | 0.25 | Covers all requested aspects. |
| Efficiency | 0.20 | Uses the minimum number of tools necessary. |
| Format | 0.20 | Follows the expected JSON/Markdown schema. |
Pattern: Complexity Stratification
Divide your test set into levels:
- Simple: A single tool call, factual search.
- Medium: Multiple tools, comparison logic.
- Complex: Multi-step reasoning, significant ambiguity.
- Strategic: Extended interaction, deep data synthesis.
🔗 Integrations
context-fundamentals: Evaluation of how the model uses the provided context.context-degradation: Detection of performance drops in long contexts.advanced-evaluation: Technical implementation of "LLM-as-a-Judge".
📚 Internal References
references/metrics.md: Detailed definitions of metrics, implementation of rubrics and evaluation runners.