Primary reference: See
CONTRIBUTING.mdand.github/PULL_REQUEST_TEMPLATE.mdin this repository.
Consistent reviews protect the quality of SkillOverflow. Every skill PR must pass this checklist before merge.
When asked to review a skill PR, work through every section below and produce a structured review comment. Do not approve if any critical item fails.
How to review
- Read the SKILL.md being added or changed
- Work through the checklist below section by section
- Produce a review comment using the output template at the end
- Mark as Approved, Request Changes, or Comment accordingly
Section 1: Frontmatter quality
Check the description field — the most critical part:
Ask yourself: "Would this description match if a developer typed [realistic prompt]?"
Test it against 3 realistic prompts:
- A prompt that SHOULD trigger this skill
- A prompt that SHOULD NOT trigger this skill (verify no false positive)
- A prompt that already triggers another skill (verify no collision)
| Check | Pass? |
|---|---|
name field present |
|
description field present |
|
| Description is specific — not vague ("use for CDS") | |
| Description contains real trigger keywords a developer would type | |
| Description doesn't collide with an existing skill's trigger zone |
Flag immediately if description is vague. Example:
❌ Use when working with SAP BTP — too broad, triggers on everything BTP-related
✅ Use when configuring BTP Destination Service: OAuth2ClientCredentials, OAuth2SAMLBearerAssertion, principal propagation, Cloud Connector, or reading destinations from CAP using cds.connect.to — specific and testable
Section 2: SAP documentation reference
Every skill must reference official SAP documentation. Accepted sources:
https://cap.cloud.sap/docs/— capire (CAP)https://ui5.sap.com/— SAPUI5 documentationhttps://help.sap.com/docs/— SAP Help Portal (BTP, Fiori, etc.)https://sap.github.io/cloud-sdk/— SAP Cloud SDKhttps://sap.github.io/ai-sdk/— SAP AI SDK
| Check | Pass? |
|---|---|
| Primary reference link is present at top of skill body | |
| Reference points to official SAP documentation (not blogs or community posts) | |
| Reference is relevant to the skill content |
Section 3: Code examples
| Check | Pass? |
|---|---|
| All code examples are copy-pasteable (not pseudocode) | |
| Code uses correct syntax for the technology | |
| No deprecated APIs used without explicit deprecation notice | |
| Examples match what the referenced SAP documentation shows |
Section 4: Anti-pattern section
| Check | Pass? |
|---|---|
| "Common mistakes to avoid" section exists | |
| Each ❌ has a specific bad pattern (not vague "don't do X") | |
| Each ❌ has a corresponding ✅ replacement | |
| Anti-patterns are real — observed in agent output or real projects |
Section 5: Eval quality (most important)
Check for eval bias — the most common review failure:
Read the eval prompt. Ask: "Could a developer type this exact prompt without having read the skill?"
If the prompt contains phrases from the skill instructions, it is biased.
❌ Biased — mirrors skill wording:
Create an entity using the cuid aspect and the managed aspect from @sap/cds/common,
with a localized title field and a Currency association
✅ Unbiased — natural developer language:
Add a Products entity with a translatable name, price, and currency to our CAP app
| Check | Pass? |
|---|---|
At least 1 eval added to tests/skillOverflow-evals.json |
|
| Eval prompt is unbiased — could be typed by a developer unfamiliar with the skill | |
| Assertions are specific and observable | |
| Assertions test outcomes, not whether the agent repeated skill instructions | |
Matching entry added to skill-test-checklist.html |
Section 6: Repository consistency
| Check | Pass? |
|---|---|
Skill lives flat under skills/<skill-name>/ with correct metadata.category |
|
| Skill name is kebab-case | |
| README.md skills table updated | |
| Smoke test passes (should be in PR description) | |
| No old references (cap-skills, etc.) in the new skill |
Review output template
Use this format for your review comment:
## SkillOverflow Skill Review
**Skill**: `<skill-name>` ([category])
**Verdict**: ✅ Approved / ⚠️ Request Changes / 💬 Comment
---
### Frontmatter
[PASS / FAIL + notes]
### SAP documentation reference
[PASS / FAIL + notes]
### Code examples
[PASS / FAIL + notes]
### Anti-patterns
[PASS / FAIL + notes]
### Eval quality
[PASS / FAIL + notes on bias if any]
### Repository consistency
[PASS / FAIL + notes]
---
### Required changes (before merge)
- [ ] [Specific change required]
### Suggestions (optional)
- [Non-blocking improvement suggestion]
Common mistakes to avoid
-
❌ Approving a skill with a vague description — it will never trigger reliably
-
✅ Request changes if the description isn't testable against 3 realistic prompts
-
❌ Missing biased evals — they give false confidence the skill works
-
✅ Always read the eval prompt critically before approving
-
❌ Accepting blog posts or community links as primary references
-
✅ Only official SAP documentation URLs count as primary references
-
❌ Approving without checking for collision with existing skill descriptions
-
✅ Test that the new description doesn't fire on prompts meant for other skills
-
❌ Rubber-stamping the PR because the skill "looks fine"
-
✅ The review is the last line of quality defense — take it seriously