SuperAPI AI Directory

review-usability

Coding & development

View repository

Updated 10-16 04:54

About this skill

Usability review

Monty exists so LLMs can write Python that calls tools. Real usage is therefore the most common patterns, not exotic corners — and a divergence in a common pattern is the worst kind of bug, because the model has no way to know it must write something else.

Think hard on this one.

git diff origin/main...HEAD
  1. For each feature the branch touches, list the idioms a model reaches for first — the obvious method, argument form, combination with another builtin. Include ones the branch does not handle; that's where the gaps are.
  2. Write real test files in playground/ (see python-playground), named recognisably.
  3. Run each under both and diff:
uv run playground/test_thing.py        # CPython
cargo run -- playground/test_thing.py  # Monty
  1. Prioritise silent divergence — same code, different result — over a clean AttributeError. A missing feature that raises is recoverable; a wrong answer isn't.

An undocumented divergence is also a ./limitations/ finding.

Report

Per divergence: the code, CPython's output, Monty's output, how likely a model is to write it. Then unsupported-but-common idioms with the error the user sees, and briefly what worked — it bounds the review. Leave the playground files in place.

Report only, unless the user asks for fixes.