find-untested-sources-polyglot

Find Untested Sources (Polyglot)

Purpose

Coverage tools answer "which lines were executed?" — they require a green build and a passing test run, which is minutes-to-tens-of-minutes on a real repo. The question this skill answers is different and much cheaper:

Which source files have no test file referencing any of their declared symbols?

That's the question an agent asks before writing a new test — and it can be answered statically in a few seconds by parsing every recognized source file with tree-sitter, with no build, no dependency resolution, no compilation.

This is the polyglot sibling of the C# find-untested-sources skill. The output schema is intentionally compatible so the same prompt patterns can consume either tool.

When to Use

The repository is not exclusively C#, or you want a tool that works uniformly across multiple languages without per-language plumbing.
User asks "where should I add tests?", "which files have no tests?", "find untested code", "give me a test gap list", "what's the next file to test".
Before invoking a test-generation agent, to produce a prioritized worklist.
After generating tests, to verify each new test file pairs to a source file.

When Not to Use

C#-only repo — prefer find-untested-sources. Its Roslyn-based namespace disambiguation is strictly better than this skill's identifier overlap on duplicated short names like Settings or Context.
Line/branch coverage — use language-native coverage tooling.
Are existing tests strong? — use test-gap-analysis or assertion-quality.

Inputs

Input	Required	Default	Description
Repo root	Yes	—	Directory to scan recursively.
`--lang LANG`	No	all	Restrict to a language (repeatable). One of `python`, `typescript`, `tsx`, `javascript`, `go`, `java`, `rust`, `csharp`, `ruby`.
`--limit-untested N`	No	0 (no limit)	Truncate the untested list to N entries.
`--include-tested`	No	off	Include `tested_sources` in the payload (large).

Prerequisites

Python 3.10+.
pip install tree-sitter-language-pack (single self-contained wheel that bundles parsers for 300+ languages and the high-level process() API used here). No native build, no per-language grammar install.

Usage

# From the skill folder
python scripts/find_untested_sources.py <repo-root>

# Restrict to a language
python scripts/find_untested_sources.py <repo-root> --lang python --lang typescript

# Truncate the report (top 20 by declared API surface)
python scripts/find_untested_sources.py <repo-root> --limit-untested 20 > pairing.json

# Iterate, highest-API-surface first
$report = Get-Content pairing.json | ConvertFrom-Json
$report.untested_sources | Select-Object -First 10 path, declaration_count, suggested_test_path

Diagnostics go to stderr; JSON goes to stdout.

Output Schema

{
  "repo_root": "<absolute path>",
  "summary": {
    "source_files": 3138,
    "test_files": 761,
    "tested_source_files": 1419,
    "untested_source_files": 1719,
    "orphan_test_files": 15,
    "languages": ["csharp"]
  },
  "untested_sources": [
    {
      "path": "src/Foo/Bar.cs",
      "language": "csharp",
      "declaration_count": 8,
      "declarations": ["Bar", "BarOptions", "IBar", "..."],
      "suggested_test_path": "src/Foo/BarTests.cs"
    }
  ],
  "orphan_tests": [
    { "path": "tests/SomeIntegrationTest.cs", "language": "csharp" }
  ]
}

Pass --include-tested to additionally emit tested_sources (same shape as untested_sources but with a covering_tests array instead of a suggested path). Omitted by default to keep the payload small for LLM consumption.

How It Works

File discovery — recursive directory walk pruning common build/vendor dirs (bin, obj, node_modules, target, dist, build, vendor, __pycache__, .venv, .git, etc.). Skips generated files (.d.ts, .g.cs, .Designer.cs, _pb2.py, *.min.js, AssemblyInfo.cs, ...).
Language detection — tree_sitter_language_pack.detect_language_from_path maps the extension to one of the supported languages. Unknown extensions are skipped silently.

Test-vs-source classification — per-language path heuristics:

Language	Test rule
Python	path contains `tests/` or `test/`; or filename starts with `test_` or ends with `_test.py`; or `conftest.py`.
JS/TS/TSX	path contains `__tests__`, `tests`, `test`, `spec`, or `e2e`; or filename contains `.test.` or `.spec.`.
Go	filename ends with `_test.go` (Go's standard convention).
Java	path contains `test` or `tests`; or filename ends with `Test.java` / `Tests.java`.
Rust	path contains `tests/` or `benches/`.
C#	path contains `tests/`; or project segment ends with `.Tests`, `.Test`, `.UnitTests`, `.IntegrationTests`; or filename ends with `Tests`/`Test`.
Ruby	path contains `spec/`, `test/`; or filename ends with `_spec.rb` / `_test.rb`.

Per-file extraction — tree_sitter_language_pack.process(text, ProcessConfig(language=lang, structure=True, imports=True, symbols=True)) returns:
- structure — top-level declared items (functions, classes, methods, traits, ...) with their names. Used as the declared-symbol set.
- imports — raw import statements (e.g. from foo import bar, import "pkg/util", using System.IO;, use crate::foo::Bar;).
- symbols — flat declared-name list, unioned with structure (acts as a fallback when structure is empty, and broadens coverage when both are populated; declaration counts may exceed pure structure parsing).
Pairing — for each test file, union the results of:
- Import resolution (per language):
  - Python: from pkg.mod import x → pkg/mod.py or pkg/mod/__init__.py.
  - TS/JS: relative ./foo / ../bar → with .ts/.tsx/.js/.jsx and /index.* candidate paths.
  - Go: "path/to/pkg" → any source file whose final path segment matches pkg.go in the index.
  - Java: import a.b.C; → a/b/C.java.
  - Rust: use a::b::C; → b.rs or C.rs (best-effort, no module tree).
  - Ruby: require 'foo/bar' → foo/bar.rb.
  - C#: using maps to namespaces, not files; intentionally a no-op — falls through to identifier overlap below.
- Identifier overlap — every word-like token in the test source is looked up in the source index of declared names (length ≥ 4 to keep noise down). Any source whose declared name appears as a token in the same-language test is paired.
JSON emit — untested_sources is ordered by declaration count descending so the highest-API-surface gap appears first.

Limitations

This is a static, parse-only heuristic. It deliberately trades a small amount of accuracy for orders-of-magnitude lower cost than coverage. Known gaps:

Reflection / DI-resolved types that a test only references through a string name or container resolution don't appear in the identifier scan.
C# specifically: namespace disambiguation is the C# tool's strength; this polyglot version intentionally skips it. If you have a .NET-only repository, prefer the Roslyn-based find-untested-sources.
Short identifier names (< 4 chars) are dropped from the overlap index to avoid noisy pairings on names like id, db, Tag.
Cross-language tests (Python tests driving a Go binary, etc.) are recorded as orphan tests since same-language pairing is the rule.
Monorepo path aliases (TS path mapping, Java module-info) are not resolved; the suffix-match fallback may pick the wrong source if two files share a trailing path segment in different sub-projects.

For these cases, run actual coverage on the unpaired candidates the agent has already triaged.

Outputs the agent should consume

untested_sources[*].path — pick the next source file to test (highest declaration_count first).
untested_sources[*].suggested_test_path — drop-in target for the new test file using the per-language convention.
(With --include-tested) tested_sources[*].covering_tests — verify a newly written test file lands in the list for the intended source.
orphan_tests — tests that don't appear to reference any same-language source file; useful for triaging stale tests or integration-only tests.