Analyze file similarity across a directory to find duplicates and related code.
When to Use
- User asks to find similar or duplicate files
- User wants to understand code overlap before refactoring
- User asks "what files are related to X?"
- User wants to detect copy-paste code
- Quick scan before detailed refactoring — use as a first pass, then
follow up with
indexion plan refactorfor actionable detail
Usage
# Basic similarity matrix (default: tfidf strategy)
indexion explore <path>
List format with threshold (most useful for finding duplicates)
indexion explore --format=list --threshold=0.7
Cluster similar files togetherindexion explore --format=cluster --threshold=0.6
JSON output for further processingindexion explore --format=json --threshold=0.5
Filter by extensionindexion explore --ext=.mbt --ext=.ts
Include/exclude patternsindexion explore --include='.ts' --exclude='_test.ts' src/
Filter out config noiseindexion explore --format=list --threshold=0.7
--include='*.mbt' --exclude='moon.pkg' cmd/indexion/
indexion explore --strategy=apted --format=list indexion explore --strategy=tsed --format=list
Hybrid strategy (auto-selects TF-IDF or APTED based on dataset size)indexion explore --strategy=hybrid --format=list
## Strategies
| Strategy | Description | Speed |
|----------|-------------|-------|
| `tfidf` (default) | TF-IDF token similarity | Fast |
| `hybrid` | Dynamic TF-IDF + APTED, auto-selects based on dataset size | Adaptive |
| `ncd` | Normalized Compression Distance | Fast |
| `apted` | All-Path Tree Edit Distance (function-level) | Slow |
| `tsed` | Tree Structure Edit Distance (function-level) | Slow |
## Output Formats
- `matrix` — Full similarity matrix (default, good for small sets)
- `list` — Sorted pairs above threshold (best for finding duplicates)
- `cluster` — Groups of similar files
- `json` — Machine-readable output
## Relationship to Other Commands
| Task | Use |
|------|-----|
| "What files are similar?" | `explore --format=list` |
| "Find nested for loops" | `grep "for ... for"` |
| "Find functions named sort" | `grep --semantic=name:sort` |
| "What exactly is duplicated?" | `plan refactor --threshold=0.9` |
| "Find code similar to a description" | `grep --semantic="similar:..."` |
## Workflow: explore → plan refactor
1. Run `indexion explore --format=list --threshold=0.7 <path>` for a quick scan
2. If high-similarity pairs exist, run `indexion plan refactor --threshold=0.9 <path>` for details
3. Fix duplicates, then re-run both to verify
## Dogfooding Lessons
- **moon.pkg files** inflate similarity scores (they all look alike) — exclude
with `--exclude='*moon.pkg*'` for meaningful results
- **96%+ similarity** between CLI files usually means duplicated utility functions
- **85-95% similarity** is often structural (same CLI patterns) — not always actionable
- **types.mbt files** showing 100% similarity is normal — type definition files
share structural patterns (pub struct + getters) that inflate TF-IDF scores