SuperAPI AI Directory

indexion-explore

General & other

View repository

Updated 10-16 06:43

About this skill

indexion explore

Analyze file similarity across a directory to find duplicates and related code.

When to Use

  • User asks to find similar or duplicate files
  • User wants to understand code overlap before refactoring
  • User asks "what files are related to X?"
  • User wants to detect copy-paste code
  • Quick scan before detailed refactoring — use as a first pass, then follow up with indexion plan refactor for actionable detail

Usage

# Basic similarity matrix (default: tfidf strategy)
indexion explore <path>
List format with threshold (most useful for finding duplicates)

indexion explore --format=list --threshold=0.7

Cluster similar files together

indexion explore --format=cluster --threshold=0.6

JSON output for further processing

indexion explore --format=json --threshold=0.5

Filter by extension

indexion explore --ext=.mbt --ext=.ts

Include/exclude patterns

indexion explore --include='.ts' --exclude='_test.ts' src/

Filter out config noise

indexion explore --format=list --threshold=0.7
--include='*.mbt' --exclude='moon.pkg' cmd/indexion/

Function-level tree edit distance (more precise, slower)

indexion explore --strategy=apted --format=list indexion explore --strategy=tsed --format=list

Hybrid strategy (auto-selects TF-IDF or APTED based on dataset size)

indexion explore --strategy=hybrid --format=list


## Strategies

| Strategy | Description | Speed |
|----------|-------------|-------|
| `tfidf` (default) | TF-IDF token similarity | Fast |
| `hybrid` | Dynamic TF-IDF + APTED, auto-selects based on dataset size | Adaptive |
| `ncd` | Normalized Compression Distance | Fast |
| `apted` | All-Path Tree Edit Distance (function-level) | Slow |
| `tsed` | Tree Structure Edit Distance (function-level) | Slow |

## Output Formats

- `matrix` — Full similarity matrix (default, good for small sets)
- `list` — Sorted pairs above threshold (best for finding duplicates)
- `cluster` — Groups of similar files
- `json` — Machine-readable output

## Relationship to Other Commands

| Task | Use |
|------|-----|
| "What files are similar?" | `explore --format=list` |
| "Find nested for loops" | `grep "for ... for"` |
| "Find functions named sort" | `grep --semantic=name:sort` |
| "What exactly is duplicated?" | `plan refactor --threshold=0.9` |
| "Find code similar to a description" | `grep --semantic="similar:..."` |

## Workflow: explore → plan refactor

1. Run `indexion explore --format=list --threshold=0.7 <path>` for a quick scan
2. If high-similarity pairs exist, run `indexion plan refactor --threshold=0.9 <path>` for details
3. Fix duplicates, then re-run both to verify

## Dogfooding Lessons

- **moon.pkg files** inflate similarity scores (they all look alike) — exclude
  with `--exclude='*moon.pkg*'` for meaningful results
- **96%+ similarity** between CLI files usually means duplicated utility functions
- **85-95% similarity** is often structural (same CLI patterns) — not always actionable
- **types.mbt files** showing 100% similarity is normal — type definition files
  share structural patterns (pub struct + getters) that inflate TF-IDF scores