defluff

Purpose

defluff is a deterministic "fluff detector" specifically designed to identify filler phrases and clichés in AI-generated text. Unlike common "AI detectors," it does not judge whether text was generated by AI, but answers a determinable question: does this text contain removable filler content? Whether the text is written by a human or an LLM, as long as it contains phrases like "at the end of the day," it will be flagged.

Main Capabilities

• Deterministic matching: Matches against a carefully curated phrase list, with no model and no API key, producing completely identical results on every run.
• Exact snippets and scoring: Returns the exact flagged text snippets (with categories) and a 0-1 "filler density" score (slop_score = flagged word count ÷ total word count, weighted by category).
• Multi-format awareness: Strips code blocks, inline code, URLs, and Markdown markup before matching.
• Whole-word matching: For example, "foster" will not trigger inside "fostering."
• Overlap handling: Longest match takes priority, and each word is counted only once (such as when "at the end of the day" overlaps with "end of the day").
• Customizable lexicon: Supports user-level overrides (~/.config/defluff/), project-level overrides (.defluff/ at the git root), custom phrase files (.txt / .md / .json), and preset domain packs.
• MCP server: Exposes three tools, allowing any MCP-compatible agent to self-check text without additional wiring.
• CI integration: Provides exit codes, JSON output, and character-offset snippets, making it easy to integrate into CI, pre-commit, and editor tools.
• Pinnable lexicon hash: Prints the lexicon content hash on every run (such as lexicon: 2cc05ba84457), ensuring the scoring standard remains unchanged across runs.

Usage

Installation

• Run with zero installation via uvx (recommended):uvx defluff or uvx defluff[mcp] The first run will pull the package and the mcp add-on.
• Install via Homebrew on macOS/Linux.
• Install directly and run the entry point.
• Published as an MCP server in the MCP Registry under the name io.github.ahmedak/defluff.

Basic usage

• Point it at a changelog, document, or agent output to get a list of filler phrases to remove, along with a CI exit code and a pinnable score.
• Common scenarios include: agent self-correction (call slop_detect on a draft and revise the flagged phrases), CI gating (fail the build when AI-drafted changelogs or docs are full of filler words), writing assistant feedback (highlight the exact phrases an editor would delete), fine-tuning reward components (experimental, easily gamed if used alone).

Lexicon coverage and versioning

• User overrides: ~/.config/defluff/, machine-wide, not committed.
• Project overrides: .defluff/ at the git root, per repository, can be committed for the team.
• Writes are atomic and cross-process locked; corrupted override files warn and are skipped, and detect() never crashes.
• Every resolved lexicon (built-in + overrides + package) carries a short content hash, available via SlopReport.lexicon_version.

Output interpretation and threshold setting

• Spans: exact filler phrases, with categories, deterministic and reliable, act on these.
• Score: weighted filler density, where 0.20 roughly means "one fifth of the text is filler from the list", usually 0.0-1.0; it is a tripwire that triggers the CI exit code, not a quality quantifier.
• The default threshold of 0.08 was hand-picked provisionally and is not calibrated on a labeled corpus. For hard CI gating, it is recommended to set your own threshold and first suppress domain vocabulary.

Usage with pre-commit and reward loops

• Can be used with pre-commit.
• In reward loops (experimental): can serve as a small component of a reward mix, but used alone it is easily gamed and needs to be paired with real quality signals (human or LLM judges).

Applicable scenarios

• Agent self-correction, CI gating, writing assistant feedback, fine-tuning reward components.

Limitations

• List-based matching: does not understand text and cannot detect novel buzzwords (such as "operationalize the ideation funnel"); this is the cost of determinism and reproducibility. For semantic detection, pair it with an LLM judge.
• Domain terms are context-dependent: for example, "leverage" may be real content in a financial document; you need to read the flagged spans and use defluff lexicon rm to suppress false positives.
• Span offsets are based on the cleaned text: because code blocks, inline code, URLs, and Markdown markup are stripped, the offsets do not align with the original document; match based on span.text.
• The custom category is read-only: phrases added via a --lexicon file fall into custom, but defluff lexicon add --category custom will be rejected.
• English only (v0).
• Short texts (< 20 words) get low_confidence: true, with a denominator floor of 20, to avoid a two-sentence input showing 100% rubbish because of a single phrase.

Evaluation and contribution

• On 50 hand-labeled examples, precision at the default threshold is reproducible:python eval/score.py eval/validation.jsonl. Misses are mainly novel buzzwords unseen by the lexicon; recall is 1.00 for filler words in the list, but cannot cover open-ended new words.
• How to contribute: add missed filler phrases to src/defluff/data/lexicon-v1.json, run pytest, submit a PR with one or two real examples. See CONTRIBUTING.md for details.