pull

$pull

Distill a file or directory of files into a pull artifact (.pull.md) that preserves all decision-relevant information while reducing token consumption. CocoHarvest uses pull files automatically when stage inputs exceed the configured token threshold.

Preconditions

.cocoplus/ must be initialized (run $pod init first)
<target> file or directory must exist

Arguments

Argument	Description
`<target>` (required)	File path or directory to distill. Eligible: `.md`, `.json`, `.sql`, `.txt`
`--validate` (optional)	Run round-trip reconstruction test after distillation
`--human` (optional)	Produce human-readable narrative summary instead of LLM-dense pull file

Two output modes:

Default (no flag): Produce <target>.pull.md — machine-dense, LLM-optimized, verbatim decision values. Used automatically by CocoHarvest.
--human: Produce <target>.pull-human.md — prose narrative for stakeholder consumption. Not used by CocoHarvest. Never overwrites the .pull.md machine output.

Step-by-Step Behavior

`$pull <target> --human`

Verify initialization and target (same as default mode steps 1–2).
Spawn Sonnet narrative subagent: Reads <target> and produces a prose summary optimized for a non-technical stakeholder audience. Rules:
- Lead with the most important conclusion or finding (inverted pyramid)
- Use plain language — no SQL, no JSON field names unless unavoidable
- Express numeric values in context: not "0.88" but "88% accuracy — above the 85% threshold set in discuss.md"
- Maximum 500 words unless source content genuinely requires more
- Do not include raw tables — translate to sentences or brief bullets

Write output to <target>.pull-human.md. Include header:

---
pull_source: [source file path]
pull_timestamp: [ISO8601]
pull_mode: human
audience: stakeholder
---

Do NOT update .cocoplus/pull/manifest.json for human-mode output — human pull files are not CocoHarvest artifacts.
Display: "Human summary written to <target>.pull-human.md" and show the full output.

`$pull <target> [--validate]`

Verify initialization: Check .cocoplus/ exists. If not, output: "CocoPlus not initialized. Run $pod init first." and exit.
Validate target: Check <target> exists. If not, output: "Target not found: <target>. Check the path and try again." and exit.
If target is a directory: Enumerate eligible files (.md, .json, .sql, .txt). Process each file independently using steps 4–7. After all files processed, produce a directory manifest at <target>/pull-manifest.md listing each file, its compression ratio, and reliability (if --validate was used).
For each target file — Structure Analysis (Haiku subagent):

Spawn a structure analysis subagent (Haiku model, background: false) that reads the target file and produces a JSON structure map identifying:
- Sections present (headings, blocks, named groupings)
- Content type of each section: schema_definition, query_result, narrative_analysis, tabular_data, decision_conclusion, code_block, metadata
- Decision-relevance classification per section: decision_bearing (thresholds, field names, criteria, constraint values — must be preserved verbatim) vs background (context and reasoning that supports decisions but would not be directly queried)
If the structure analysis subagent fails: proceed to step 5 with direct distillation (no structure map). Note in the pull file header: structure_guided: false.
For each target file — Distillation (Sonnet subagent):

Spawn a distillation subagent (Sonnet model, background: false) that reads the structure map (if available) and the target file. Apply these distillation rules in order:

Rule 1 — Verbatim preservation (non-negotiable): Content classified as decision_bearing — accuracy thresholds, schema field names, evaluation criteria, constraint values, model names, warehouse assignments — is copied verbatim into the pull file. No paraphrasing, no rounding, no synonym substitution.

Rule 2 — Narrative compression: Narrative explanation sections are compressed to their conclusion:
- Before: Three paragraphs explaining why a threshold was chosen
- After: "The analysis determined that [conclusion — e.g., 88% accuracy threshold]." The conclusion must be extracted from the narrative, not inferred.
Rule 3 — Deduplication: Content repeated across sections (e.g., a column name referenced seven times in a result set narrative) is deduplicated to a single canonical reference with a note: "[field-name] — appears [N] times in source; referenced once here."

Rule 4 — Tabular data compression: Tabular data with more than 20 rows is replaced with:
- First 5 rows (verbatim)
- Statistical summary: row count, column-level min/max/mean for numeric columns, distinct-value count for categorical columns
- Note: "Source contains [N] rows. First 5 rows shown above."
Write distilled output to <source-name>.pull.md alongside the source file. Include a header block at the top of the pull file:
```
---
pull_source: [source file path]
pull_timestamp: [ISO8601]
original_tokens_estimate: [N]
pull_tokens_estimate: [N]
compression_ratio: [N%]
structure_guided: [true|false]
reliability: [pending|high|low]  ← updated by --validate step
---
```
Compression check: If the pull file is larger than the source file (no compression achieved), write a note in the header: distillation_outcome: no_reduction. Use the source file as-is for CocoHarvest — do not force distillation that inflates the artifact.

Update pull manifest: Append an entry to .cocoplus/pull/manifest.json (create file if absent):

{
  "source": "<source path>",
  "pull": "<pull file path>",
  "timestamp": "<ISO8601>",
  "compression_ratio": "<N%>",
  "reliability": "pending|high|low",
  "distillation_outcome": "compressed|no_reduction"
}

`--validate` — Round-Trip Reconstruction Test

Run after distillation (steps 4–7 complete). Spawns three sequential subagents:

Step V1 — Probe Question Generator (Haiku): Reads the SOURCE file (not the pull file) and generates exactly 10 probe questions designed to cover:

Primary decision-relevant facts (≥4 questions)
Key constraints stated in the source (≥2 questions)
At least one question from each major section of the source

Write probe questions to <target>.pull-probes.json (temporary file, deleted after validation).

Step V2 — Verification Agent (Sonnet): Reads ONLY <target>.pull.md (not the source). Answers all 10 probe questions using only the pull file content. Writes answers to temporary memory (not a file).

Step V3 — Ground Truth Extractor (Sonnet): Reads the SOURCE file. Answers the same 10 probe questions. Writes answers to temporary memory.

Comparison: Compare V2 answers against V3 answers. A match is defined as: same numeric value (within 1% tolerance for percentages), same field/function name, same yes/no conclusion. Compute match score: matched_answers / 10 * 100.

Append validation summary to the pull file header:

reliability: high   ← score ≥85%
validation_score: [N]%
divergent_probes: [list of question numbers that did not match, or "none"]

Or if score < 85%:

reliability: low
validation_score: [N]%
divergent_probes: [list of question numbers with brief description of divergence]

Update manifest with final reliability value.

Delete <target>.pull-probes.json (temporary file).

If score < 85%, output warning:

⚠ Pull reliability is low ([N]%). CocoHarvest will not auto-use this file — original will be loaded instead.
  Divergent probes: [list]
  Review the pull file and re-run $pull if needed.

Automatic Use by CocoHarvest

When CocoHarvest prepares stage inputs, it checks each input file's estimated token count:

If estimated tokens exceed cocoHarvest.pullThreshold (default: 8,000 tokens) in .cortex-plugin/plugin.json:
1. Check for <file>.pull.md counterpart
2. If found AND reliability is high: use the pull file; log: "Using pull file for [stage-id]: [source] → [pull file] ([compression]%)"
3. If found AND reliability is low: use the source; log warning: "Pull file exists but reliability is low — loading original for [stage-id]"
4. If not found: invoke $pull <file> automatically; use result; log: "Auto-pulled [file] for [stage-id]"

This behavior is transparent — stage logs always record whether source or pull file was used.

Storage

Pull artifacts: <source-name>.pull.md alongside source files
Pull manifest: .cocoplus/pull/manifest.json
Pull files are gitignored by default (they are derived artifacts)
To commit pull files: $pod config set pull.commit=true

Error Cases

Target not found: Output: "Target not found: <target>. Check the path and try again."
Structure analysis subagent fails: Proceed with direct distillation; note structure_guided: false in header
Distillation produces file larger than source: Note distillation_outcome: no_reduction in header; do not write an inflated pull file — keep source as-is
Validation V1/V2/V3 subagent fails: Output: "Validation could not complete — [which step failed]. Pull file written but reliability is unverified (reliability: pending)."
Cannot write pull manifest: Log warning; do not block the distillation output

Exit Criteria

This skill is complete when:

<target>.pull.md is written alongside the source (or distillation_outcome: no_reduction is recorded)
.cocoplus/pull/manifest.json is updated with the new artifact entry
If --validate: validation summary is appended to pull file header and reliability is set to high or low
Output is surfaced showing: original token estimate, pull token estimate, compression %, and reliability (if validated)

Anti-Rationalization

Do NOT:

Paraphrase numeric values, thresholds, field names, or constraint values in default mode — Rule 1 is non-negotiable
Run the validation V2 agent before V1 has produced the probe questions
Allow V2 to read the source file — it reads only the pull file (circular test prevention)
Use the same agent to both generate probe questions and answer them
Mark reliability as high for scores below 85%
Auto-use a low reliability pull file in CocoHarvest — always fall back to source
Delete the source file — CocoPull only writes new files alongside sources
Use --human output in CocoHarvest — human pull files are stakeholder artifacts, not context files
Overwrite an existing .pull.md when running --human — the two modes produce separate files