ctx-flatten (ContextStrategies)

ctx-flatten (published as ctx-flatten, CLI ctx) converts a repo into a structured Markdown context file you can paste into an LLM, then adds “guardrails” tools around that workflow: focusing context with AI, patching Python symbols with drift checks, duplicate detection, dependency slicing, and git-history export.

At a glance (from the codebase)

Measured numbers (2026-01-16)

• Package version: 5.0.0 (pyproject.toml).
• Python requirement: >= 3.9.
• cloc (core surface: src/, tests/, docs/, README.md, pyproject.toml): 12,454 LOC across 105 files.
• cloc (full repo, excluding .git,node_modules,dist,build,.venv): 12,892 LOC across 115 files.
• CLI command modules (src/ctx_flatten/commands/*.py): 11 files (10 excluding __init__.py).
• Prompt templates (src/ctx_flatten/prompts/*.md): 4.
• Python test files: 15 (tests/**/*.py).

Problem

LLM context is expensive and easy to waste: copy files by hand, miss an import, paste too much, then burn tokens on irrelevant code.

Even worse, AI-generated patches are brittle when the target changes between context capture and application. Without drift checks, you can end up editing the wrong symbol or applying a change to stale code.

Constraints

• Respect .gitignore by default when collecting files.
• Redact obvious secrets so generated context is safer to share.
• Keep AI features optional and explicit (core flattening works without LLM access).
• When patching code, avoid “silent drift” edits to the wrong symbol.

Solution (what shipped)

The system is built around durable artifacts:

• The flattening core is deterministic: scan files (respecting .gitignore), read content, redact secrets, then emit Markdown.
• AI features are optional and explicit: they operate on the flattened context, not on your filesystem blindly.
• Patching is scoped and guarded: it targets named Python symbols and checks for drift before applying edits.

The goal is inspectable workflows: context files, reports, and history artifacts you can read and version, not a black-box “agent” that edits your repo blindly.

Architecture

flowchart TB
  subgraph Surfaces["Interfaces"]
    CLI["CLI (Typer)"]
    TUI["TUI (Textual)"]
  end
 
  Surfaces --> CORE["Core engine<br/>discover • filter • redact • render"]
  CORE --> OUT["Outputs<br/>context.md • reports • snapshots"]
 
  CORE --> AI["AI layer (optional)<br/>focus • patch"]
  AI --> OR["OpenRouter"]
 
  CORE --> ANALYSIS["Static analysis (optional)"]
  CORE --> HIST["History export (optional)"]

flowchart LR
  Flatten[Flatten repo] --> Focus[Focus context (optional)]
  Focus --> Patch[Patch Python symbols (optional)]
  Patch --> Analyze[Analyze + report]
  Analyze --> History[History export]

Evidence (placeholders)

• Screenshot (TODO): case-studies/ctx-flatten/flatten-output.png
  • Capture: a small excerpt of a generated context.md showing the “File Structure” section and one file block.
  • Alt text: “Generated context markdown showing repo tree and an inlined file.”
  • Why it matters: demonstrates the core “repo → single context file” output format.
• Screenshot (TODO): case-studies/ctx-flatten/tui-home.png
  • Capture: the Textual TUI home screen (command selection / navigation).
  • Alt text: “Textual TUI showing ctx-flatten commands and navigation.”
  • Why it matters: demonstrates a second surface for the same core engine.
• Screenshot (TODO): case-studies/ctx-flatten/patch-drift-error.png
  • Capture: a terminal output example showing PatchDriftError on a stale symbol span.
  • Alt text: “CLI output showing a patch rejected due to drift detection.”
  • Why it matters: supports the claim that patching fails safely on drift by default.
• Screenshot (TODO): case-studies/ctx-flatten/history-json.png
  • Capture: a snippet of a generated per-commit history JSON file (commit meta + file changes).
  • Alt text: “History export JSON snippet showing commit metadata and file change summary.”
  • Why it matters: proves the tool produces versionable history artifacts.

Deep dive: the guardrails

1) Deterministic flattening you can trust

The “context file” is produced by DirectoryFlattener (src/ctx_flatten/core/flattener.py) which orchestrates:

• scanning (src/ctx_flatten/core/scanning.py)
• reading + redaction (src/ctx_flatten/core/reading.py)
• markdown emission (src/ctx_flatten/processing/render_markdown.py)

The output is inspectable, versionable Markdown, not a proprietary format.

2) AI focus, but bounded

The AI client (src/ctx_flatten/ai/client.py) uses pydantic-ai and targets OpenRouter, but it is an optional dependency group. If you don’t install [ai], the core flattening still works.

3) Drift-checked patching (Python-only, by design)

ctx patch is intentionally scoped to top-level Python symbols:

• It uses the Python ast module to locate symbol spans (src/ctx_flatten/patching/ast_utils.py).
• Before applying an AI edit, it verifies the symbol still exists in the expected span; otherwise it raises PatchDriftError unless you pass --force.

That’s the difference between “helpful edits” and “surprise refactors”.

4) History export without rewriting your working tree

The history tool (src/ctx_flatten/commands/history.py) walks commits via git commands, exports JSON snapshots per commit, and can compute bounded unified diffs for changed files. It never checks out your working directory as part of the process.

Tech stack

Key decisions

• One deterministic core, multiple surfaces: the same engine drives CLI + TUI.
• Optional AI dependency group: AI features are isolated behind extras and can be disabled entirely.
• Symbol-scoped patching: only patch named top-level Python symbols to keep edits bounded.
• Drift detection before write: fail closed unless explicitly overridden (--force).

Tradeoffs

• Drift detection is heuristic, it can block valid patches when code moves significantly.
• Patching is Python-focused (top-level symbols) rather than “edit anything anywhere”.
• A single flattened file can still be too large for some model limits, requiring focus/slicing.

Security and reliability

• Secret redaction runs during reading to reduce accidental leakage in context output.
• File path validation is used for patch operations to prevent writing outside the repo root.

Testing and quality

• Tests live under tests/ and cover core behaviors (flattening, patching, focus stub flows).

Outcomes

• Faster, safer “bring context to the model” workflow for real codebases.
• Scoped patching that fails loudly on drift instead of silently changing the wrong code.
• A single tool that outputs durable artifacts (context files, analysis reports, history snapshots) you can commit and diff.