Architecture

Query-aware extractive compression: a tiny MLP scores blocks, low-value ones drop, originals stay. ~60ms on CPU — no rewriting, no extra LLM call.

What it is

SuperCompress sits between your prompt and inference. It is extractive (keeps original spans), query-aware (n-gram / entity overlap with the current request is a scoring feature), and stateless per call. The same history can keep a different block set when the query changes. Blocks are not frozen once they enter context.

Property	Meaning
Extractive	Original text only — no paraphrase / summary rewrite
Query-aware	Scoring uses overlap with the current question
Per-call	Re-runs over the context you pass each time
Not a cache	Does not implement provider prompt caching; it shrinks what you send

The compression pipeline

Long context + user question
        ↓
  Tokenizer + feature extraction (CPU)
        ↓
  Semantic block segmentation
        ↓
  Block scoring (tiny MLP)
        ↓
  Duplicate/noise removal
        ↓
  Dependency-preserving packer
        ↓
  Verifier (estimates important context kept)
        ↓
  Compressed evidence prompt → LLM

End-to-end ~60ms on a single CPU core. No GPU, no model download at serve time, no extra LLM call.

The learned policy

Production weights are in web/assets/data/model.json (hosted API + browser). Rough shape:

Property	Value
Feature dim	16 per token
Hidden dim	64
Size	~10K parameters, tens of KB on disk
Output	Keep / eviction oriented scores for extractive selection
Inference	Pure JS / NumPy-style math — no GPU required

Features include position, query n-gram overlap, line shape, structure signals, entropy, and related cues. Compiler mode then packs at block granularity with dependency rules — not blind per-token truncation.

Prompt caching & agent loops

Provider prompt cache (OpenAI prefix cache, Anthropic cache_control, vLLM APC) needs a byte-identical prefix across turns.

Because compression is query-aware and re-runs each call, recompressing the entire growing history every turn can change the kept set → different prefix → cache miss on that blob. The compressed body may then bill as normal (uncached) input even though it is smaller.

cache_prefix=true (CacheAligner) wraps output in a deterministic ~60-token XML preamble so the envelope is stable. It does not freeze kept content across turns. See CacheAligner. Sticky / append-only session compression is the path for automatic cache-friendly loops.

Cost model

• Hosted API: $1 / 1M tokens on original tokens processed (5M/month free, then PAYG).
• Self-hosted Python / in-browser engine: $0 compression fee.
• End-to-end savings = model input dollars avoided − SuperCompress fee (if any).

Against full-price uncached input, fewer tokens usually wins after the fee. Against a loop that is already mostly cache reads, recompressing history can lose — use dump-only compression or sticky prefixes instead.

Compiler mode

Compiler mode is the production path (default on the hosted API). No user budget. It:

1. Segments context into semantic blocks — headings, code, logs, chat turns, prose, tables, config
2. Downranks repeated tool output, boilerplate, noise, duplicates, irrelevant context
3. Preserves nearby headings, imports, trace/log context, balanced markdown fences
4. Verifies important-kept % and risk (low / medium / high)
5. Reports kept/dropped blocks with reasons

Other modes

	Compiler	Fixed-ratio	Precision / CCR
Budget?	No — maximize removal safely	Yes — budget_ratio	Verifier-gated / reversible markers
Risk reporting	Yes	No	Yes (precision)
Best for	Production	Research / legacy	Stricter keep guarantees / retrieve

Browser & modules

The same engine runs in the browser via compress-engine.js and model.json. The playground needs no API key — compression stays in-browser.