Architecture

Query-aware extractive compression: a tiny MLP scores blocks, low-value ones drop, originals stay. ~60ms on CPU — no rewriting, no extra LLM call.

What it is

SuperCompress sits between your prompt and inference. It is extractive (keeps original spans), query-aware (n-gram / entity overlap with the current request is a scoring feature), and stateless per call. The same history can keep a different block set when the query changes. Blocks are not frozen once they enter context.

Property	Meaning
Extractive	Original text only — no paraphrase / summary rewrite
Query-aware	Scoring uses overlap with the current question
Per-call	Re-runs over the context you pass each time
Not a cache	Does not implement provider prompt caching; it shrinks what you send

The compression pipeline

Long context + user question
        ↓
  Tokenizer + feature extraction (CPU)
        ↓
  Semantic block segmentation
        ↓
  Block scoring (tiny MLP)
        ↓
  Duplicate/noise removal
        ↓
  Dependency-preserving packer
        ↓
  Verifier (estimates important context kept)
        ↓
  Compressed evidence prompt → LLM

End-to-end ~60ms on a single CPU core. No GPU, no model download at serve time, no extra LLM call.

The learned policy

Production weights are in web/assets/data/model.json (hosted API + browser). Rough shape:

Property	Value
Feature dim	16 per token
Hidden dim	64
Size	~10K parameters, tens of KB on disk
Output	Keep / eviction oriented scores for extractive selection
Inference	Pure JS / NumPy-style math — no GPU required

Features include position, query n-gram overlap, line shape, structure signals, entropy, and related cues. Compiler mode then packs at block granularity with dependency rules — not blind per-token truncation.

Prompt caching & agent loops

SuperCompress compresses text before the model call. It never reads or writes the model’s KV cache. Fewer tokens simply means less prefill work for whatever happens next on the provider.

Provider prompt cache (OpenAI prefix cache, Anthropic cache_control, vLLM APC) needs a byte-identical prefix across turns. Providers often implement that with their own KV reuse — that is their cache, not ours.

Because compression is query-aware and re-runs each call, recompressing the entire growing history every turn can change the kept set → different prefix → cache miss on that blob. The compressed body may then bill as normal (uncached) input even though it is smaller.

cache_prefix=true (CacheAligner) wraps output in a deterministic ~60-token XML preamble so the envelope is stable. It does not freeze kept content across turns. See CacheAligner. Sticky / append-only session compression is the path for automatic cache-friendly loops.

Cost model

• Hosted API: $0.30 / 1M tokens on original tokens processed (5M/month free, then PAYG).
• Self-hosted Python / in-browser engine: $0 compression fee.
• End-to-end savings = model input dollars avoided − SuperCompress fee (if any).

Against full-price uncached input, fewer tokens usually wins after the fee. Against a loop that is already mostly cache reads, recompressing history can lose — use dump-only compression or sticky prefixes instead.

Compiler mode

Compiler mode is the production path (default on the hosted API). No user budget. It:

1. Segments context into semantic blocks — headings, code, logs, chat turns, prose, tables, config
2. Downranks repeated tool output, boilerplate, noise, duplicates, irrelevant context
3. Preserves nearby headings, imports, trace/log context, balanced markdown fences
4. Verifies important-kept % and risk (low / medium / high)
5. Reports kept/dropped blocks with reasons

Other modes

	Compiler	Fixed-ratio	Precision / CCR
Budget?	No — maximize removal safely	Yes — budget_ratio	Verifier-gated / reversible markers
Risk reporting	Yes	No	Yes (precision)
Best for	Production	Research / legacy	Stricter keep guarantees / retrieve

Browser & modules

The same engine runs in the browser via compress-engine.js and model.json. The playground needs no API key — compression stays in-browser. Function-by-function reference: Compress engine.