Architecture

Query-aware extractive compression: hosted Neural Keep reads the question, keeps original lines that answer it, and drops the rest, no rewriting, no extra LLM call.

What it is

SuperCompress sits between your prompt and inference. It is extractive (keeps original spans), query-aware (n-gram / entity overlap with the current request is a scoring feature), and stateless per call. The same history can keep a different block set when the query changes. Blocks are not frozen once they enter context.

Property	Meaning
Extractive	Original text only, no paraphrase / summary rewrite
Query-aware	Scoring uses overlap with the current question
Per-call	Re-runs over the context you pass each time
Not a cache	Does not implement provider prompt caching; it shrinks what you send

The compression pipeline

Long context + user question
        ↓
  Tokenizer + feature extraction (CPU)
        ↓
  Semantic block segmentation
        ↓
  Block scoring (tiny MLP)
        ↓
  Duplicate/noise removal
        ↓
  Dependency-preserving packer
        ↓
  Verifier (estimates important context kept)
        ↓
  Compressed evidence prompt → LLM

Compiler path: end-to-end ~ms on a single CPU core, no GPU. Neural v2 (hosted): query-aware Neural Keep on Fly CPU, seconds-scale on long dumps. Do not mix latency claims.

The learned policy

Production weights are in web/assets/data/model.json (hosted API + browser). Rough shape:

Property	Value
Feature dim	16 per token
Hidden dim	64
Size	Small local scorer. Production compression runs on the hosted API.
Output	Keep / eviction oriented scores for extractive selection
Inference	Pure JS / NumPy-style math, no GPU required

Features include position, query n-gram overlap, line shape, structure signals, entropy, and related cues. Compiler mode then packs at block granularity with dependency rules, not blind per-token truncation.

Prompt caching & agent loops

SuperCompress compresses text before the model call. It never reads or writes the model’s KV cache. Fewer tokens simply means less prefill work for whatever happens next on the provider.

Provider prompt cache (OpenAI prefix cache, Anthropic cache_control, vLLM APC) needs a byte-identical prefix across turns. Providers often implement that with their own KV reuse, that is their cache, not ours.

Because compression is query-aware and re-runs each call, recompressing the entire growing history every turn can change the kept set → different prefix → cache miss on that blob. The compressed body may then bill as normal (uncached) input even though it is smaller.

cache_prefix=true (CacheAligner) wraps output in a deterministic ~60-token XML preamble so the envelope is stable. It does not freeze kept content across turns. See the architecture notes on cache_prefix. Sticky / append-only session compression is the path for automatic cache-friendly loops.

Cost model

• Product path: the hosted API (Neural Keep). Current plans are in the dashboard.
• Coding agents: npm install -g supercompress-proxy then supercompress setup.
• End-to-end savings = model input tokens avoided − the hosted compress fee (if any).

Against full-price uncached input, fewer tokens usually wins after the fee. Against a loop that is already mostly cache reads, recompressing history can lose, use dump-only compression or sticky prefixes instead.

Compiler mode

Compiler mode is the production path (default on the hosted API). No user budget. It:

1. Segments context into semantic blocks, headings, code, logs, chat turns, prose, tables, config
2. Downranks repeated tool output, boilerplate, noise, duplicates, irrelevant context
3. Preserves nearby headings, imports, trace/log context, balanced markdown fences
4. Verifies important-kept % and risk (low / medium / high)
5. Reports kept/dropped blocks with reasons

Other modes

	Compiler	Fixed-ratio	Precision / CCR
Budget?	No, maximize removal safely	Yes, budget_ratio	Verifier-gated / reversible markers
Risk reporting	Yes	No	Yes (precision)
Best for	Production	Research / legacy	Stricter keep guarantees / retrieve

Browser & modules

The legacy compiler also runs in the browser via compress-engine.js and model.json (CPU, millisecond-class). That is not the hosted Neural Keep path. The benchmarks and playground demos may use the local compiler for instant preview; production API quality is Neural v2. Function-by-function reference: Compress engine.