# Rampart

By Saurabh Kumar · TIL note, Oct 11, 2026 · Tags: llm, security

Source: https://saurabh-kumar.com/til/1791720000/

Small on-device models are getting good enough to sit in front of the big ones. [Rampart](https://ndstudio.gov/posts/say-hello-to-rampart) from the National Design Studio redacts PII in the browser before a prompt reaches any LLM: regexes with checksum or structural checks for cards (Luhn), SSNs, emails, URLs and IPs, plus a fine-tuned MiniLM for names, phones, account numbers, government IDs and street addresses.
- 14.7 MB Q4 ONNX (~15 MB with tokenizer), 98.4% private-term recall across seven Latin-script languages. p50 is 3.9 ms on WebGPU but 12.6 ms on the default WASM backend. OpenAI's Privacy Filter is ~2.8 GB; Presidio scores 65% on the same test set.
- `guard.protect()` swaps PII for placeholders like `[GIVEN_NAME_1]`, and `guard.reveal()` fills them back into the reply. The LLM never sees the name; the user still does.
- Default-deny: every detected label is redacted except city, state and ZIP, so the assistant keeps coarse location without the street line.
- Know the gaps: names in non-Latin scripts recall ~14% (Hindi 6%, Arabic 5%), and it scores 86.4% on a hostile-input suite. The repo calls it harm reduction, not perfect protection.
- Same size class, different job: [Cactus Whistle](https://cactuscompute.com/blog/whistle) is 16.9 MB speech-to-text in seven languages, 11 ms to first token on an M4 Pro CPU (Whisper base: 145 MB, 73 ms). It loads into the same engine as [Needle 3](https://huggingface.co/Cactus-Compute/needle3), their tool-calling model, so one call takes audio in and returns `set_lights(room="kitchen", on=false)`. Voice commands for home automation with no cloud round trip.
- The pattern: a tiny specialist at the edge does one narrow job (redact, transcribe, route), and the big model only sees what it needs. Both sets of benchmarks are vendor-reported.