Rampart

Small on-device models are getting good enough to sit in front of the big ones. Rampart from the National Design Studio redacts PII in the browser before a prompt reaches any LLM: regexes with checksum or structural checks for cards (Luhn), SSNs, emails, URLs and IPs, plus a fine-tuned MiniLM for names, phones, account numbers, government IDs and street addresses.

• 14.7 MB Q4 ONNX (~15 MB with tokenizer), 98.4% private-term recall across seven Latin-script languages. p50 is 3.9 ms on WebGPU but 12.6 ms on the default WASM backend. OpenAI’s Privacy Filter is ~2.8 GB; Presidio scores 65% on the same test set.
• guard.protect() swaps PII for placeholders like [GIVEN_NAME_1], and guard.reveal() fills them back into the reply. The LLM never sees the name; the user still does.
• Default-deny: every detected label is redacted except city, state and ZIP, so the assistant keeps coarse location without the street line.
• Know the gaps: names in non-Latin scripts recall ~14% (Hindi 6%, Arabic 5%), and it scores 86.4% on a hostile-input suite. The repo calls it harm reduction, not perfect protection.
• Same size class, different job: Cactus Whistle is 16.9 MB speech-to-text in seven languages, 11 ms to first token on an M4 Pro CPU (Whisper base: 145 MB, 73 ms). It loads into the same engine as Needle 3, their tool-calling model, so one call takes audio in and returns set_lights(room="kitchen", on=false). Voice commands for home automation with no cloud round trip.
• The pattern: a tiny specialist at the edge does one narrow job (redact, transcribe, route), and the big model only sees what it needs. Both sets of benchmarks are vendor-reported.