Home | TIL — Saurabh Kumar

Latest notes

Oct 11, 2026
Small on-device models are getting good enough to sit in front of the big ones.
Continue reading
Rampart from the National Design Studio redacts PII in the browser before a prompt reaches any LLM: regexes with checksum or structural checks for cards (Luhn), SSNs, emails, URLs and IPs, plus a fine-tuned MiniLM for names, phones, account numbers, government IDs and street addresses. discuss
- 14.7 MB Q4 ONNX (~15 MB with tokenizer), 98.4% private-term recall across seven Latin-script languages. p50 is 3.9 ms on WebGPU but 12.6 ms on the default WASM backend. OpenAI's Privacy Filter is ~2.8 GB; Presidio scores 65% on the same test set.
- guard.protect() swaps PII for placeholders like [GIVEN_NAME_1], and guard.reveal() fills them back into the reply. The LLM never sees the name; the user still does.
- Default-deny: every detected label is redacted except city, state and ZIP, so the assistant keeps coarse location without the street line.
- Know the gaps: names in non-Latin scripts recall ~14% (Hindi 6%, Arabic 5%), and it scores 86.4% on a hostile-input suite. The repo calls it harm reduction, not perfect protection.
- Same size class, different job: Cactus Whistle is 16.9 MB speech-to-text in seven languages, 11 ms to first token on an M4 Pro CPU (Whisper base: 145 MB, 73 ms). It loads into the same engine as Needle 3, their tool-calling model, so one call takes audio in and returns set_lights(room="kitchen", on=false). Voice commands for home automation with no cloud round trip.
- The pattern: a tiny specialist at the edge does one narrow job (redact, transcribe, route), and the big model only sees what it needs. Both sets of benchmarks are vendor-reported.
ai security
Oct 10, 2026
Sebastian Raschka's history of text classifiers puts numbers on why Jev feels new.

Continue reading
On IMDb, bag-of-words + logistic regression gets 89.9%, a fine-tuned ModernBERT about 95%, and Jev 96.5% zero-shot for $0.65 across 25k reviews. Under the hood it is likely still a pretrained transformer with a one-number head that scores each answer option. The hard part is broad training data and calibrated probabilities. What changes is the cost of starting: no labelling, training or hosting per task. Specialists still win on huge volume, the last point of accuracy, or data that can't leave your network. Within a month OpenAI and Microsoft shipped the same yes/no, choice and score contract. discuss

ai
Oct 10, 2026
Microsoft eXecution Container (MXC) is an embeddable SDK for running untrusted commands under OS-enforced policies, not a hosted sandbox service. discuss
Continue reading
- Defaults: Seatbelt on macOS, Bubblewrap on Linux, ProcessContainer on Windows. SDKs for Node, Rust, and .NET.
- Restricts filesystem and network access. macOS hostname allowlists need a caller-managed proxy; not every backend is a VM.
- Useful boundary: sandbox the command where it executes, not just the agent's tool-composition script. Isolation limits damage; it does not make generated output trustworthy.
ai tools security
Oct 9, 2026
Pi Pocket's subagent implementation gives each worker a retained conversation and uses durable tasks to deliver messages and report answers. Request IDs and recorded answer IDs prevent duplicate delivery and reporting after restart. The idea worth stealing: make the handoff durable, not just the worker. discuss

ai architecture
Oct 9, 2026
Arnav Gupta's pi-subagent-manager makes context inheritance explicit. Nested threads receive their parent's snapshot; independent roots start without its history. Threads stay around for steering and resume. I want synchronous coordination by default, forks for continuity and fresh context for independent review. discuss

ai tools
Oct 9, 2026
Armin Ronacher's subagent extension runs one observable Pi child at a time in tmux. The parent waits for the result, while I can attach to inspect or intervene. Worth preserving when adding hierarchy: synchronous coordination must not mean invisible execution. discuss

ai tools
Oct 8, 2026
Back on Wispr Flow after six months on local speech-to-text (FluidVoice with Parakeet, plus Whisper). Local transcription is fine. The cleanup is what's missing: fixing misheard jargon ate the time dictation saved.

Continue reading
Flow's correction pass, OS-level integration and phone keyboard win. It matters more now because models follow high-level intent well: I speak the intent plus constraints in one go and let the LLM structure it. Whisper mode works even when you speak softly. Want to try it? My referral link gets you a free month of Pro, and I get one once you've dictated 2,000 words. discuss

ai macos tools
Oct 7, 2026
Pi Durable is Earendil's experimental agent harness, shipped alongside Pi 1.0. TannerMidd/pi-pocket is a mobile-first, multiplayer web app for Pi built on it. discuss
Continue reading
- Every model call and tool call is a task that checkpoints to storage (SQLite, JSONL, or memory) before moving on. Kill the process and a new one opens the same file and resumes.
- Recovery is per step: a cut-off model request is resent with the partial answer kept and marked aborted. A cut-off tool call reruns only if it is replay-safe; otherwise the model is told it was interrupted.
- A requestId makes a submission exactly-once, so a client retrying after a crash gets the original back.
- Forks see the parent transcript up to the fork point without copying it.
- ~15k lines total, small enough for an agent to read end to end.
- pi-pocket: restart the server mid-run and the work continues. Several people can watch, steer, or queue messages on the same session.
- npm start -- --access tailscale binds only to the tailnet address, so my phone gets it with no tunnel.
- The shift: the session lives in a SQLite file, not a terminal process, so the phone is just another client. Could replace my Moshi + tmux setup.
- Caveat: anyone who can steer runs shell commands as you.
ai tools architecture

All 70 notes, by week →

Articles

All articles →

Things I Learned

Latest notes

Articles

Archive

Tags