#performance
Optimization, benchmarks, speed
Writing
- How Prompt Caching Works
What LLM APIs cache, what breaks the cache, and why long agent loops get 5–10× cheaper with it.
- Django ORM: From sync_to_async Threads to Native psycopg3
A deep dive into how Django's ORM became truly async — how the old thread-pool approach worked, what changed in Django 6.0 with psycopg3 and ContextVar, and why asyncpg was left out.
- How I Optimized My Zsh Startup Time
An audit of my zsh config turned up shell forks, redundant plugins, and eager evals. Here are the eight fixes that shaved an estimated 400-700 ms off cold start.
- Django Native PostgreSQL Connection Pooling
Configure Django’s native PostgreSQL connection pool with psycopg, handle connection cleanup, and measure performance before and after.
TIL notes
pi-schema
pi-schema is a JSON Schema validator from the Pi monorepo that never calls
evalornew Function. Mario Zechner says he built it with a single prompt. It compiles each schema into plain closures, so it stays fast in places that block code generation.- The need is real. Cloudflare Workers block
eval()andnew Function. Chrome MV3 extensions cannot allow'unsafe-eval'. Ajv, the default JSON Schema validator in JS, compiles schemas withnew Function. It broke MCP elicitation on Workers (typescript-sdk#689). - How people cope today. Ajv can generate validators at build time, but that only works for schemas you know in advance. The MCP SDK lists @cfworker/json-schema, an interpreter, as a peer dependency. Zod and ArkType have
jitlessflags, and TypeBox falls back to its interpreter. Zod’s flag has to be set before any import that uses Zod (zod#5789). - The numbers. Without eval, the README puts a nested schema at 407 ns for pi-schema, 42 µs for TypeBox and 43 µs for cfworker. pi-schema passes 2,222 of the 2,223 core tests in the official suite.
Agents make this matter. An agent learns its tools from MCP servers while it runs:
tools/listreturns each tool’s JSON Schema. You cannot compile those ahead of time, and the harnesses are moving to the edge. Pi’s Chord design notes plan a Cloudflare Durable Object driver, and Pi Durable already stores agent work as checkpointed tasks. The catches: these are the author’s own benchmarks, Ajv is not in the comparison, the package is not on npm yet, and TypeBox’s JIT still wins on small flat schemas (9 ns vs 31 ns).- The need is real. Cloudflare Workers block
django-bolt decides where your code runs by reading it
django-bolt serves Django from a Rust server, with one Python worker thread per process. In an independent benchmark it ran 1.6 to 1.7x django-ninja on real database endpoints. The catch: a slow sync handler must move to a thread pool, or every request in that process waits. Bolt decides at startup by scanning each handler’s source for ORM calls like
.objects.get.The scan reads only the handler.
return Order.objects.get(id=id)goes to the pool.return services.get_order(id)runs the same query inline on the worker thread, and the whole process stalls until it returns. A codebase with a service layer hits this on nearly every route. The way out is async handlers:async defputs the waiting in the signature, so it can’t hide one call down.A scan of one function sees what’s written there, not what it calls. A fact that has to cross calls belongs in the signature, which is Ron Minsky’s argument for types.
marcelroed/gigatoken
Embedding pipelines do not feed whole documents straight into the model. They first use the embedding model’s tokenizer to measure and split text into token-bounded, overlapping chunks, then embed each chunk and store the vectors. Tokenizer must match the model. Tools like marcelroed/gigatoken accelerate this preparation at corpus scale but do not create embeddings themselves.
karpathy/autoresearch
[karpathy/autoresearch] is a concept/framework that allows one to leverage LLM to run an optimization loops based on a criteria. davebcn87/pi-autoresearch takes this further has built a generic optimization long run loop method on top of pi agent. gemini
Hono
Hono based on web standards is a great option for js runtimes esp. if you plan to deploy your code to cloudflare workers, bun or deno. You get faster startup time and cross-platform compatibility.
how uv got so fast
how uv got so fast. UV performance is because of design decisions while rust contributes to micro optimizations.