performance on Saurabh Kumar

marcelroed/gigatoken

Fri, 24 Jul 2026 12:00:00 +0000

Embedding pipelines do not feed whole documents straight into the model. They first use the embedding model’s tokenizer to measure and split text into token-bounded, overlapping chunks, then embed each chunk and store the vectors. Tokenizer must match the model. Tools like marcelroed/gigatoken accelerate this preparation at corpus scale but do not create embeddings themselves.

How Prompt Caching Works

Sun, 21 Jun 2026 00:00:00 +0000

Today I learned how prompt caching actually works under the hood in LLM APIs (Anthropic, OpenAI, Bedrock, etc.), and why it makes long agent loops 5–10× cheaper and noticeably faster.

The problem it solves

In an agent / chat loop, every request to the model looks almost identical to the previous one:

The same big system prompt (rules, tool descriptions, skills…)
The same set of tool definitions
The same history of past user/assistant/tool messages, with only a few new tokens appended at the end

Without caching, the provider has to re-tokenize and run a full attention pre-fill over all of that prefix on every turn. For a 100k-token context, that prefill dominates both cost and latency.

What gets cached: the KV state

When a transformer processes a prompt, it computes a key/value tensor pair for every token in every attention layer. That KV state is what attention queries against on the next decoded token.

Prompt caching is, mechanically, “save those KV tensors for a prefix on the server, and reuse them next time the same prefix shows up.” Output tokens are never cached — only the prompt-side KV state.

The lifecycle

Mark a cache breakpoint. The client places a cache_control marker (Anthropic) or relies on automatic prefix detection (OpenAI). Everything before the marker is the cacheable prefix.
First request → cache write. The provider hashes the cacheable prefix (model id + tools + messages + cache key, byte-exact), runs the prefill, and stores the resulting KV tensors keyed by that hash. You’re billed cache_write tokens, usually a bit more than normal input (~1.25× for Anthropic 5-minute cache, ~2× for the 1-hour “extended” cache).
Next request → cache read. If the new request starts with exactly the same prefix, the provider looks up the hash, loads the stored KV state, and only runs fresh compute for the suffix after the last hit. You’re billed cache_read tokens — often ~10% of the normal input price.
TTL. Entries expire after a TTL (Anthropic: 5 min default, 1 hour opt-in; OpenAI: a few minutes, automatic). Each hit typically refreshes the TTL, so an active session keeps the cache warm.

What invalidates the cache

The hash is over the exact bytes of the prefix. Anything that changes those bytes forces a fresh cache_write:

Editing the system prompt (even reordering sentences or adjusting whitespace)
Adding, removing, or reordering a tool definition
Inserting/editing/reordering an earlier message in the history
Switching model id or provider
Changing the cache key (some providers expose one for tenant isolation)

The corollary: to maximize hit rate, agents should only append at the end and keep system prompt and tools byte-stable across turns.

Reading the usage block

Most providers return a usage object on every response with at least these fields:

{
 "input": 312, // fresh, non-cached prompt tokens this turn
 "output": 845, // generated tokens
 "cacheRead": 47210, // prompt tokens served from cache
 "cacheWrite": 0 // prompt tokens written to cache this turn
}

A useful per-turn metric is the cache hit ratio:

cacheHitRate = cacheRead / (input + cacheRead + cacheWrite)

0% → nothing reused. Normal on the very first turn, or right after the prefix changed (tool list updated, history compacted, model swapped).
70–95% → healthy steady state for a long agent loop.
cacheWrite > 0, cacheRead = 0 → you just primed the cache; the next turn should be cheap if you don’t break the prefix.

Pricing back-of-the-envelope

Rough Anthropic numbers (Claude Sonnet, illustrative):

Token type	Price multiplier vs. input
`input`	1×
`cache_write`	~1.25× (5 min) / ~2× (1h)
`cache_read`	~0.1×
`output`	~5×

So a 100k-token prefix that’s hit 20 times across a session costs roughly:

1 × 1.25 × 100k (one write)
+ 19 × 0.1 × 100k (nineteen reads)
= 1.25 × 100k + 1.9 × 100k
≈ 3.15 × 100k input-equivalent

vs. uncached:
20 × 1 × 100k = 20 × 100k

A ~6× saving on the prompt side, plus a big latency win because the server skips the prefill.

Gotchas

Minimum prefix size. Most providers require the cacheable prefix to be above some threshold (e.g. 1024 tokens for Anthropic) before caching kicks in at all.
Position of the breakpoint matters. Putting it after volatile content (like the latest user message) means nothing ever gets reused.
Tool definitions are part of the prefix. Dynamically generated tool lists with shuffled order will silently destroy your hit rate.
Streaming vs. non-streaming use the same cache; cache fields show up in the final usage event.
Cost accounting: cache_write is more expensive than input, so a flaky prefix that constantly re-writes can be worse than no caching at all.

Django ORM: From sync_to_async Threads to Native psycopg3

Sat, 13 Jun 2026 00:00:00 +0000

Django shipped async ORM methods back in 3.1. The a-prefixed methods, including aget(), afilter(), and acreate(), let calling code use coroutines, but the database work still blocked an OS thread. I want to explain that distinction and how the execution path changed.

Django 6.0 fixed it for real in December 2025. I want to walk through the path from “async via threads” to “native async via psycopg3,” because it is not a story about a feature landing. It is a story about three constraints that boxed Django in for five years: psycopg2 blocks, connection state lived in thread-locals, and the entire ORM assumed DB-API2. Once you see those three, every design decision Django made falls out of them almost mechanically.

The Thread-Based Async Path

When async views arrived in 3.1, the ORM underneath them was entirely synchronous. psycopg2 — the default PostgreSQL driver for most of Django’s life — wraps libpq, a C library with no async I/O surface at all. There is no flag you set to make psycopg2 async. You either replace the driver or you find a workaround.

Django found the workaround: asgiref.sync.sync_to_async.

# Simplified — what aget() actually did under the hood in Django 5.x
from asgiref.sync import sync_to_async

async def aget(self, *args, **kwargs):
 return await sync_to_async(self.get, thread_sensitive=True)(*args, **kwargs)

sync_to_async pushes the callable into a ThreadPoolExecutor. The event loop is not blocked, because the coroutine awaits the thread’s future. But a real OS thread is blocked, sitting on a TCP socket waiting for Postgres to answer. You moved the blocking, you did not remove it.

The thread_sensitive=True flag is the part that gives the game away. Django’s database connections lived in thread-locals:

# django/db/backends/base/base.py (pre-6.0)
import threading
_thread_local = threading.local()

Without thread_sensitive=True, two consecutive sync_to_async calls from the same request might land on different threads, see different thread-locals, and tear your transaction state in half. The flag pins every call within the same coroutine context to one dedicated thread, so the connection still looks like a single connection per request. This preserves the connection’s thread affinity.

Put it together and the cost is obvious. Five hundred concurrent async requests, each touching the database, need five hundred threads. That is the thread-per-connection model, even though the calling code uses coroutines.

Why psycopg2 Was a Dead End

You cannot retrofit async I/O onto psycopg2. It calls into libpq synchronously and blocks the calling thread until the query returns. The only way to run it in an async context is exactly what Django did: park it in a thread and await that thread. There was never a clever escape here.

The obvious alternative was asyncpg, a fully async PostgreSQL driver that beats psycopg3 on raw throughput. Django ruled it out, and the reasons are worth their own section further down. The short version is that asyncpg does not implement DB-API2 (PEP 249)¹, and Django’s ORM is built on DB-API2 from the ground up — cursor objects, execute(), fetchone(), fetchall(). You do not bolt a different contract onto twenty years of ORM and call it a driver swap.

psycopg3 is the answer that fits, because it does not force that choice.

psycopg3 Does Async And DB-API2 At The Same Time

psycopg3 — the package is just psycopg, not psycopg3 — was redesigned to support both synchronous and asynchronous execution through one shared core². The same query compilation logic runs both paths. What changes is the connection and cursor type you hand it.

# psycopg3 — synchronous path
with psycopg.connect(dsn) as conn:
 with conn.cursor() as cur:
 cur.execute("SELECT * FROM posts WHERE id = %s", [pk])
 row = cur.fetchone()

# psycopg3 — async path (same SQL, same parameters, different objects)
async with await psycopg.AsyncConnection.connect(dsn) as aconn:
 async with aconn.cursor() as cur:
 await cur.execute("SELECT * FROM posts WHERE id = %s", [pk])
 row = await cur.fetchone()

AsyncConnection and AsyncCursor use asyncio directly. The await cur.execute(...) suspends the coroutine and releases the event loop while the network I/O happens. No thread sits idle on a socket. That is the whole difference, and it is the difference that matters.

Django 6.0 made psycopg3 the recommended PostgreSQL driver and wired AsyncConnection and AsyncCursor into the backend.

What Django 6.0 Actually Changed

Three refactors enabled the switch. None of them are glamorous, and that is the point.

1. ContextVar Replaces Thread-Locals

Thread-locals are indexed by thread ID. Coroutines do not have thread IDs of their own — they share threads and switch between each other at await points. So thread-local connection storage is not merely awkward under async, it is structurally wrong. Think of a thread-local as a coat hook bolted to a single chair: it works only as long as one person stays in that chair the whole time. Coroutines stand up and swap chairs constantly.

The fix is contextvars.ContextVar, which arrived in Python 3.7. A ContextVar is scoped to the current execution context, and each asyncio Task gets its own context, copied from its parent when it is created. Django 6.0 moved connection storage off threading.local() and onto a ContextVar:

# django/db/backends/base/base.py (Django 6.0)
from contextvars import ContextVar

_connection: ContextVar = ContextVar('django_db_connection', default=None)

Two concurrent requests, both awaiting a query, each live in their own asyncio Task, each with their own context copy, each seeing their own connection. No locking, no thread affinity, no thread_sensitive gymnastics. The coat hook now travels with the person.

2. DatabaseWrapper Gets an Async Connection Path

DatabaseWrapper is the Django object that represents a database connection. Before 6.0 it exposed one path, self.connection, a synchronous psycopg2 connection. After 6.0 it also exposes async_connection():

# Conceptual — the async context manager on DatabaseWrapper
@asynccontextmanager
async def async_connection(self):
 conn = await psycopg.AsyncConnection.connect(**self.get_connection_params())
 _connection.set(conn)
 try:
 yield conn
 finally:
 await conn.close()

Django’s connection-handling code detects whether it is running inside an asyncio event loop and routes to the right path. You do not pick the path. The runtime does.

3. SQLCompiler Gets async_execute

The query compiler, SQLCompiler and its subclasses, is where Django turns a QuerySet into SQL and sends it to the database. Before 6.0 there was one path, execute_sql(). Django 6.0 adds async_execute_sql():

async def async_execute_sql(self, result_type=MULTI, chunked_fetch=False, chunk_size=GET_ITERATOR_CHUNK_SIZE):
 async with self.connection.async_connection() as conn:
 async with conn.cursor() as cursor:
 await cursor.execute(self.as_sql())
 if result_type == MULTI:
 return [row async for row in cursor]
 return await cursor.fetchone()

aget(), afilter(), aiterator() and the rest now route through async_execute_sql() instead of wrapping execute_sql() in sync_to_async. The thread pool drops out of the picture entirely.

The New Execution Stack

Here is how await Post.objects.aget(id=1) flows through Django 6.0:

await Post.objects.aget(id=1)
 └── QuerySet.aget()
 └── SQLCompiler.async_execute_sql()
 └── DatabaseWrapper.async_connection() # ContextVar-scoped
 └── psycopg.AsyncConnection
 └── AsyncCursor.execute()
 └── asyncio TCP socket write
 └── await socket read ← event loop released here
 └── row data returned

And here is the same call under Django 5.x:

await Post.objects.aget(id=1)
 └── QuerySet.aget()
 └── sync_to_async(thread_sensitive=True)(QuerySet.get)
 └── ThreadPoolExecutor — OS thread blocked
 └── SQLCompiler.execute_sql()
 └── psycopg2 connection (thread-local)
 └── libpq blocking TCP call
 └── thread wakes, result returned

The event loop is no longer handing work off to a thread. It suspends the coroutine, does other work, and resumes when the socket is readable. That is the entire reform, and it is enough.

What This Means For Concurrency

The practical payoff is in connection count. With the old thread approach, five hundred concurrent async requests meant up to five hundred threads and up to five hundred Postgres connections. Threads are cheap but not free — each costs roughly 8MB of stack by default, plus scheduler overhead.

With native async, those same five hundred requests can be multiplexed over a connection pool sized to what your database can actually handle, typically 10 to 50 connections, with the event loop running the wait queue. You use far fewer threads, one or a handful for the event loop workers, and far fewer database connections.

Django 6.0 also ships native connection pooling for the async path³, built on psycopg_pool.AsyncConnectionPool:

# settings.py — async pooling in Django 6.0
DATABASES = {
 "default": {
 "ENGINE": "django.db.backends.postgresql",
 "OPTIONS": {
 "pool": {
 "min_size": 2,
 "max_size": 10,
 },
 },
 "CONN_MAX_AGE": 0,
 }
}

This is the same pool option I covered in Django Native PostgreSQL Connection Pooling, with one difference that matters. In that post I warned against native pooling under ASGI — Django’s own docs recommended PgBouncer instead — because the sync pool and the event loop did not cooperate. Django 6.0’s AsyncConnectionPool is what closes that gap. The pool itself is now async-aware, so the ASGI caveat no longer applies on the native psycopg3 path. I was right to warn you then. You can stop worrying about it now.

Is sync_to_async Gone?

No, and it should not be. It is the fallback for drivers that cannot go native async:

Driver	Async path
psycopg3 (PostgreSQL)	Native `AsyncCursor` — no threads
psycopg2 (PostgreSQL)	`sync_to_async` thread wrap — still blocked
mysqlclient / MySQL	`sync_to_async` thread wrap
sqlite3	`sync_to_async` thread wrap

Run Django 6.0 against psycopg2 or MySQL in an async context and Django still falls back to the thread pool. The native path only exists when psycopg3 is installed and configured. The thread trick did not become wrong. It became the path of last resort.

The way forward for other databases is psycopg3-equivalent async drivers. MySQL has aiomysql, SQLite has aiosqlite. Django contributions for those backends are in progress but not stable in 6.0, so for now PostgreSQL is the database that gets the real thing.

asyncpg Is Still Faster, And Still Not Supported

Benchmark asyncpg against psycopg3 on raw async throughput and asyncpg wins, often by 20 to 40 percent on bulk read workloads. Django will not adopt it. The reasons are structural, not political, and I think Django is right.

No DB-API2. asyncpg uses positional $1, $2 parameters, not %s. The entire ORM query compiler assumes %s.
No synchronous fallback. Management commands, migrations, and sync views would need a whole separate code path.
A different transaction model. asyncpg handles transactions differently from the DB-API2 autocommit and explicit model Django manages internally.

Using asyncpg would require changes to the ORM, not just a driver swap. A driver benchmark alone does not measure that work. If you genuinely need asyncpg throughput with a Django-like ORM, look at SQLAlchemy 2.0 with its async session support and asyncpg backend, running under Starlette or FastAPI. For Django, psycopg3 is the correct answer.

Upgrading

To get the native async path today:

pip install "psycopg[binary,pool]"

# settings.py
DATABASES = {
 "default": {
 "ENGINE": "django.db.backends.postgresql",
 "NAME": "mydb",
 "USER": "myuser",
 "PASSWORD": "mypassword",
 "HOST": "localhost",
 "CONN_MAX_AGE": 0,
 "OPTIONS": {
 "pool": True, # or dict with min_size/max_size
 },
 }
}

If you are on Django 5.2 LTS with psycopg3, you already get the synchronous psycopg3 path and connection pooling. The native async query path needs Django 6.0. You can upgrade the driver without upgrading Django and still get the pooling benefits in sync contexts, which is a fine thing to do while you wait.

Why It Took Five Years

Django’s async ORM story ran from “async with threads” in 3.1 in 2020 to “native async with psycopg3” in 6.0 in 2025. Five years sounds slow until you look at what had to be true first:

psycopg3 had to be stable enough to depend on, which it reached around 2022 to 2023.
Thread-local connection state was woven through the codebase, and the ContextVar migration touched hundreds of files.
The Django project is conservative about breaking changes, especially in database handling, where bugs are not cosmetic.

The result is correct and maintainable rather than the absolute fastest thing money can buy, and I do not think that was the wrong trade. For most Django applications — web services handling thousands of concurrent HTTP requests — native async with psycopg3 is already more than fast enough, and it integrates cleanly with everything else in the ecosystem.

Django Native PostgreSQL Connection Pooling — the sync side of the same story: configuring the psycopg3 pool, sizing min_size/max_size, and avoiding the CONN_MAX_AGE foot-gun.

PEP 249 — Python Database API Specification v2.0 — The DB-API2 spec that Django’s ORM is built on top of; asyncpg deliberately does not implement it. ↩︎
psycopg3 — Async operations — Official psycopg documentation on the async connection and cursor API. ↩︎
Django 6.0 Release Notes — Full changelog including async ORM changes, background tasks, and template partials. ↩︎

karpathy/autoresearch

Mon, 08 Jun 2026 12:00:00 +0000

[karpathy/autoresearch] is a concept/framework that allows one to leverage LLM to run an optimization loops based on a criteria. davebcn87/pi-autoresearch takes this further has built a generic optimization long run loop method on top of pi agent. gemini

How I Optimized My Zsh Startup Time

Mon, 11 May 2026 00:00:00 +0000

My zsh prompt was taking close to a second to appear on a fresh terminal. Not catastrophic. But you feel it every time tmux spawns a new pane, and for me that is constantly. The config had grown for years, one eval and one plugin at a time, and nobody had been auditing it, least of all me.

What follows is the audit, and the eight things I changed. The numbers are estimates, not measurements, and I will say so again at the end.

Measure first

Before I touched anything, I measured. Two tools did the work:

# Cold-start benchmark (run a few times, watch the variance):
for i in {1..5}; do time zsh -i -c exit; done

# In-shell flamegraph:
# 1. Uncomment `zmodload zsh/zprof` at the top of .zshrc
# 2. Add `zprof` at the bottom
# 3. Open a new shell
zprof | head -40

zprof is the one that helped most. It prints which function calls and evals are burning time, sorted by self-time and call count. That is enough to stop guessing.

1. Stop forking `brew --prefix` five times per shell

My macOS config had this:

export PATH=$(brew --prefix coreutils)/libexec/gnubin:$PATH
export PATH=$(brew --prefix findutils)/libexec/gnubin:$PATH
export PATH=$(brew --prefix gnu-sed)/libexec/gnubin:$PATH
export PATH=$(brew --prefix grep)/libexec/gnubin:$PATH
export PATH=$(brew --prefix curl)/bin:$PATH

Five forks of brew, which is a Ruby program, on every shell, just to resolve paths that always end up under ${HOMEBREW_PREFIX}/opt/<formula> anyway. Each fork is about 35 ms on Apple Silicon, so that is roughly 180 ms gone before I have typed a single character.

Fix:

_brew_opt="${HOMEBREW_PREFIX:-/opt/homebrew}/opt"
[ -d "$_brew_opt" ] || _brew_opt="/usr/local/opt"
export PATH="$_brew_opt/coreutils/libexec/gnubin:$_brew_opt/findutils/libexec/gnubin:$_brew_opt/gnu-sed/libexec/gnubin:$_brew_opt/grep/libexec/gnubin:$_brew_opt/curl/bin:$PATH"
unset _brew_opt

A path-prefix lookup costs nothing. The /usr/local/opt fallback is there for Intel macOS, where HOMEBREW_PREFIX may be unset, for example in bash without brew shellenv.

Saves about 180 ms.

2. Defer plugins with zinit’s `wait` ice

If you use a plugin manager, this is probably the biggest single win on the table. Syntax highlighting, autosuggestions, history-substring-search, completion add-ons: none of these need to load before the first prompt renders. They only need to be there by the time you type something.

With zinit’s wait ice, you defer them until after the first prompt:

zinit wait lucid light-mode for \
 atinit"ZINIT[COMPINIT_OPTS]=-C; zicompinit; zicdreplay" \
 zdharma-continuum/fast-syntax-highlighting \
 atload"_zsh_autosuggest_start" \
 zsh-users/zsh-autosuggestions \
 blockf atpull'zinit creinstall -q .' \
 zsh-users/zsh-completions \
 atload"bindkey '^[[A' history-substring-search-up; bindkey '^[[B' history-substring-search-down" \
 zsh-users/zsh-history-substring-search \
 wfxr/forgit

The atinit ice on the first plugin runs compinit once and replays cached completions via zicdreplay. One compinit covers every turbo-loaded plugin. The atload"_zsh_autosuggest_start" line re-initializes autosuggestions after deferred loading. Without it the plugin silently no-ops, which took me an embarrassing amount of time to work out.

Saves 100-300 ms.

3. Compile `.zcompdump` to bytecode

compinit parses thousands of completion functions on every startup. Compiling its dump file to bytecode means subsequent shells skip the parser:

zcompdump="${ZDOTDIR:-$HOME}/.zcompdump"
[[ -s "$zcompdump" && (! -s "${zcompdump}.zwc" || "$zcompdump" -nt "${zcompdump}.zwc") ]] \
 && zcompile "$zcompdump"

It recompiles only when the source is newer than the cached .zwc. Saves 30-80 ms.

4. Pick one version manager

I had both fnm and mise active, which is one more than anyone needs. Mise manages everything I actually use, Node, Python, Go; fnm env in .zshrc was a leftover that nothing depended on. Deleting it saved another 30-80 ms per shell.

Mise itself has a second optimization, --shims mode:

eval "$(~/.local/bin/mise activate zsh --shims)"

Default mise activate installs a precmd hook that runs on every prompt to keep tool versions in sync with .mise.toml. Shim mode replaces that hook with symlinks on PATH that resolve the right version the first time you call each tool. That is about 17 ms saved at startup, paid back as 10-30 ms on the first invocation per shell. For interactive use it is a fine trade.

5. Cache the slow `eval`s

A lot of tools want you to run eval "$(tool init zsh)" at startup. Each one forks the tool, runs its setup, and pipes the result back. You pay that cost every shell, even though the output rarely changes between runs.

A small helper fixes it:

_cache_eval() {
 local name=$1; shift
 local cache=~/.zsh/cache/$name.zsh
 [[ -s $cache && $cache -nt ${commands[$1]:-/dev/null} ]] \
 || { mkdir -p ~/.zsh/cache; "$@" > $cache; }
 source $cache
}
_cache_eval zoxide zoxide init zsh --cmd j
_cache_eval uv uv generate-shell-completion zsh

The $cache -nt ${commands[$1]:-/dev/null} test compares mtimes. When brew or mise upgrades the binary, the cache invalidates on its own. No stale-cache surprises after a brew upgrade.

Keep starship init zsh eager. You need it for the prompt anyway, and it is already fast.

Saves 50-100 ms.

6. Audit plugins; cut what you don’t use

I was loading the Oh My Zsh git plugin via zinit’s snippet feature. The plugin ships roughly 150 aliases.

So I grepped my last few months of shell history against its alias list. Five OMZ-only aliases came back: gsw, glo, gf, gclean, gss. Everything else I either did not use or had already redefined to my own taste. I copied those five into shell/.aliases and dropped the OMZP::git snippet from zinit.

There is a second reason to do this, and it has nothing to do with startup time. Because the OMZ plugin loaded in turbo mode, after my eager shell/.aliases, it was quietly overwriting aliases I had defined myself. My gc='git commit -m' was getting clobbered to git commit --verbose, and gg='lazygit' to git gui. The plugin “worked” because turbo-loaded aliases land before you type, but the behavior was not the one I had configured. Dropping the plugin put my own aliases back in charge.

While I was at it I found zsh/.zsh/plugins/git/git.plugin.zsh in my dotfiles, 150 lines of git aliases, sourced from nowhere. Years of dead code I had been carrying around. Gone.

7. Move PATH and env to `.zshenv`

.zshrc runs only for interactive shells. If you set PATH there, then cron jobs, zsh -c "command" invocations, and IDE-spawned non-interactive shells do not see your tools. That is how you end up burning half a day on “works in my terminal, breaks in CI.”

The right home is .zshenv, which runs for every zsh invocation:

# ~/.zshenv
export PNPM_HOME="$HOME/Library/pnpm"

typeset -U path PATH # auto-dedupe
path=(
 "$HOME/.local/bin"
 "$PNPM_HOME"
 $path
)

[ -f "$HOME/.cargo/env" ] && . "$HOME/.cargo/env"

typeset -U path PATH ties the path array to the PATH scalar and deduplicates automatically. No more 8 KB PATH strings with the same entry six times because nested shells kept prepending.

This is not a cold-start win, since .zshenv still runs. It is a correctness fix. A shorter PATH does make command lookups faster, so you get a little of both.

8. Fix the things you didn’t know were broken

While I was in there, zprof and a slow re-read of the config surfaced a stack of small bugs that had nothing to do with cold start. They were just wrong:

~/.profile was being sourced twice, once from .zshrc and once from .zshenv. 20-80 ms of pure waste.
compinit’s cache-staleness check had inverted-looking logic and a misleading comment. It happened to do the right thing, but only by accident.
zsh/.zsh/lib/completions.zsh referenced $ZSH, an Oh My Zsh leftover that was unset in my setup. The completion cache-path was silently expanding to /cache/, the filesystem root. Nothing got written there because the directory was not writable, so completions had no cache at all. Caching had quietly been off the whole time.
compdef calls from eager tool initializations were arriving before turbo-mode compinit had run, so completions for those tools were not registering. I fixed it with a small shim that queues the calls until compinit is ready:

if ! (( ${+functions[compdef]} )); then
 typeset -ga __deferred_compdefs
 compdef() { __deferred_compdefs+=("${(j: :)@}") }
fi

Then in the turbo block’s atinit, after zicompinit; zicdreplay:

(( ${#__deferred_compdefs} )) && \
 for c in "${__deferred_compdefs[@]}"; do eval "compdef $c"; done
unset __deferred_compdefs

None of those moved the startup number. They were still bugs, and now they are not.

Doing this on your own config

Measure before you touch anything. time zsh -i -c exit for cold start, zprof for the in-shell breakdown. Skip that step and you will spend an afternoon optimizing the wrong thing. I have done exactly that.

Then grep your shell history against the alias and function lists of whatever plugins you load. You will usually find you are loading hundreds of definitions to use a handful.

Defer what can be deferred. First-prompt latency is what you feel, and almost nothing actually has to load before you can type. Cache expensive evals whose output only changes when the binary changes, and key the cache on the binary’s mtime so upgrades invalidate it for you. Hardcode the constants: brew --prefix coreutils is always ${HOMEBREW_PREFIX}/opt/coreutils. Forking a Ruby program to ask is silly.

And keep environment variables in .zshenv, not .zshrc. Future-you, debugging why something works in iTerm but not in a cron job, will be grateful.

Estimated total

Change	Savings
Stop forking `brew --prefix` × 5	~180 ms
Defer plugins with `zinit wait`	100-300 ms
Cache remaining `eval`s	50-100 ms
Compile `.zcompdump`	30-80 ms
Drop `fnm` (mise covers it)	30-80 ms
Total	~400-700 ms

These are estimates, not measurements. Your actual numbers depend on your hardware, your plugin set, and how much cruft you had accumulated over the years. Under 200 ms on Apple Silicon is doable with stock zsh and a small, well-deferred plugin set.

My dotfiles, with all of this applied, are at github.com/theskumar/dotfiles. The zsh/OPTIMIZATION.md file there is the running log of what is done and what is still open.

I plan to repeat the audit as the config changes.

Hono

Tue, 10 Feb 2026 12:00:00 +0000

Hono based on web standards is a great option for js runtimes esp. if you plan to deploy your code to cloudflare workers, bun or deno. You get faster startup time and cross-platform compatibility.

Redis’s License Change and the Valkey Fork

Mon, 12 Jan 2026 00:00:00 +0000

The License Change

On March 20, 2024, Redis Inc. announced a switch from the BSD 3-clause license to dual source-available licensing: RSALv2 and SSPLv1. The new licensing takes effect with Redis 7.4 and restricts how the code can be offered as a service.

I see this as a loss for the open-source community, even if Redis Inc. considers it a business necessity.

The Thing About Open Source

Redis was created by Salvatore Sanfilippo in 2009, known as “antirez” in the community. He built Redis to solve a simple problem: web applications were slow because databases couldn’t keep up. What if, instead of storing data on disk, you kept everything in memory? What if you made it blazingly fast?

Redis became legendary. By 2024, it was everywhere—caching layers for social media platforms, session stores for e-commerce sites, real-time analytics for financial systems. Developers loved it. Not just because it was fast, but because it was theirs. They could use it, modify it, deploy it however they wanted.

Then Redis Inc. changed the rules.

The Rebellion

Kyle Davis, a longtime Redis contributor, said something interesting after the split: “From this point forward, Redis and Valkey are two different pieces of software.” Not just different licenses—different software.

Just eight days later, on March 28, 2024, something remarkable happened. The Linux Foundation announced Valkey, a fork starting from Redis 7.2.4, the last truly open version. But this wasn’t some ragtag group of angry developers. This was AWS, Google Cloud, Oracle, Ericsson, and Snap Inc.—companies with deep pockets and deeper technical expertise.

They weren’t just copying Redis and giving it a new name. They were going to improve it.

The Performance Revolution

Here’s where the story gets interesting, because it stops being about philosophy and starts being about engineering.

Redis, for all its speed, had a design constraint: it was single-threaded. Salvatore made this choice deliberately. Single-threading avoids all the complexity of concurrent programming—no locks, no race conditions, no debugging nightmares. For years, this was fine. Processors were getting faster, and Redis was fast enough.

But by 2024, we’re not making processors faster anymore—we’re making them wider. Your laptop has eight cores, maybe sixteen. Your cloud server has dozens. And Redis, with its elegant single-threaded design, was using… one.

The Valkey team saw their opportunity.

Valkey 8.0, released on September 15, 2024, introduced enhanced multi-threaded I/O. Not for the core data operations—those stayed single-threaded to preserve Redis’s simplicity and safety—but for network operations, for handling connections, for moving data in and out.

The results were dramatic. In benchmarks on AWS hardware, Valkey 8.0 hit 1.19 million requests per second. That’s 230% more than Valkey 7.2. That’s the kind of improvement that makes CTOs take notice.

But the engineers didn’t stop there.

The Memory Game

Valkey 8.1, released on March 31, 2025, arrived with something even more impressive: a complete redesign of how data is stored in memory.

The team rebuilt the hash table—the fundamental data structure at the heart of any key-value store—using modern algorithms inspired by Google’s “Swiss Tables” design. Every key-value pair now uses about 20-30 bytes less memory. That might not sound like much, but scale it up.

When researchers tested Valkey 8.1 against Redis 8.2 with 50 million entries, Valkey consumed 3.77 GB while Redis used 4.83 GB. That’s a 28% reduction. For a company running terabytes of cached data across hundreds of servers, that’s not just impressive—it’s millions of dollars in annual savings.

The Feature Race

Of course, efficiency isn’t everything. Redis Inc. didn’t just sit back and watch. In May 2025, Redis released version 8.0, bundling their enterprise features—JSON support, vector search for AI applications, time series databases—directly into the core product. These are sophisticated capabilities that took years to develop.

Valkey’s roadmap is playing catch-up here. Version 8.1 added JSON and Bloom filters. Vector search support became available through the valkey-search module. Time series support is planned. But there’s still a gap.

This is the classic innovator’s dilemma: Redis has more features today. Valkey has better fundamentals and a trajectory that suggests they’ll have those features tomorrow—and they’ll run faster when they do.

The Cloud Complication

Here’s where it gets messy. When you go to AWS and spin up what was formerly called “Redis,” you might now get Valkey. Same with Google Cloud’s Memorystore.

These managed services have their own quirks and limitations, shaped by what the cloud providers choose to support. The result? More fragmentation.

Managed Redis-compatible services now include both projects, with different feature sets and provider support.

The Choice

So where does this leave developers in 2026?

If you’re starting a new project, Valkey offers true open-source freedom, better performance on multi-core systems, and superior memory efficiency. You sacrifice some advanced features, but its BSD license permits use, modification, and redistribution under its terms.

If you need those advanced features right now—vector search for AI applications, sophisticated time series analysis, enterprise support—Redis has them. But you’re accepting the terms Redis Inc. sets, and those terms have already changed once.

For existing Redis deployments, the calculation is harder. Migration costs are real. But so is the risk of depending on a vendor that rewrote the rules mid-game.

My Take

Valkey started with an established codebase and backing from major companies, including AWS, Google Cloud, Oracle, Ericsson, and Snap Inc. Its performance and memory work give developers reasons to evaluate it beyond the license.

I prefer Valkey’s permissive licensing and foundation governance. To me, the fork shows why the ability to keep developing an existing open-source codebase matters when a vendor changes terms. Both projects continue to evolve, and the choice still depends on features, migration costs, and workload measurements.

how uv got so fast

Sat, 27 Dec 2025 12:00:00 +0000

how uv got so fast. UV performance is because of design decisions while rust contributes to micro optimizations.

Django Native PostgreSQL Connection Pooling

Wed, 18 Jun 2025 00:00:00 +0000

Opening PostgreSQL connections adds overhead. Django 5.1 (released August 7, 2024) added native connection pooling¹, which lets an application reuse connections. Measure connection overhead and response times before and after enabling it; the benefit depends on your workload.

If you’re on Django 6+ and running async/ASGI, also see the async pool notes in How Django’s ORM Went From sync_to_async Threads to Native psycopg3.

Native Pooling vs External Pooling

PgBouncer runs as a separate pooling server. Django’s native pooling runs in the application process, so it does not require a separate server. It does require psycopg’s pool package. Third-party Django pooling packages and manual connection management are other options, with their own maintenance and configuration needs.

Configure Native Pooling

Requirements: Django 5.1+ and PostgreSQL (this feature does not work with psycopg2)²

Install the correct package:

pip install "psycopg[binary,pool]"

Critical: The package is called psycopg (not psycopg3). The pool functionality requires a separate package that this command installs automatically³.

Add these lines to your settings.py:

DATABASES = {
 "default": {
 "ENGINE": "django.db.backends.postgresql",
 "NAME": "your_db_name",
 "USER": "your_db_user",
 "PASSWORD": "your_db_password",
 "HOST": "your_db_host",
 "PORT": "5432",
 "CONN_MAX_AGE": 0, # Required - pooling fails without this
 "OPTIONS": {
 "pool": True,
 },
 }
}

Deploy this change. Your app now reuses connections instead of creating new ones for every request.

Prevent the Configuration Error That Breaks Production

Must set CONN_MAX_AGE = 0. Without this, Django throws ImproperlyConfigured: Pooling doesn't support persistent connections and your app crashes⁴. The pool manages connection lifetimes, making Django’s persistence redundant.

Scale for High-Traffic Production

Replace "pool": True with specific parameters for production loads:

DATABASES = {
 "default": {
 "ENGINE": "django.db.backends.postgresql",
 "NAME": "your_db_name",
 "USER": "your_db_user",
 "PASSWORD": "your_db_password",
 "HOST": "your_db_host",
 "PORT": "5432",
 "CONN_MAX_AGE": 0,
 "OPTIONS": {
 "pool": {
 "min_size": 4, # Keeps connections warm
 "max_size": 16, # Handles traffic spikes
 "timeout": 10, # Fails fast under extreme load
 "max_lifetime": 1800, # 30 minutes maximum connection age
 "max_idle": 300, # Close idle connections after 5 minutes
 },
 },
 }
}

Start with max_size equal to expected concurrent users divided by 10. Scale up if you see timeout errors in logs.

Monitor Your Performance Gains

Enable pool monitoring to track your improvements:

# In settings.py
import logging
logging.getLogger('psycopg.pool').setLevel(logging.INFO)

Watch pool utilization, connection wait times, timeout errors, and response times. Record median and 95th percentile latency before and after the change under comparable load. Use those measurements to size the pool and judge whether pooling helps your application.

Critical Fix for Threading and Celery Applications

Essential for background processing: This isn’t a pooling bug — it’s expected Django behavior that pooling simply makes painful. Any connection opened outside Django’s request-response cycle (for example, in a thread you spawn yourself) stays open until you explicitly close it. Without a pool you might not notice; with one, those un-returned connections exhaust max_size and you start seeing PoolTimeout errors⁵. Add explicit cleanup:

from django.db import connection
from threading import Thread

class DataProcessingThread(Thread):
 def run(self):
 # Your database operations here
 User.objects.filter(active=True).update(last_seen=timezone.now())

 # Critical: returns the connection to the pool
 connection.close()

For code that may touch multiple database aliases, Django’s documented helper django.db.close_old_connections() is the more robust choice — it closes all old or unusable connections across every configured database:

from django.db import close_old_connections

class DataProcessingThread(Thread):
 def run(self):
 try:
 User.objects.filter(active=True).update(last_seen=timezone.now())
 finally:
 close_old_connections()

Without this cleanup, each background thread permanently holds a connection from your pool.

Deployment Scenarios That Require Different Approaches

ASGI applications: On Django 5.1–5.2, Django’s documentation recommends against native pooling with ASGI — use PgBouncer or similar external poolers instead. This changed in Django 6.0, which ships an async-aware AsyncConnectionPool that works correctly under ASGI. I cover how that fits together with the truly-async ORM in How Django’s ORM Went From sync_to_async Threads to Native psycopg3.

Serverless deployments: Connection pools don’t persist across serverless invocations. Skip this optimization for Lambda/Cloud Functions.

Multi-tenant applications: Each tenant database needs its own pool configuration in DATABASES.

When to Evaluate Pooling

Evaluate pooling when connection establishment is a measurable part of request latency, concurrency is growing, or the application is approaching PostgreSQL connection limits. Compare measurements before and after rather than assuming a fixed latency or cost saving.

Django 5.1 Release Notes - Official Django documentation announcing native connection pooling support ↩︎
Django Database Configuration - Complete guide to Django database settings including pooling requirements ↩︎
psycopg Installation Guide - Official psycopg documentation for package installation and pool dependencies ↩︎
Stack Overflow: Django Pooling Configuration Error - Community discussion of the CONN_MAX_AGE requirement ↩︎
Django Ticket #35672 - Closed as invalid: connections opened in threads must be returned manually. See also the Databases → Caveats docs, which recommend django.db.close_old_connections() for connections created outside the request-response cycle. ↩︎