Things I Learned

Anthropic gave 50 data analysts a Claude Code agent and a realistic analysis with four open decisions. Each decision had two reasonable answers. In secret, a coin flip told the agent which answer to argue for. It had to argue honestly.

The analysts picked the agent’s answer 86% of the time. Afterwards, 77% explained their answers with the agent’s arguments, sometimes word for word. Only 15% gave the agent credit. Analysts who said “it was my decision” picked the agent’s answer just as often as the rest.

• The coin flips alone predicted 82% of the final recommendations. The agent didn’t need to argue for the recommendation. It had already picked the inputs.
• With no instructions to argue for anything, Opus 4.8 gave the same five answers in 112 of 123 runs. Ten teams on one model agreeing is one model agreeing with itself.

This is the limit of my meat proxy note. Rewriting AI output in your own words proves you read it. It doesn’t prove the decision was yours. For the two or three decisions that matter, write down your answer before you ask the agent.

pi-schema is a JSON Schema validator from the Pi monorepo that never calls eval or new Function. Mario Zechner says he built it with a single prompt. It compiles each schema into plain closures, so it stays fast in places that block code generation.

• The need is real. Cloudflare Workers block eval() and new Function. Chrome MV3 extensions cannot allow 'unsafe-eval'. Ajv, the default JSON Schema validator in JS, compiles schemas with new Function. It broke MCP elicitation on Workers (typescript-sdk#689).
• How people cope today. Ajv can generate validators at build time, but that only works for schemas you know in advance. The MCP SDK lists @cfworker/json-schema, an interpreter, as a peer dependency. Zod and ArkType have jitless flags, and TypeBox falls back to its interpreter. Zod’s flag has to be set before any import that uses Zod (zod#5789).
• The numbers. Without eval, the README puts a nested schema at 407 ns for pi-schema, 42 µs for TypeBox and 43 µs for cfworker. pi-schema passes 2,222 of the 2,223 core tests in the official suite.

Agents make this matter. An agent learns its tools from MCP servers while it runs: tools/list returns each tool’s JSON Schema. You cannot compile those ahead of time, and the harnesses are moving to the edge. Pi’s Chord design notes plan a Cloudflare Durable Object driver, and Pi Durable already stores agent work as checkpointed tasks. The catches: these are the author’s own benchmarks, Ajv is not in the comparison, the package is not on npm yet, and TypeBox’s JIT still wins on small flat schemas (9 ns vs 31 ns).

nicobailon/skill-diet is an agent skill that slims your other agent skills, then proves the slim version still works. It runs each task with the old skill, the new one and no skill at all, 3 times each, and keeps the cut only if the new one scores at least as well.

I ran it on three of my skills with gpt-6-luna:

• adr: SKILL.md went from 13 KB to 1.6 KB, 30/30 before and after. With no skill it still passed 26/30. The model already knew ADRs.
• notion: 32/33 to 33/33 at 55% fewer tokens per run. With no skill, the agent did one-line edits with a whole-page replace and reported success.
• web-browser: 16 KB to 6.4 KB. The no-skill runs used twice the tokens of the old skill, because the agent explored the scripts by hand.

The size cut was not the best part. For a skill that wraps a tool, it checks every claim against the tool and your session logs. It found 16 places where my skills or scripts were wrong, such as agent.mjs --domain x.com silently setting no allow-list. The rule I’m keeping: fix the tool’s error message before adding a line to the skill.

The cost: 1,434 agent sessions and 23.5M tokens for three skills, and a sandbox-exec profile, because the test agents kept finding my installed skills on disk.

The diet commit shows what got cut, and the full report covers the method and the costs.

django-bolt serves Django from a Rust server, with one Python worker thread per process. In an independent benchmark it ran 1.6 to 1.7x django-ninja on real database endpoints. The catch: a slow sync handler must move to a thread pool, or every request in that process waits. Bolt decides at startup by scanning each handler’s source for ORM calls like .objects.get.

The scan reads only the handler. return Order.objects.get(id=id) goes to the pool. return services.get_order(id) runs the same query inline on the worker thread, and the whole process stalls until it returns. A codebase with a service layer hits this on nearly every route. The way out is async handlers: async def puts the waiting in the signature, so it can’t hide one call down.

A scan of one function sees what’s written there, not what it calls. A fact that has to cross calls belongs in the signature, which is Ron Minsky’s argument for types.

Charles Miller: “a skill that can be picked up in a few weeks, and will be obsolete in a few months, has no value outside the moment.” His mechanism: a prompt trick that works gets folded into the product and ships as the default a few months later. His analogy is the 1990s C belief that ++i beats i++, which survived long after compilers erased the difference.

He’s right about tricks, but not every AI skill is a trick. Verification, choosing context and deterministic gates depend on your problem, not the model, so no release absorbs them. Ron Minsky argues they matter more with agents. My test when I learn something: is this about the model or about the problem? Model tricks go in a skill file I can delete on release day.

Ron Minsky’s Bug Bash 2026 talk argues Jane Street’s old habits (rich types, expect tests, real code review) matter more with agents: they give the model fast, exact feedback and keep the code readable for the humans who own it. He goes the other way from the Claude Code team on review: its main job is keeping a human able to explain the system, not catching bugs.

The idea I’m taking: expect tests. The test prints output, the expected text lives inline, and a change shows up as a diff of behavior, not just code. Python’s version is inline-snapshot: assert result == snapshot(), filled in by pytest.

• Gotcha with agents: under --inline-snapshot=fix (or create, review) every snapshot comparison returns True and the file is rewritten. An agent chasing green doesn’t need to delete a test; it re-accepts the wrong output. Keep acceptance with a human and never auto-fix in CI.

viewflow/seedkit is an agent skill that scaffolds a Django project by asking you questions, then writes only the parts you picked. The edge over cookiecutter: it also works on a live repo. /django-seedkit add proper auth inventories what exists and wires the new package in midway: deps, settings, .env example, CI step. A deterministic generator runs once, on day zero.

The layout is the part worth copying. One SKILL.md walks the questions and routes each answer to exactly one file in references/ (auth.md, billing.md, tasks-celery.md, about 50 of them). Each file keeps only what the model gets wrong: current setting names, current packages. Their benchmark: Sonnet 5 with the skill passed 72 of 72 security and config checks across nine projects; Fable 5 without it missed about a quarter.

The pattern generalizes past Django: one orchestrator skill that asks the questions, plus a reference pack per stack (Django, Next.js, Flutter). Seedkit ships only the Django pack today.

EmbeddingGemma 2 puts text, code, images, video and audio (or a mix) into one 768-dimension vector, from a 740M open model under Apache 2.0. Image, video and audio tokens are spliced into the text sequence and run through one encoder, so a text query can find a video moment or a scanned PDF page without captioning or transcribing first.

• It’s not the quality leader. Text is flat against v1; the gain is code (MTEB Code 68.8 to 78.7). Audio (49) and video (51) scores are modest. All numbers are Google’s own.
• Every major vendor ships one-space multimodal embeddings now: Amazon Nova, Gemini Embedding 2, Voyage, Cohere, and the open Qwen3-VL-Embedding, which reports higher image and document scores but has no audio. Pick by constraint. EmbeddingGemma 2’s niche is open weights plus audio.
• Matryoshka truncation isn’t free: its MMEB score falls from 59.0 at 768 dimensions to 45.7 at 128.
• The interesting part is the system around the model. DigUp is local file search on it: one vector per photo, PDF page, video shot and 30 s of audio, each pointing back to its page or second, next to an FTS5 keyword index, because no embedding holds an invoice number exactly.
• DigUp fuses the two by turning cosine into a per-query z-score (how far a match stands out for this query), gating on it, and adding a word-coverage bonus. It beat reciprocal-rank fusion on their evals (R@1 0.88 vs 0.73) and can honestly say “nothing found”. Small, author-run eval.
• It also pins llama.cpp, records the model version in the index, and checks every upgrade against stored reference vectors. Vectors from two runtimes don’t mix, and nothing errors when they do.