#harness

TIL notes

• Oct 27, 2026

  Ron Minsky’s Bug Bash 2026 talk argues Jane Street’s old habits (rich types, expect tests, real code review) matter more with agents: they give the model fast, exact feedback and keep the code readable for the humans who own it. He goes the other way from the Claude Code team on review: its main job is keeping a human able to explain the system, not catching bugs.

  The idea I’m taking: expect tests. The test prints output, the expected text lives inline, and a change shows up as a diff of behavior, not just code. Python’s version is inline-snapshot: assert result == snapshot(), filled in by pytest.

  • Gotcha with agents: under --inline-snapshot=fix (or create, review) every snapshot comparison returns True and the file is rewritten. An agent chasing green doesn’t need to delete a test; it re-accepts the wrong output. Keep acceptance with a human and never auto-fix in CI.
  pythoncode-review
• Oct 23, 2026

  viewflow/seedkit is an agent skill that scaffolds a Django project by asking you questions, then writes only the parts you picked. The edge over cookiecutter: it also works on a live repo. /django-seedkit add proper auth inventories what exists and wires the new package in midway: deps, settings, .env example, CI step. A deterministic generator runs once, on day zero.

  The layout is the part worth copying. One SKILL.md walks the questions and routes each answer to exactly one file in references/ (auth.md, billing.md, tasks-celery.md, about 50 of them). Each file keeps only what the model gets wrong: current setting names, current packages. Their benchmark: Sonnet 5 with the skill passed 72 of 72 security and config checks across nine projects; Fable 5 without it missed about a quarter.

  The pattern generalizes past Django: one orchestrator skill that asks the questions, plus a reference pack per stack (Django, Next.js, Flutter). Seedkit ships only the Django pack today.

  django
• Oct 17, 2026

  How the Claude Code team does code review. Claude writes ~90% of Claude Code, so nobody reads every diff. They swapped “did a human read it?” for “which check, automated or human, should catch what could go wrong here?”, and optimise for the least human time per change, with revert and incident rates as the guardrail.

  • The bit I missed on first read: review noise costs more when the author is an agent. A person waves off a nit in seconds; an agent fixes every one, which is a new commit and another full review. One PR ran 50+ rounds overnight, and ~59% of later findings were on code already there at round one. Fix: cap at three rounds, re-review only the new diff, flag the class of bug, never repeat a finding.
  • The bot posts findings, never approvals. Human approvals (zero, one or two) are set by the change’s risk, not by who wrote it. Whoever prompted the agent counts as a reviewer, and an approval covers one commit, so an agent’s next push dismisses it.
  • Their reviewer’s eval is their own past incidents and reverts, run daily. Noise is measured by what authors did with each comment: fixed it, said it was wrong, intentional, or not worth it.
  • Versus Ankit Jain’s review layers I noted in April: this is the production version, with no spec-review layer and fast rollback under everything.
  • Read the numbers carefully: the share of PRs reverted fell 40% while PR volume doubled, so the count of reverts still rose (~1.2x). Also: the authors sell the reviewer.
  • Where I push back: “nobody needs a full mental model, ask Claude for an explainer of the slice you’re changing.” An explainer recovers what the code does, not why it was built that way, which is the comprehension debt Addy Osmani (a co-author here) wrote about. People should spend review on whether to ship it and on direction, not line-level correctness.
  • How I read it: had Claude explain it against my notes, then quiz me. The question I got wrong taught me the most.
  claude-codecode-reviewDiscuss
• Oct 10, 2026

  Microsoft eXecution Container (MXC) is an embeddable SDK for running untrusted commands under OS-enforced policies, not a hosted sandbox service.

  • Defaults: Seatbelt on macOS, Bubblewrap on Linux, ProcessContainer on Windows. SDKs for Node, Rust, and .NET.
  • Restricts filesystem and network access. macOS hostname allowlists need a caller-managed proxy; not every backend is a VM.
  • Useful boundary: sandbox the command where it executes, not just the agent’s tool-composition script. Isolation limits damage; it does not make generated output trustworthy.
  securityDiscuss
• Oct 9, 2026

  Pi Pocket’s subagent implementation gives each worker a retained conversation and uses durable tasks to deliver messages and report answers. Request IDs and recorded answer IDs prevent duplicate delivery and reporting after restart. The idea worth stealing: make the handoff durable, not just the worker.

  pi-agentDiscuss
• Oct 9, 2026

  Arnav Gupta’s pi-subagent-manager makes context inheritance explicit. Nested threads receive their parent’s snapshot; independent roots start without its history. Threads stay around for steering and resume. I want synchronous coordination by default, forks for continuity and fresh context for independent review.

  pi-agentDiscuss
• Oct 9, 2026

  Armin Ronacher’s subagent extension runs one observable Pi child at a time in tmux. The parent waits for the result, while I can attach to inspect or intervene. Worth preserving when adding hierarchy: synchronous coordination must not mean invisible execution.

  pi-agentDiscuss
• Oct 7, 2026

  Pi Durable is Earendil’s experimental agent harness, shipped alongside Pi 1.0. TannerMidd/pi-pocket is a mobile-first, multiplayer web app for Pi built on it.

  • Every model call and tool call is a task that checkpoints to storage (SQLite, JSONL, or memory) before moving on. Kill the process and a new one opens the same file and resumes.
  • Recovery is per step: a cut-off model request is resent with the partial answer kept and marked aborted. A cut-off tool call reruns only if it is replay-safe; otherwise the model is told it was interrupted.
  • A requestId makes a submission exactly-once, so a client retrying after a crash gets the original back.
  • Forks see the parent transcript up to the fork point without copying it.
  • ~15k lines total, small enough for an agent to read end to end.
  • pi-pocket: restart the server mid-run and the work continues. Several people can watch, steer, or queue messages on the same session.
  • npm start -- --access tailscale binds only to the tailnet address, so my phone gets it with no tunnel.
  • The shift: the session lives in a SQLite file, not a terminal process, so the phone is just another client. Could replace my Moshi + tmux setup.
  • Caveat: anyone who can steer runs shell commands as you.
  pi-agentDiscuss
• Sep 5, 2026

  Every coding agent (Claude Code, Codex, pi) compacts long sessions the same way: ask an LLM to prose-summarize, then summarize that summary, until load-bearing details erode (why a decision was made, which approaches got rejected). pi-blackhole swaps that for deterministic compaction: a zero-token compile() that extracts structured sections (goal, files touched, commits, preferences, brief transcript). No LLM, no drift, no cost. The compile is only lossy in the window, not on disk: blackhole also ships a recall tool (BM25 + regex over the raw session JSONL and the memory ledger) that re-fetches any dropped turn on demand, so compile + recall together is lossless where an LLM summary-of-a-summary destroys the original. It also adds observational memory: background observer/reflector workers append durable facts to per-session ledgers that survive every compaction, and /blackhole-export walks every past-session JSONL under a git-root into a ranked context file. The clever part is what export trusts. A fact’s rank is driven by cross-session recurrence, damped as log(1 + distinctSessions) with a burst penalty, not raw counts, so one chatty session can’t inflate it. Recurrence-as-reinforcement is basically Hebbian: facts that keep firing across sessions get stronger. Dedup is a 3-pass funnel, exact then Levenshtein then Sørensen-Dice token-set, with a Levenshtein floor so keyword overlap alone can’t merge two distinct facts. Bonus: the OM workers run on their own model and billing, so I point them at openai-codex/gpt-5.6-luna while the main thread stays on Claude.

  pi-agentDiscuss
• Jul 28, 2026

  Python’s Ellipsis literal (...) is a real object (type(...) is <class 'ellipsis'>), not just a placeholder. Beyond type stubs and numpy slicing, NVIDIA’s NOOA framework repurposes it as a runtime dispatch marker: methods with ... bodies are completed by an LLM at runtime, while methods with real bodies stay deterministic Python. The boundary between “code I control” and “code the model controls” becomes greppable: grep -n '\.\.\.' agent.py. Clever reuse of existing syntax that linters and type checkers already accept.

  pythonDiscuss
• Jun 8, 2026

  [karpathy/autoresearch] is a concept/framework that allows one to leverage LLM to run an optimization loops based on a criteria. davebcn87/pi-autoresearch takes this further has built a generic optimization long run loop method on top of pi agent. gemini

  performance
• May 29, 2026

  Dynamic workflows in Claude Code. Claude dynamically writes orchestration scripts that spawn tens to hundreds of parallel subagents. It plans, decomposes into subtasks, fans out agents, and checks results before returning a coordinated answer. Adversarial agents try to refute findings, iterating until convergence. Enable via ultracode setting or ask Claude to “create a workflow”. Notable: Jarred Sumner used it to port Bun from Zig to Rust (~750K lines, 11 days). Available on Max, Team, Enterprise plans. Consumes significantly more tokens than typical sessions.

  claude-code
• May 9, 2026

  Use HTML instead of markdown to effectively plan and review. Claude Code: The Unreasonable Effectiveness of HTML" / X

  claude-codemarkdown
• May 9, 2026

  colbymchenry/codegraph: Pre-indexed code knowledge graph for Claude Code — fewer tokens, fewer tool calls, 100% local

  claude-code
• Feb 15, 2026

  K-dense known for using skills to enable deep research has published 140+ skills related to scientic research including literature review, data analysis, etc.

• Dec 24, 2025

  “LangGraph is an orchestration framework for building stateful multi-agent applications using LLMs. It provides low-level primitives such as nodes and edges, along with built-in features that give developers granular control over agent workflows, memory management and state persistence. This means developers can start with a simple pre-built graph and scale to complex, evolving agent architectures. With support for streaming, advanced context management and resilience patterns like model fallbacks and tool error handling, LangGraph enables you to build robust, production-grade agentic applications. Its graph-based approach ensures predictable, customizable workflows and simplifies debugging and scaling.”

  python