Saurabh Kumar

#code-review

Code review practice, human and AI

TIL notes

  • Oct 27, 2026

    Ron Minsky’s Bug Bash 2026 talk argues Jane Street’s old habits (rich types, expect tests, real code review) matter more with agents: they give the model fast, exact feedback and keep the code readable for the humans who own it. He goes the other way from the Claude Code team on review: its main job is keeping a human able to explain the system, not catching bugs.

    The idea I’m taking: expect tests. The test prints output, the expected text lives inline, and a change shows up as a diff of behavior, not just code. Python’s version is inline-snapshot: assert result == snapshot(), filled in by pytest.

    • Gotcha with agents: under --inline-snapshot=fix (or create, review) every snapshot comparison returns True and the file is rewritten. An agent chasing green doesn’t need to delete a test; it re-accepts the wrong output. Keep acceptance with a human and never auto-fix in CI.
    pythonharness
  • Oct 17, 2026

    How the Claude Code team does code review. Claude writes ~90% of Claude Code, so nobody reads every diff. They swapped “did a human read it?” for “which check, automated or human, should catch what could go wrong here?”, and optimise for the least human time per change, with revert and incident rates as the guardrail.

    • The bit I missed on first read: review noise costs more when the author is an agent. A person waves off a nit in seconds; an agent fixes every one, which is a new commit and another full review. One PR ran 50+ rounds overnight, and ~59% of later findings were on code already there at round one. Fix: cap at three rounds, re-review only the new diff, flag the class of bug, never repeat a finding.
    • The bot posts findings, never approvals. Human approvals (zero, one or two) are set by the change’s risk, not by who wrote it. Whoever prompted the agent counts as a reviewer, and an approval covers one commit, so an agent’s next push dismisses it.
    • Their reviewer’s eval is their own past incidents and reverts, run daily. Noise is measured by what authors did with each comment: fixed it, said it was wrong, intentional, or not worth it.
    • Versus Ankit Jain’s review layers I noted in April: this is the production version, with no spec-review layer and fast rollback under everything.
    • Read the numbers carefully: the share of PRs reverted fell 40% while PR volume doubled, so the count of reverts still rose (~1.2x). Also: the authors sell the reviewer.
    • Where I push back: “nobody needs a full mental model, ask Claude for an explainer of the slice you’re changing.” An explainer recovers what the code does, not why it was built that way, which is the comprehension debt Addy Osmani (a co-author here) wrote about. People should spend review on whether to ship it and on direction, not line-level correctness.
    • How I read it: had Claude explain it against my notes, then quiz me. The question I got wrong taught me the most.
    claude-codeDiscuss
  • Aug 4, 2026

    Someone finally named it: meat proxy. You paste Claude’s answer into Slack or a PR review without reading it, and you have made yourself the pipe between me and a model I could have prompted myself, with better context. Everyone already knows that annoyance. Nobody had the word for it. The word is the useful part, because “don’t be a meat proxy” is something you can actually say to a colleague: it names a role instead of calling them lazy. Best line is about code review. Paste the ticket into Claude, paste the reviewer feedback back in, read neither, and the reviewers implemented your PR while you held the pipe. The bar before forwarding anything is to write it in your own words, “a decent certificate that you’ve done the prior steps.” 704 comments on HN, barely any disagreement.

    ai
  • Apr 22, 2026

    A good collection of practices on automated AI code reviews by Ankit Jain

    • The Scalability Crisis: Manual post-PR review is no longer viable. AI agents have nearly doubled code output, causing human review time to spike by 91%, creating a bottleneck that traditional workflows cannot solve.
    • The Upstream Pivot: Human value must shift from reviewing implementation to defining intent. Instead of checking syntax, humans spend their energy writing rigorous specs and acceptance criteria before the code is written, which the machine then uses to self-verify.
    • The Swiss-Cheese Defense: Rather than one “perfect” human gate, the model uses a stack of imperfect automated layers. By layering signals like agent competition, deterministic guardrails, and adversarial “red-team” agents, the system catches errors where their individual failure modes don’t overlap.
    ai
  • Mar 17, 2026

    Every layer of review makes you 10x slower via apenwarr