Sun, Jul 26, 2026

django-crawl by Adam Johnson: BFS-crawls your Django site using the test client, no real HTTP overhead. Origin story: diffing CSP headers during a Django 6.0 migration accidentally caught 7 non-CSP bugs in a 100%-coverage project. HTML parsing via Rust (html5ever), follows links/sitemaps/feeds. Works as a management command or a TestCase smoke test that raises ExceptionGroup on errors. discuss

Embedding pipelines do not feed whole documents straight into the model. They first use the embedding model's tokenizer to measure and split text into token-bounded, overlapping chunks, then embed each chunk and store the vectors. Tokenizer must match the model. Tools like marcelroed/gigatoken accelerate this preparation at corpus scale but do not create embeddings themselves.

Go interfaces are implicit: a type satisfies an interface by having the right methods, no implements keyword.

• Closest Python equivalent is typing.Protocol (structural subtyping, checked by mypy), but Go enforces it at compile time, not via a separate type checker.
• Sharp edge: a nil concrete pointer wrapped in an interface value is not nil. var dog *Dog = nil; var s Speaker = dog gives s != nil because the interface holds (*Dog, nil), a typed pair. Bites hardest when returning error interfaces.
• Go proverb: "the bigger the interface, the weaker the abstraction." Keep them to 1-3 methods, define where consumed not where implemented.