# django-crawl

By Saurabh Kumar · TIL note, Jul 24, 2026 · Tags: django

Source: https://saurabh-kumar.com/til/1784894401/

[django-crawl](https://django-crawl.readthedocs.io/en/latest/) by Adam Johnson: BFS-crawls your Django site using the test client, no real HTTP overhead. [Origin story](https://adamj.eu/tech/2026/07/22/introducing-django-crawl/): diffing CSP headers during a Django 6.0 migration accidentally caught 7 non-CSP bugs in a 100%-coverage project. HTML parsing via Rust (html5ever), follows links/sitemaps/feeds. Works as a management command or a TestCase smoke test that raises `ExceptionGroup` on errors.