webtelemetry.dev on a laptop and a phone: the homepage with live figures and the AI crawlers page

Building webtelemetry.dev: A Live Survey of the Web as It Launches

Why and how I built webtelemetry.dev: a live survey of new websites, and my way of learning how to process large amounts of data with Kafka and PostgreSQL. Plus what it found about AI crawlers and llms.txt.

September 28, 2026 · 10 min · 2124 words · Feda Hadziselimovic