Blog

A newly updated URL analyser checks robots.txt, sitemaps, noindex conditions, dead ends, schema and various AI-search signals in one audit. Useful tooling, but it remains primarily a static crawl-readiness analyser.…

Internal-link restructuring reportedly increased Google impressions but reduced AI citations Reddit case report — 2 September A site owner reports restructuring around 80 pages into a more deliberate hub-and-spoke architecture.…

Cloudflare’s 15 September changes classify crawlers by Search, Agent and Training behaviour. Because some crawlers are multi-purpose, blocking training activity can also block a crawler involved in Search.

Only 51.8% of Sites Can Answer “Nothing Changed” We sent 195 conditional requests to 60 sites on 5 September 2026. Only 51.8% answered 304, so AI crawlers keep re-downloading files…

Let an Agent Read Your AI Crawler Logs and Propose robots.txt Fixes A concrete loop where an agent reads AI crawler logs, finds the pages crawlers cannot reach, and proposes…

The complete point-in-time record: the question asked, every verbatim answer from the 9 AI models asked, and the ranking computed from them. Captured September 12, 2026; the answers are kept…