Known Agents reviewed: AI-agent analytics and robots.txt, formerly Dark Visitors
Known Agents (formerly Dark Visitors) tracks AI agent and bot traffic to your site in real time, generates a continuously-updated robots.txt to block misbehaving scrapers, and offers an agent-identification API plus a public agent directory.
What Known Agents does
Known Agents gives a site owner three core capabilities: real-time analytics of AI agent and bot traffic, automatic robots.txt generation that updates as new agents appear, and an agent-identification API for authenticating and controlling access to specific agents. It also maintains a public agent directory and adds ecommerce-focused features to monitor AI shopping activity on product pages.
Where it fits in the agentic web
This is a crawler-analytics tool — it answers “which specific bots fetch my pages, and should I allow or block them?” Its catalogue overlaps the entities in the AI Crawler & Agent Registry, and its identification work is the same job that crawler verification and Web Bot Auth address. Its directory is a useful cross-reference for the exact user-agent tokens you set rules against.
Our take
The specialist for identifying and gating individual crawlers, and its live agent directory is genuinely useful when you are writing robots.txt rules. Like all analytics tools it measures and lists traffic — it does not certify that your site is agent-ready. Use its directory to know who is visiting, then the Agent-Readiness Engineering guide to decide what to expose to them.