# The Agentic-Web Tooling Landscape

> The agentic-web tooling landscape spans four jobs — measuring adoption (Cloudflare Radar), analyzing crawlers (Known Agents), tracking AI visibility (Profound, Ahrefs Brand Radar, Semrush), and checking readiness (Cloudflare's scanner, AgentReady.org, llms.txt generators, specification.website). This page reviews each one neutrally — including our own Agent-Readiness Audit.
> · Markdown twin of /tools · last_verified: 2026-06-15

## Agentic-web tools, defined

An agentic-web tool is a product that measures, analyzes, tracks, or audits how a website participates in the agentic web — reviewed here across four jobs: adoption measurement, crawler analytics, AI-visibility tracking, and readiness checking. Each review carries six fields: what_it_does, strength, weakness, monetization, our_take, source. Neutrality rule: every tool gets a genuine strength; every weakness is a fact, never a put-down. A tool measures or checks; a standard specifies; a service engineers and certifies.

## Four tool categories

| Category | What it does | Tools | Bridges to |
|---|---|---|---|
| Adoption-measurement | Counts what crawls the web | Cloudflare Radar | /state-of-the-agentic-web |
| Crawler-analytics | Names who crawls your site | Known Agents (ex-Dark Visitors) | /crawlers |
| GEO / AI-visibility | Tracks whether AI cites you | Profound · Ahrefs Brand Radar · Semrush | /geo |
| Readiness-checking | Tests whether your site is agent-ready | Cloudflare scanner · AgentReady.org · llms.txt generators · specification.website | /services |

## Cloudflare reviewed

Cloudflare is the broadest agentic-web tool: a free agent-readiness scanner, the only large-scale crawl-traffic dataset (Cloudflare Radar), and pay-per-crawl access control — the default starting point, but CF-flavored, not vendor-neutral.

- **Strength:** best free first scan; canonical live crawl-traffic data (Radar /bots — verify figures against primary at build); authoritative on verification standards.
- **Weakness:** ecosystem-locked, CF-flavored, no teaching/reference layer.
- **Monetization:** freemium scanner → paid edge/CDN + pay-per-crawl revenue share.
- **Our take:** best free first scan and canonical adoption source — pair with a neutral reference.

## Known Agents reviewed (formerly Dark Visitors)

Known Agents catalogs ~600 bots across 16 categories with per-agent pages and a robots.txt generator — the deepest crawler list, though gated and thin on protocols, models and glossary (verify bot count and rebrand against knownagents.com at build).

- **Strength:** bot-coverage breadth; per-agent detail; usable robots.txt generator.
- **Weakness:** gated tiers; crawler-only; block-first framing.
- **Monetization:** freemium analytics + paid plans.
- **Our take:** the tool to identify and allow/block specific crawlers.

## Profound, Ahrefs Brand Radar and Semrush reviewed

GEO / AI-visibility tools that track whether AI answer engines cite your brand — they measure the outcome (visibility), not the cause (the readiness signals that earn citations). Profound is purpose-built; Ahrefs Brand Radar bolts onto an SEO suite; Semrush is a module in a marketing platform.

- **Strength:** quantify share-of-model across engines.
- **Weakness:** mostly enterprise-priced; measurement-only.
- **Monetization:** SaaS subscription, mostly enterprise (no pricing asserted — verify at build).
- **Our take:** measure visibility here; engineer it with the GEO guide and the Audit.

## llms.txt generators reviewed

Lightweight tools that write an llms.txt (and often llms-full.txt) declaring your content to agents — the cheapest, fastest first step, but a starting point, not a finished readiness posture.

- **Strength:** near-zero effort; immediate machine-readability; often free/open-source.
- **Weakness:** single-signal; validates nothing else; output quality varies.
- **Monetization:** mostly free / open-source, some freemium SaaS.
- **Our take:** generate the file, then verify the whole readiness posture (Layer-1 discovery only).

## AgentReady.org and specification.website reviewed

AgentReady.org is an emerging neutral readiness checklist; specification.website is Joost de Valk's deliberately minimal, high-authority spec. Both are neutral and unmonetized; AgentReady.org is the closest public overlap with our Audit.

- **AgentReady.org** — strength: neutral, standards-adjacent checklist. Weakness: a checklist, not a full audit-with-certification.
- **specification.website** — strength: strong E-E-A-T (Yoast founder), neutral, sparse. Weakness: deliberately non-pedagogical, unmonetized, minimal.
- **Our take:** cite both as neutral peers; our differentiation is the complete reference plus validated audit and certification neither ships.

## Tooling comparison

| Tool | Category | Monetization | Free tier? | Best job | Where it stops |
|---|---|---|---|---|---|
| Cloudflare | Readiness + adoption | Freemium + revenue-share | Yes (scanner) | First scan & adoption data | CF-flavored; no neutral reference |
| Known Agents | Crawler-analytics | Freemium | Partial | Identify/allow/block crawlers | Crawler-only; gated |
| Profound | GEO / AI-visibility | Enterprise SaaS | No | Track answer-engine presence | Measures, does not engineer |
| Ahrefs Brand Radar | GEO / AI-visibility | SaaS subscription | No | AI visibility for Ahrefs users | Measures, does not engineer |
| Semrush | GEO / AI-visibility | SaaS subscription | No | AI visibility for Semrush users | Measures, does not engineer |
| llms.txt generators | Readiness (declaration) | Free / open-source | Yes | Cheapest first readiness step | Single-signal; validates nothing |
| AgentReady.org | Readiness-checking | Unmonetized | Yes | Neutral readiness checklist | Checklist, not audit + cert |
| specification.website | Readiness (spec) | Unmonetized | Yes | A reference spec to build against | Minimal by design |

Every tool measures, lists, or checks — none certifies.

## The Agent-Readiness Audit positioned among the tools

Among these tools, the Agent-Readiness Audit is a validating readiness check that ties its result to a vendor-neutral reference and an 'Agents Welcome' certification badge — the cleanest fully-unowned lane in the category. It does not win on any single axis: Cloudflare is broader on edge/data, Known Agents deeper on bots, the GEO trackers stronger on visibility. Our differentiation is neutral reference + validated audit + certification. Self-demonstrating: this site scores 100 — verify it live.

## From measuring the agentic web to making your site ready for it

Every tool above tells you where you stand; turning that reading into shipped changes — and a certified result — is the job of the Agent-Readiness Engineering guides (/agent-readiness) and the Audit (/services). See also: /state-of-the-agentic-web, /crawlers, /geo, /protocols, /glossary, /models.