The Agentic Web Lexicon

The semantic representation of a page that assistive tech — and browser-driving agents — read instead of pixels.

Derived from semantic HTML and ARIA and specified by the W3C Core Accessibility API Mappings, it exposes roles, labels and structure. Agents that drive a browser act through this tree, which is why accessible markup doubles as an agent interface: one investment, two audiences.

source https://www.w3.org/TR/core-aam-1.1/

see also: webmcp, agent-experience

A visible, timestamped record of the actions an agent has taken.

The 'what did happen' half of agent trust (intent preview is the 'what will happen' half). An auditable trail lets a principal verify and, if needed, undo what an agent did.

source https://biilmann.blog/articles/introducing-ax/

see also: intent-preview, agent-experience

A JSON document an A2A agent publishes at /.well-known/agent-card.json that advertises its identity, skills, endpoints and authentication.

The Agent Card is A2A's discovery primitive — a machine-readable 'business card' hosted at a well-known URI (per RFC 8615) listing the agent's name, description, version, service endpoints, supported interfaces, capabilities (e.g. streaming) and the AgentSkill objects it offers. A client reads the card to decide whether and how to delegate a task to the agent.

source https://a2a-protocol.org/dev/topics/agent-discovery/

see also: a2a, agent-identity, agntcy

The discipline of designing products and websites so that AI agents can use them effectively — the agent-era counterpart to UX.

Coined by Netlify CEO Matt Biilmann in January 2025. AX asks: when an agent (not a human) is the user, can it discover what your service does, understand its options, take action, and have its principal trust the result? Biilmann's framework has four pillars — Access, Context, Tools and Orchestration.

source https://biilmann.blog/articles/introducing-ax/

see also: agentic-web, intent-preview, action-audit

A proxy that sits between agents and the tools or models they call, enforcing security, access-control and observability policies on agent traffic.

An agent gateway (sometimes 'agent firewall' for the security-focused variant) is a networking layer built on agent-native protocols like MCP and A2A. It inspects and governs agent-to-tool, agent-to-model and agent-to-agent calls — applying policy, redacting secrets and PII, blocking prompt-injection and SSRF, and logging everything. It is where a site can centrally control what visiting or internal agents are allowed to do.

source https://agentgateway.dev/

see also: prompt-injection, agent-identity, mcp

A verifiable answer to 'which agent, acting for whom, is making this request?'

Built from signed requests (Web Bot Auth / RFC 9421), declared user-agents and operator-published verification (IP ranges, reverse DNS). Strong agent identity is the precondition for agent-native access control and commerce.

source https://datatracker.ietf.org/doc/html/rfc9421

see also: web-bot-auth, agentic-commerce

The coordination layer that routes tasks between multiple agents and tools — deciding what runs, in what order, and how results are combined.

Orchestration is the control plane of a multi-agent system: an orchestrator (or 'supervisor' agent) decomposes a goal, dispatches sub-tasks to the right agents or tools, manages dependencies and parallelism, and assembles the final result. It is the difference between a pile of agents and a system.

source https://en.wikipedia.org/wiki/Orchestration_(computing)

see also: multi-agent-system, delegation, a2a

An open protocol from Google that gives an AI agent a cryptographically signed mandate proving a human authorized it to spend, before any payment is made.

Announced by Google on 16 September 2025 with more than sixty payments and technology partners (including Mastercard, PayPal, American Express, Coinbase and Adyen), AP2 introduces tamper-proof 'mandates' — signed digital contracts that prove a user authorized a specific transaction. It is payment-rail agnostic (cards, bank transfers, stablecoins) and is designed to layer on top of A2A and settlement protocols like x402.

source https://cloud.google.com/blog/products/ai-machine-learning/announcing-agents-to-payments-ap2-protocol

see also: x402, agentic-commerce, a2a

An open standard from Anthropic that packages procedural knowledge as a folder with a SKILL.md file an agent discovers and loads on demand.

A Skill is a directory containing a SKILL.md (metadata plus instructions), optional scripts and resources, which an agent loads dynamically only when relevant — keeping the context window lean. Anthropic unveiled Agent Skills on 16 October 2025 and released it as an open standard at agentskills.io on 18 December 2025; Microsoft, OpenAI, GitHub, Figma and Cursor adopted it.

source https://claude.com/blog/skills

see also: context-engineering, mcp, tool-use

The pattern where an AI agent, not a human, searches, evaluates and completes a purchase on the user's behalf.

Agent-as-buyer is the demand side of agentic commerce: the agent compares options, makes the decision and transacts, with the human authorizing scope in advance. It changes optimization targets — product data must be machine-parseable, APIs and structured feeds matter more than visual merchandising, and checkout must accept protocols like ACP, AP2 and x402. McKinsey has projected agentic commerce could reach $1 trillion in US retail by 2030.

source https://en.wikipedia.org/wiki/Agentic_commerce

see also: agentic-commerce, acp, agentic-seo

A protocol for agents to discover and delegate tasks to each other, each publishing an Agent Card of its skills.

Where MCP connects an agent to tools, A2A connects agents to other agents. Announced by Google on 9 April 2025 and donated to the Linux Foundation in June 2025 for neutral governance. It defines Agent Cards (capability advertisements), Tasks (the work exchanged) and a transport over HTTP/SSE/JSON-RPC 2.0, with payment extensions (x402, AP2) layered on the task lifecycle.

source https://en.wikipedia.org/wiki/Agent2Agent

see also: mcp, ap2, agentic-commerce

A Linux Foundation body, formed in December 2025, that provides neutral governance for core agentic-web standards including MCP, the goose agent and AGENTS.md.

The AAIF (Agentic AI Foundation) was formed under the Linux Foundation on 9 December 2025 to steward agent infrastructure as vendor-neutral commons. Platinum members are AWS, Anthropic, Block, Bloomberg, Cloudflare, Google, Microsoft and OpenAI; its inaugural projects are the Model Context Protocol (MCP), the goose agent and AGENTS.md.

source https://www.linuxfoundation.org/press/linux-foundation-announces-the-formation-of-the-agentic-ai-foundation

see also: mcp, a2a, agents-md

Transactions initiated and completed by AI agents on a user's behalf, through protocols like x402, AP2 and the Agentic Commerce Protocol.

Spans micropayments for data and tools (x402), mandate-based settlement across rails (AP2) and in-conversation checkout (ACP / Instant Checkout). The shared challenge is authorization: proving the agent had the user's permission to spend.

source https://openai.com/index/buy-it-in-chatgpt/

see also: x402, agent-identity

An open standard, maintained by OpenAI and Stripe, that connects buyers, their AI agents and merchants so a purchase can complete inside a conversation.

ACP defines an interaction model for in-conversation checkout — the standard behind ChatGPT's Instant Checkout. Maintained jointly by OpenAI and Stripe as Founding Maintainers under the Apache 2.0 license, it uses date-based (YYYY-MM-DD) versioning, with OpenAI and Stripe providing the first reference implementations.

source https://github.com/agentic-commerce-protocol/agentic-commerce-protocol

see also: agentic-commerce, x402, ap2

The observe-decide-act-observe cycle an agent repeats until its task is complete.

Each turn the agent reads the current state, decides on the next action (often a tool call), takes it, and incorporates the result. The loop ends when the goal is reached, a budget is exhausted, or the agent asks for input.

source https://en.wikipedia.org/wiki/OODA_loop

see also: ai-agent, tool-use

Retrieval-augmented generation in which an agent plans, retrieves, evaluates and re-retrieves iteratively, instead of fetching context once.

Where naive RAG runs a single similarity search and hands the results to the model, agentic RAG turns retrieval into a control loop: the agent decides when and what to retrieve, judges whether the results are sufficient, retries or switches tools (web search, SQL, APIs) and validates before answering. It trades latency and cost for reliability on complex, multi-step questions.

source https://arxiv.org/abs/2501.09136

see also: rag, embeddings, vector-database

A model of search optimization in which autonomous AI agents continuously plan, execute and refine SEO actions across both classic search and AI answer systems.

Agentic SEO shifts optimization from a periodic human task to a real-time agent loop: agents analyze live signals, decide the next action, and apply updates directly. It has two readings — using agents to DO SEO (automation), and optimizing a site so agent-buyers and AI systems can discover, parse and surface it (the agent-readiness reading). The second is the one that bridges to agent-as-buyer thinking.

source https://searchengineland.com/guide/agentic-ai-in-seo

see also: geo, aeo, agent-as-buyer

The web reimagined for AI agents as first-class visitors — machine-readable content, callable tools, agent identity and agent-native payments.

Where the traditional web optimized pages for human eyes and search-engine crawlers, the agentic web adds a parallel layer agents can read and act on: markdown twins, structured data, MCP/WebMCP tools, signed agent identity, and inline payment protocols.

source https://radar.cloudflare.com/

see also: agent-experience, mcp, x402

An open, plain-Markdown file placed in a repository that gives coding agents project-specific build, test, style and security instructions.

Conceived as a 'README for agents', AGENTS.md is plain CommonMark with no required schema; agents scan for conventional headings like '## Build & Test' or '## Code Style'. Agents read the nearest file walking up the directory tree, so monorepo subprojects can ship tailored instructions. Originated in the OpenAI ecosystem and now stewarded by the Agentic AI Foundation (AAIF) under the Linux Foundation; adopted by Codex, Cursor, Zed, Jules, Aider and others.

source https://agents.md/

see also: agents-txt, llms-txt, mcp

A proposed root-level file that declares a site's identity, terms of use, service catalog and agentic endpoints to visiting AI agents.

Where robots.txt answers 'may you look at this?', agents.txt aims to answer 'what can you do here, and on what terms?' — a B2A (business-to-agent) capability manifest. The namespace is contested: several independent projects have used the agents.txt name for different purposes, and one prominent proposal was renamed agent-manifest.txt in March 2026 to disambiguate. It is an emerging, not-yet-standardized convention.

source https://github.com/asturwebs/agents-txt

see also: agents-md, robots-txt, llms-txt

An open-source 'Internet of Agents' infrastructure project — originally from Cisco — providing agent discovery, identity, messaging and observability across vendors.

Open-sourced by Cisco in March 2025 (with LangChain and Galileo) and welcomed by the Linux Foundation on 29 July 2025 with Dell, Google Cloud, Oracle and Red Hat as formative members. AGNTCY supplies cross-framework infrastructure: agent discovery via the Open Agent Schema Framework (OASF), cryptographically verifiable identity, multi-modal messaging and end-to-end observability.

source https://www.linuxfoundation.org/press/linux-foundation-welcomes-the-agntcy-project-to-standardize-open-multi-agent-system-infrastructure-and-break-down-ai-agent-silos

see also: a2a, mcp, agent-identity

A software system that uses a language model to pursue a goal by reasoning, planning and taking actions through tools.

Unlike a single prompt-and-response, an agent runs a loop: it observes, decides on an action (often a tool call), executes it, observes the result, and repeats until the goal is met. Autonomy and tool use are the distinguishing features.

source https://en.wikipedia.org/wiki/Intelligent_agent

see also: tool-use, agentic-loop, mcp

An automated bot that fetches web content for an AI system — to train a model, build a search index, or answer a user's question in real time.

AI crawlers split by purpose (training vs search vs inference) and by behavior (whether they honor robots.txt). Their user-agent strings are spoofable, so genuine ones are confirmed via published IP ranges or reverse DNS — and increasingly via Web Bot Auth signatures.

source https://radar.cloudflare.com/

see also: robots-txt, web-bot-auth, agent-identity

Google Search's AI-generated answer summaries, shown above the classic links, which cite the sources they synthesize.

Launched in the US on 14 May 2024 at Google I/O as the rebrand and general-availability release of Search Generative Experience (SGE, previewed May 2023). AI Overviews synthesize an answer from multiple pages and link the sources, expanding zero-click search and making source citation — the GEO/AEO target — the new prize for content owners.

source https://en.wikipedia.org/wiki/AI_Overviews

see also: geo, aeo, zero-click

Structuring content so AI answer engines and assistants extract, trust and cite it as a direct answer rather than ranking it as a link.

AEO optimizes for extractability, factual density and cross-source consensus — the signals that get content lifted into AI Overviews, voice answers and featured snippets. It overlaps heavily with GEO and LLMO (often treated as the same practice under different names); the distinction is emphasis: AEO leans toward answer-engine and snippet surfaces, GEO toward generative-engine citation.

source https://www.tryprofound.com/resources/articles/what-is-answer-engine-optimization

see also: geo, llmo, ai-overviews

A prompting technique in which a model is elicited to produce intermediate reasoning steps before its final answer, improving accuracy on multi-step problems.

Chain-of-thought (CoT) makes the model 'show its work' — laying out the sub-steps of a calculation or deduction — which raises performance on arithmetic, logic and planning tasks. Modern reasoning models internalize this behavior, spending inference-time compute on a reasoning trace before answering.

source https://en.wikipedia.org/wiki/Prompt_engineering

see also: react-pattern, ai-agent, context-engineering

Splitting a document into smaller passages so each can be embedded, indexed and retrieved independently in a RAG pipeline.

Chunking decides the unit of retrieval: documents are cut into passages (by size, by heading, or semantically) before embedding, so a query returns the few most relevant chunks rather than a whole document. Chunk size is a tradeoff — too large dilutes relevance, too small loses context.

source https://en.wikipedia.org/wiki/Retrieval-augmented_generation

see also: rag, embeddings, reranking

An HTTP mechanism where the same URL returns different representations based on request headers such as Accept.

Standardized in HTTP/1.1 (RFC 9110, HTTP Semantics), it is the clean way to serve HTML to browsers and markdown to agents from one URL, advertised with a Vary: Accept response header so caches behave.

source https://www.rfc-editor.org/rfc/rfc9110.html#name-content-negotiation

see also: markdown-twin

The practice of curating exactly what information enters a model's context window at each step, so the task is solvable.

Popularized in June 2025 when Shopify CEO Tobi Lütke and AI researcher Andrej Karpathy endorsed the term over 'prompt engineering'. Karpathy called it 'the delicate art and science of filling the context window with just the right information for the next step'. In production agents the prompt is a tiny fraction of context; the rest is retrieved documents, tool outputs, history and state — all of which must be engineered.

source https://simonwillison.net/2025/Jun/27/context-engineering/

see also: agent-skills, agentic-rag, prompt-caching

The maximum amount of text (measured in tokens) a language model can consider at once — its prompt plus everything it has generated so far.

The context window is the model's working memory: everything the model can 'see' for a single response — system prompt, retrieved documents, conversation history and tools — must fit inside it. Larger windows let an agent hold more context, but cost and latency rise with how much of the window is used.

source https://en.wikipedia.org/wiki/Large_language_model

see also: token-economics, tokenization, context-engineering

A globally unique identifier that its owner controls directly, without a central registration authority, verifiable by cryptography.

A DID is a URI (did:method:id) that resolves to a DID document containing public keys and service endpoints, letting the controller prove ownership by signing. Because no central registry issues it, a DID gives an agent a portable, self-controlled identity — a building block for agent identity and verifiable credentials.

source https://www.w3.org/TR/did-core/

see also: agent-identity, verifiable-credentials, web-bot-auth

Dense numeric vectors that represent text (or other data) so that semantically similar items sit close together in vector space.

The modern approach was crystallized by Word2vec (Mikolov, Chen, Corrado and Dean at Google, 2013), which learned high-quality dense word vectors that captured meaning by context. Today, embedding models turn documents and queries into vectors so similarity search can find relevant content by meaning rather than keyword — the retrieval engine under RAG and vector databases.

source https://en.wikipedia.org/wiki/Word_embedding

see also: vector-database, rag, agentic-rag

A data model that stores facts as entity-attribute-value triples, so an entity's properties can be listed, typed and queried one attribute at a time.

In the EAV model, each fact is a row: which entity, which attribute, what value. It suits sparse, evolving schemas where entities have many possible attributes but each has only some — exactly the shape of a semantic reference. Model the entity as typed attributes first, and every representation (HTML, markdown, JSON-LD, API) renders from that one record.

source https://en.wikipedia.org/wiki/Entity%E2%80%93attribute%E2%80%93value_model

see also: knowledge-graph, json-ld, agentic-seo

Continuing a pretrained model's training on a smaller, task-specific dataset to specialize its behavior for a particular domain or format.

Fine-tuning adapts an already-trained model by updating its weights on curated examples, so it learns a style, a domain vocabulary or a structured-output format the base model does not reliably produce. It is a form of transfer learning, distinct from retrieval, which adds knowledge at inference time without changing weights.

source https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning)

see also: rlhf, rag, embeddings

Optimizing content to be cited and surfaced by AI answer engines, the way SEO optimized for search rankings.

The term was coined in the 2023 research paper 'GEO: Generative Engine Optimization' by Pranjal Aggarwal, Vishvak Murahari et al. (Princeton University and collaborators), later published at KDD 2024. Because AI engines summarize and cite rather than list ten blue links, GEO favors clear, structured, quotable, well-sourced content. Often discussed alongside AEO (Answer Engine Optimization).

source https://arxiv.org/abs/2311.09735

see also: json-ld, grounding, agentic-web

A robots.txt user-agent token that lets a site opt out of having its content used to train and ground Google's Gemini models, while leaving Google Search indexing unaffected.

Google-Extended is a control token, not a crawler. Adding 'User-agent: Google-Extended' with 'Disallow: /' to robots.txt tells Google not to use the site's content for training or grounding Gemini and Vertex AI generative models; normal Googlebot search crawling continues. Introduced by Google in September 2023.

source https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers

Tying a model's output to verifiable external sources rather than its parametric memory.

A grounded answer can cite where each claim came from. Structured data and retrievable content make grounding easier; AI answer engines increasingly cross-check claims against the live page.

source https://en.wikipedia.org/wiki/Symbol_grounding_problem

see also: rag, json-ld, geo

When a language model generates fluent output that is false or unsupported by its sources, presented as if it were fact.

A hallucination is a confident but ungrounded generation: the model predicts plausible tokens rather than retrieving a verified fact, so the answer can be wholly invented — a citation, a number, an API that does not exist. It is a property of next-token prediction, not a bug, which is why grounding and retrieval are used to constrain it.

source https://en.wikipedia.org/wiki/Hallucination_(artificial_intelligence)

see also: grounding, rag, ai-agent

An IETF standard (RFC 9421) for cryptographically signing components of an HTTP message so a server can verify who sent it and that it was not altered.

Published as a Standards Track RFC in February 2024 (editors A. Backman and J. Richer, with M. Sporny), RFC 9421 defines how to sign chosen parts of a request or response and supports algorithms including EdDSA over Curve25519 (Ed25519). It is the cryptographic foundation Web Bot Auth builds on to prove agent identity, since user-agent strings are spoofable.

source https://www.rfc-editor.org/rfc/rfc9421.html

see also: web-bot-auth, agent-identity, verifiable-credentials

A design in which a human reviews, approves or corrects an agent's actions at defined checkpoints rather than letting it run fully autonomously.

Human-in-the-loop (HITL) inserts a person into the agent's decision cycle — to confirm a high-stakes action (a payment, a delete), label an ambiguous case, or catch an error before it propagates. It trades some autonomy for safety and accountability, and is common where mistakes are costly.

source https://en.wikipedia.org/wiki/Human-in-the-loop

see also: delegation, ai-agent, agent-as-buyer

Showing what an agent action will do before it does it, so a human can approve or cancel.

An AX trust pattern: rather than acting silently, the agent surfaces the planned call and its effect ('this will POST and write one record') for confirmation. Pairs with an action audit log of what actually happened.

source https://biilmann.blog/articles/introducing-ax/

see also: action-audit, agent-experience

A JSON-based format for embedding Schema.org structured data in a page — the lingua franca that AI engines extract.

JSON-LD 1.1 is a W3C Recommendation (2020). A single script tag with an @graph can describe a site, its pages, its author and its key entities in a way that Google, Bing, Perplexity and ChatGPT all parse. By 2026, engines cross-check schema claims against page content, so accuracy matters more than volume.

source https://www.w3.org/TR/json-ld11/

see also: grounding, geo, agentic-web

A structured network of entities and the typed relationships between them, letting machines reason over facts rather than parse prose.

A knowledge graph stores facts as connected entities (nodes) and relationships (edges) — 'x402 is-created-by Coinbase', 'MCP is-a protocol'. Search and answer engines use knowledge graphs to disambiguate entities and answer directly, which is why consistent, structured entity data (JSON-LD, sameAs) helps a site be understood as an entity, not just indexed as pages.

source https://en.wikipedia.org/wiki/Knowledge_graph

see also: json-ld, eav-model, grounding

Optimizing content for how large language models evaluate, trust and select it as a source — a practitioner sibling of GEO and AEO.

LLMO (sometimes AIO, AI Optimization) focuses specifically on large-language-model citation behavior: clear claims, source-able facts, structure a model can parse. It shares roughly 80% of its methods with GEO; the main difference is provenance and scope — GEO came from academia and covers all generative engines, LLMO arose among practitioners and targets LLMs specifically.

source https://ahrefs.com/blog/geo-is-just-seo/

see also: geo, aeo, share-of-model

A markdown file at a domain's root that gives language models a curated index of the site's most important content.

Proposed by Jeremy Howard (Answer.AI) on 3 September 2024. It mirrors robots.txt in spirit but is written for ingestion rather than exclusion: a concise, linkable map that helps agents find and prioritize content. A companion llms-full.txt inlines the full content.

source https://www.answer.ai/posts/2024-09-03-llmstxt.html

see also: markdown-twin, agentic-web

An open, HTTP-native standard co-authored by Stripe and Tempo that lets an agent request, authorize and settle a payment within the same HTTP request.

MPP is an internet-native machine-to-machine payment standard, proposed to the IETF, that lets agents pay for services inline across stablecoins, cards and other methods using Shared Payment Tokens (SPTs). Businesses configure spend limits, merchant-category restrictions and approval workflows in advance, so agents transact only within explicitly granted permissions.

source https://stripe.com/blog/machine-payments-protocol

see also: x402, agentic-commerce, ap2

A clean markdown version of an HTML page, served from the same URL via content negotiation when a client requests text/markdown.

Markdown carries the meaning of a page in roughly a tenth of the tokens of the equivalent HTML. Agent fetchers (Claude Code's WebFetch, Cursor, Cloudflare's edge) request it with an Accept: text/markdown header; humans still get the styled HTML.

source https://datatracker.ietf.org/doc/html/rfc7763

see also: content-negotiation, llms-txt, token-economics

A very small financial transaction — often a fraction of a cent to a few cents — historically impractical because fees exceeded the payment itself.

Micropayments enable pay-per-use at tiny amounts: a few cents for a single API call, an article, or a crawl. Traditional card rails make them uneconomic (fixed fees dwarf the amount), which is why agent-native rails like x402 and pay-per-crawl exist — programmable settlement makes per-request micropayments finally viable.

source https://en.wikipedia.org/wiki/Micropayment

see also: x402, pay-per-crawl, stablecoin

An open standard from Anthropic that connects AI agents to tools and data through a single JSON-RPC interface — the de facto agent-to-tool standard.

MCP standardizes how an agent discovers and calls tools, reads resources and uses prompts, so any compatible agent can talk to any compatible server without bespoke integration. Introduced by Anthropic in November 2024 and now governed under the Linux Foundation via the AAIF (formed December 2025).

source https://en.wikipedia.org/wiki/Model_Context_Protocol

see also: webmcp, tool-use, ai-agent

A system in which several autonomous agents interact — cooperating, delegating or competing — to solve a problem no single agent handles alone.

In a multi-agent system, work is decomposed across specialized agents (a planner, a researcher, a coder) that exchange messages and results. Coordination can be centralized (an orchestrator routes tasks) or peer-to-peer (agents negotiate directly), which is exactly what agent-to-agent protocols standardize.

source https://en.wikipedia.org/wiki/Multi-agent_system

see also: orchestration, a2a, delegation

Microsoft's standard for turning a website into a conversational endpoint that answers natural-language queries — and is itself an MCP server.

Introduced by Microsoft at Build 2025 (May 2025) and led by Schema.org/RSS/RDF creator R.V. Guha, NLWeb combines a site's Schema.org data, a vector index and an LLM to answer questions grounded in the site's own content, exposing the result over a simple endpoint that doubles as an MCP server.

source https://en.wikipedia.org/wiki/NLWeb

see also: mcp, json-ld, agentic-web

An open standard for delegated authorization: it lets an app or agent act on a user's behalf with scoped, revocable access, without sharing the user's password.

OAuth issues an access token that grants a specific scope for a limited time, so a user can authorize an agent to read their calendar or place an order without handing over credentials. Tokens can be scoped and revoked, which makes OAuth the mainstream way to delegate authority to software — and a natural fit for authorizing agents.

source https://en.wikipedia.org/wiki/OAuth

see also: delegation, agent-identity, verifiable-credentials

A Cloudflare marketplace mechanism that lets a site charge AI crawlers per request — returning HTTP 402 to unpaid bots with a price in response headers.

Announced by Cloudflare on 1 July 2025 (private beta). A site sets a price; when an AI crawler requests a page without payment the edge answers HTTP 402 Payment Required with crawler-price headers, and Cloudflare acts as merchant of record to settle. It turns crawling from a free externality into a metered transaction.

source https://blog.cloudflare.com/introducing-pay-per-crawl/

Reusing the model's processed state for a repeated prompt prefix so identical leading context is not recomputed, cutting latency and cost.

Introduced by Anthropic on 14 August 2024, prompt caching marks a content block as a cache breakpoint; a later request that begins with the same exact bytes reads the cached state instead of reprocessing it. Cached input typically costs a fraction of normal input tokens (with a one-time write surcharge). It rewards stable, front-loaded context — a direct incentive to put durable, machine-readable material first.

source https://claude.com/blog/prompt-caching

see also: token-economics, context-engineering, agent-skills

An attack that hides instructions in content an agent reads, hijacking its behavior against its principal's intent.

The term was coined by Simon Willison in September 2022, framed as the LLM analogue of SQL injection. Because agents act on the text they ingest, malicious or invisible instructions on a page ('ignore previous instructions and...') can manipulate them. Hidden agent-only text is therefore an anti-pattern indistinguishable from an attack; trustworthy sites keep their machine layer transparent.

source https://simonwillison.net/2022/Sep/12/prompt-injection/

see also: agent-experience, web-bot-auth

An agent pattern that interleaves reasoning and acting: the model thinks, takes a tool action, observes the result, and reasons again — looping to a goal.

ReAct (Reason + Act) combines chain-of-thought with tool use so an agent alternates 'thought' and 'action' steps: it reasons about what to do, calls a tool, reads the observation, and updates its plan. It is the canonical loop most tool-using agents implement.

source https://arxiv.org/abs/2210.03629

see also: agentic-loop, tool-use, chain-of-thought

A second retrieval stage that re-scores an initial set of candidate passages with a more precise model, promoting the most relevant to the top.

Reranking improves retrieval precision: a fast first stage (vector or keyword search) returns many candidates, then a slower, more accurate cross-encoder re-scores each against the query and reorders them. The agent then reads only the top few, so quality rises without a larger context budget.

source https://en.wikipedia.org/wiki/Learning_to_rank

see also: rag, semantic-search, chunking

Fetching relevant documents at query time and giving them to the model as context, so answers are grounded in current, specific data.

The term was coined by Patrick Lewis and colleagues at Facebook AI Research (now Meta AI), University College London and NYU in a 2020 NeurIPS paper. RAG reduces hallucination and lets a model answer about information it was never trained on. Agent-friendly sites help RAG by exposing clean, chunkable content (markdown twins, llms.txt) and structured data.

source https://arxiv.org/abs/2005.11401

see also: grounding, markdown-twin

Reinforcement Learning from Human Feedback — training a model to prefer responses humans rate higher, aligning it with human intent.

RLHF fine-tunes a model using human preference data: people rank model outputs, a reward model learns those preferences, and reinforcement learning nudges the model toward higher-rated behavior. It is a core step in turning a raw pretrained model into a helpful, instruction-following assistant.

source https://en.wikipedia.org/wiki/Reinforcement_learning_from_human_feedback

see also: fine-tuning, grounding, transformer

The root-level file that tells crawlers — including AI crawlers — what they may and may not fetch.

The web's oldest crawler contract, originally defined by Martijn Koster in 1994 and standardized as RFC 9309 in 2022. In the agentic era it is where sites name AI crawlers explicitly (GPTBot, ClaudeBot, Google-Extended), and where RSL licensing terms are referenced via a License directive.

source https://www.rfc-editor.org/rfc/rfc9309.html

see also: ai-crawler, rsl, llms-txt

An open standard for machine-readable content-licensing terms, referenced from robots.txt, that tells AI systems how content may be used and at what price.

Really Simple Licensing (RSL) launched on 10 September 2025, backed by Reddit, Yahoo, Medium and People Inc. among others. It defines license models — free, attribution, subscription, pay-per-crawl, pay-per-inference — in an XML file referenced from robots.txt, so AI crawlers can read the terms before using content.

source https://rslstandard.org/press/rsl-standard

Granting an agent a limited, explicit set of permissions to act on a principal's behalf — bounded in scope, budget and time.

Delegation answers the second half of agent identity: not just 'which agent?' but 'authorized to do what, for whom, within what limits?'. Scoped delegation expresses bounded authority — e.g. spend up to a cap, only with approved merchants, for a fixed window — and underpins agent payment mandates (AP2) and permissioned payment rails (MPP). It is the principle that keeps an autonomous agent from exceeding what its principal allowed.

source https://datatracker.ietf.org/doc/html/rfc6749

see also: agent-identity, ap2, verifiable-credentials

Search that matches on meaning rather than exact keywords, using embeddings to find passages whose sense is close to the query.

Semantic search embeds both the query and the corpus into a vector space and retrieves by similarity, so 'how do I make my site readable to bots' can match a page titled 'agent-readiness' with no shared keywords. It is the retrieval backbone of RAG and the reason meaning-rich, well-structured content is found even without literal keyword overlap.

source https://en.wikipedia.org/wiki/Semantic_search

see also: embeddings, vector-database, rag

A metric for how often a brand appears in AI-generated answers across prompts, relative to competitors — the AI-era analogue of share of voice.

Share of model (closely related to 'AI share of voice') measures a brand's slice of the AI conversation: if models mention brands 100 times across tracked prompts and a brand accounts for 25, its share is 25%. It is the headline output of GEO/AEO measurement tools (Profound, Ahrefs Brand Radar, Semrush) and reframes visibility from rankings to presence inside model answers.

source https://www.tryprofound.com/resources/articles/what-is-answer-engine-optimization

see also: geo, aeo, zero-click

A cryptocurrency designed to hold a stable value, typically pegged to a fiat currency such as the US dollar.

A stablecoin (e.g. USDC) aims to keep a 1:1 peg to a reference asset, usually backed by reserves, so it can be transferred on-chain without the volatility of other crypto. For agent payments, stablecoins offer fast, programmable, low-friction settlement — which is why several agent-payment protocols settle in them.

source https://en.wikipedia.org/wiki/Stablecoin

see also: agentic-commerce, x402, micropayment

The MCP transport, introduced in spec version 2025-03-26, that carries client-server messages over a single HTTP endpoint and supersedes the older HTTP+SSE transport.

Streamable HTTP replaces MCP's original 2024-11-05 HTTP+SSE transport with a single-endpoint design: the server handles POST and GET requests and may optionally use Server-Sent Events to stream multiple messages, but can also run fully statelessly behind a load balancer. The TypeScript SDK v1.10.0 (17 April 2025) was the first to support it.

source https://modelcontextprotocol.io/specification/2025-03-26/basic/transports

see also: mcp, content-negotiation

The cost structure of agent interactions, where every token of input and output is billed — making concise, structured content a direct cost saving.

Because agents pay per token, a markdown twin that is ~90% smaller than its HTML equivalent is not just faster but cheaper to consume. Agent-friendly design is partly an economic argument.

source https://en.wikipedia.org/wiki/Large_language_model

see also: markdown-twin, content-negotiation

Breaking text into tokens — the sub-word units a language model actually reads and generates — before it can be processed.

Before a model sees text, a tokenizer splits it into tokens (whole words, word-pieces or characters) and maps each to an integer. Token counts, not character counts, drive context limits and pricing, so how efficiently your content tokenizes affects both what fits and what it costs an agent to read.

source https://en.wikipedia.org/wiki/Large_language_model

see also: context-window, token-economics, embeddings

An LLM's ability to call external functions — search, code execution, APIs — by emitting a structured request the host executes.

The model does not run the tool itself; it outputs a tool call, the host runs it and returns the result, and the model continues. Tool use (also called function calling) is what turns a chat model into an agent.

source https://platform.openai.com/docs/guides/function-calling

see also: ai-agent, mcp, agentic-loop

The neural-network architecture, built on self-attention, that underlies virtually all modern large language models.

The transformer processes a sequence in parallel using self-attention, letting every token weigh its relationship to every other token. Introduced in 2017, it replaced recurrent architectures and made today's large language models — and the agents built on them — possible.

source https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)

see also: context-window, fine-tuning, embeddings

An open commerce standard that orchestrates A2A, AP2 and payment rails into one end-to-end agentic-commerce journey.

UCP defines a common language and functional primitives so consumer surfaces, businesses and payment providers can transact through agents. Attributed by secondary reports to Google and retail partners including Shopify, Etsy, Wayfair, Target and Walmart (not confirmed at the primary spec source), it composes existing protocols — A2A for agent communication and AP2 for payment mandates — rather than replacing them.

source https://developers.googleblog.com/under-the-hood-universal-commerce-protocol-ucp/

see also: agentic-commerce, ap2, a2a

A database built to store embeddings and retrieve the nearest ones to a query vector using approximate nearest-neighbor search.

A vector database (or vector store) indexes high-dimensional embeddings and finds the most similar ones with Approximate Nearest Neighbor (ANN) algorithms — commonly HNSW graphs or quantization — under metrics like cosine distance. It is the storage-and-retrieval backbone of RAG: documents go in as vectors, a query vector comes in, and the closest chunks come out as context.

source https://en.wikipedia.org/wiki/Vector_database

see also: embeddings, rag, agentic-rag

A W3C standard for tamper-evident, cryptographically verifiable digital credentials that prove a claim about a subject without contacting the issuer.

The Verifiable Credentials Data Model 2.0 became a W3C Recommendation on 15 May 2025. A VC binds claims (e.g. 'this agent is operated by X' or 'this principal authorized this scope') to an issuer's signature, so a verifier can check authenticity and integrity offline. In the agentic web they are a candidate mechanism for portable agent and delegation identity.

source https://www.w3.org/TR/vc-data-model-2.0/

see also: agent-identity, http-message-signatures, delegation

Cryptographically verifying which agent is making a request using HTTP Message Signatures (RFC 9421), since user-agent strings are spoofable.

An agent signs its requests with an Ed25519 key tied to a published identity (a JWKS directory at /.well-known/http-message-signatures-directory, advertised via the Signature-Agent header); the server verifies the signature per RFC 9421. This lets sites distinguish a genuine ClaudeBot or GPTBot from an impostor, and is the foundation for agent-aware rate limits and paid access.

source https://blog.cloudflare.com/web-bot-auth/

see also: agent-identity, prompt-injection, x402

A W3C-draft standard that lets a web page expose callable tools to a visiting agent via the navigator.modelContext browser API, bringing MCP into the browser.

Instead of an agent screenshotting a page and guessing where to click, a WebMCP-enabled page declares its capabilities as typed tools the agent can invoke directly through navigator.modelContext. Published as a W3C Draft Community Group Report on 10 February 2026 (developed in the Web Machine Learning Community Group) and available as an early preview in Chrome 146.

source https://www.w3.org/community/webml/

see also: mcp, accessibility-tree

A protocol that uses the HTTP 402 'Payment Required' status so an agent can pay for a resource inline with a stablecoin micropayment.

Launched by Coinbase in May 2025. The server answers an unpaid request with 402 and machine-readable payment terms (amount, asset, network, recipient); the agent pays in a stablecoin such as USDC and retries with cryptographic proof of payment. On 23 September 2025 Coinbase and Cloudflare announced the x402 Foundation to steward the standard.

source https://www.coinbase.com/developer-platform/discover/launches/x402

see also: agentic-commerce, agent-identity

A machine-readable file listing a site's URLs so crawlers and agents can discover every page without following links.

An XML sitemap enumerates a site's canonical URLs (optionally with last-modified dates and priorities) at /sitemap.xml, giving crawlers a complete, authoritative index instead of relying on link discovery. For agents, it is the fast path to the full surface area of a site — a discovery affordance alongside llms.txt and robots.txt.

source https://www.sitemaps.org/protocol.html

see also: llms-txt, robots-txt, ai-crawler

A search where the user gets their answer on the results page itself — via a snippet, knowledge panel or AI Overview — without clicking through to any site.

Popularized by Rand Fishkin of SparkToro, whose 2019 research found that over half of Google searches ended without a click to an external property. AI Overviews and answer engines accelerate the trend, making 'being cited in the answer' more valuable than 'ranking for the click' — the economic premise behind GEO and AEO.

source https://sparktoro.com/blog/less-than-half-of-google-searches-now-result-in-a-click/

see also: ai-overviews, geo, aeo