Transformer

The neural-network architecture, built on self-attention, that underlies virtually all modern large language models.

The transformer is the engine under every agent on the agentic web; its self-attention is why context — and how cheaply your content fits in it — matters so much.

term
Transformer
category
knowledge-memory
short_def
The neural-network architecture, built on self-attention, that underlies virtually all modern large language models.
long_def
The transformer processes a sequence in parallel using self-attention, letting every token weigh its relationship to every other token. Introduced in 2017, it replaced recurrent architectures and made today's large language models — and the agents built on them — possible.
see_also
context-window fine-tuning embeddings
etymology_origin
— verify-against-primary-at-build ↗ https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture) — introduced in 'Attention Is All You Need', Vaswani et al., Google, 2017
related_to
context-window fine-tuning embeddings tokenization
contrast_with
Unlike earlier recurrent networks that read tokens one at a time, the transformer attends to the whole sequence in parallel, which is what enabled scaling to large context windows.
example
Every frontier model in the Model Matrix — Claude, GPT, Gemini — is a transformer at its core.
source
https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)
status
active
why_it_matters
The transformer is the engine under every agent on the agentic web; its self-attention is why context — and how cheaply your content fits in it — matters so much.
sameAs
https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)
bridge_entity
context-window
last_verified
2026-07-06
md_twin
/glossary/transformer.md

last verified · by

← all The Agentic Web Lexicon · .md · JSON