Almost every AI system you've used in the last few years — ChatGPT, Claude, Gemini, the autocomplete in your inbox — traces back to a single 2017 paper from Google researchers: *Attention Is All You Need*. It's short, dense and mathematical. This is what it actually says, without the equations.
(We're summarising and explaining the paper in our own words — the original is linked throughout so you can read the source.)
The problem it solved
Before 2017, the best language models read text one word at a time, in order, like a person reading left to right. That made them slow to train and bad at connecting words that sit far apart in a sentence — by the time the model reached the end, it had half-forgotten the start.
The idea: "attention"
The paper's core insight — attention — lets the model look at every word at once and decide which other words matter for understanding each one. In the sentence "the trophy didn't fit in the suitcase because it was too big," attention is what lets the model figure out that "it" means the trophy, not the suitcase.
The authors' bold claim was in the title: you don't need the old sequential machinery at all. Attention alone is enough. That architecture is the "Transformer."
Why it changed everything
- It's parallel. Because the model reads all words simultaneously, training can use modern hardware fully — which is what made today's enormous models economically possible.
- It scales. The same design works whether the model is tiny or has hundreds of billions of parameters. Almost every major model since is a Transformer variant.
- It generalised. The same idea now powers image, audio, video and code models — not just text.
One paper replaced a decade of specialised architectures with a single, scalable idea. That's why it's the most-cited AI paper of its generation.
Why a business leader should care
You don't need the math, but the takeaway is strategic: the entire AI wave runs on one general-purpose architecture that gets better mostly by adding scale and data. That's why capabilities have moved so fast — and why inference costs have collapsed as the industry optimised the same design over and over.
Sources
- Vaswani et al. (2017) — *Attention Is All You Need*
- Google Research — Transformer announcement