Transformers and attention.
Every modern language model runs on one idea: let every word look at every other word at once, and decide which ones matter. That idea is attention. The design around it is the transformer.
This is an educational walk through the field, in plain language. What attention is, why reading long text used to be hard, how a transformer reads everything at once, what that unlocked, and where the design is heading. The papers behind each idea are linked, so you can read the source yourself.
Attention is the model deciding which other words matter.
A transformer is a neural network for sequences, built around one mechanism: attention. Everything else is in service of it.
When you read "the rep sent the contract because it was overdue," you decide that "it" points at the contract, not the rep. You look back across the sentence and weigh which earlier words matter for the one in front of you. That single habit is the idea at the center of the transformer.
Stated one layer down: for each position it is processing, the model weighs every other position and decides how much each one should influence the current one. To represent the word "it," it pulls mostly from the word "it" refers to. To represent "bank," it pulls from "river" or "money" nearby. Meaning comes from context, and attention is how context gets mixed in.
The break from older designs is that a transformer does not read strictly in order and carry a memory forward. Every position can attend to every other position at the same time. That is what makes it fast to train and good at links across long distances.
“The rep sent the contract because it was overdue.”
The hard part is that meaning lives far apart, and reading in order is slow.
Before attention took over, two problems shaped every sequence model: distant words that depend on each other, and a reading order that a parallel processor cannot speed up.
Meaning depends on far-apart words.
The subject of a sentence can sit thirty words from its verb. The answer to what a pronoun refers to can be a paragraph back. A reader has to connect distant pieces, not just neighbors.
Older models faded over distance.
Recurrent networks read one token at a time and compressed the past into a single running state. Information from early in a long passage had to survive many steps to still matter at the end, and often it did not.
Reading strictly in order will not parallelize.
If step ten cannot start until step nine finishes, a modern parallel processor sits idle. Training on internet-scale text one token at a time is painfully slow.
Attention is not free.
Letting every word look at every other word means the work grows with the square of the length. A passage twice as long costs roughly four times as much. This cost is the central tax of the design and shapes everything downstream.
Looking at everything at once is order-blind.
Because attention looks at all positions together, the raw mechanism does not know which word came first. Word order, which carries real meaning, has to be supplied deliberately.
Inside a transformer.
Weigh the words with queries, keys, and values. Run several views at once. Add order back. Stack the block. Then choose how to read: in both directions to understand, or left to right to generate.
Recurrent computation
w1 → state → w2
→ state → w3Attention computation
Compare allowed positions in parallel during training.
Older models read in order and carried a fading memory. A transformer lets every word look at every other word in parallel.
1Self-attention
Queries, keys, and values.
For each word the model makes three vectors. A query, what am I looking for. A key, what do I offer. A value, what I will pass along. Each word's query is compared against every word's key to get attention weights, and those weights blend the values.
The result is that a word's new representation is a weighted mix of the whole sentence, leaning on the words most relevant to it. This weighted mixing by query-key-value scores is the core idea of the 2017 transformer paper (Vaswani et al.). It is not the model "focusing like a person." It is arithmetic over vectors.
- Query
The current position asks for relevant context.
- Keys
Compare the query against representations of allowed positions.
- Weights
Normalize the compatibility scores.
- Values
Combine the corresponding value vectors using those weights.
2Multi-head attention and word order
Several views at once, with position added back.
One set of attention weights captures one kind of relationship. Transformers run several attention heads in parallel, each free to focus on a different pattern. One head tracks pronoun references, another subject-verb links, another nearby phrasing. Their outputs are combined into more angles on the same sentence.
Because attention is order-blind, each position also gets a signal added that encodes where it sits, called positional encoding. That is why "rep sent contract" and "contract sent rep" are not treated as the same bag of words.
- Head 1
- One learned projection of queries, keys, and values.
- Head 2
- Another learned projection over the same input.
- Head 3
- A further learned projection.
Mix them through the output projection.
3The block, stacked
Attention, a small network, repeat.
A transformer layer is attention followed by a small per-word feed-forward network, each wrapped with a residual connection, which adds the input back, and normalization for stable training. Stack many such blocks and the representations get progressively richer, from surface word to phrase to sentence meaning.
The original design had an encoder that reads the input and a decoder that writes the output, built for translation. Later models specialize: encoder-only for understanding, decoder-only for generation. Same block, arranged for the job.
- Attention sublayer
Mix information across allowed token positions.
- Residual and normalization
Combine and stabilize the representations.
- Feed-forward sublayer
Transform each position’s representation.
- Next block
Repeat with the updated representations.
4Two pretraining recipes
Read both directions to understand, or left to right to generate.
Encoder-style models like BERT (Devlin et al., 2018) read a passage in both directions and learn by filling in masked-out words. That suits understanding: classification, pulling structured facts out of text, answering from a passage.
Decoder-style models read left to right and learn by predicting the next word, which is what powers fluent generation. Same attention mechanism, different reading direction and training objective. In an enterprise setting these are the same machines behind reading a sales call, summarizing it, drafting a follow-up, or answering a question from a set of notes. (Stated as where the field's tools show up, not as a claim about any one product.)
Bidirectional
Causal
5Why it scales, and what it costs
Parallel training, with a quadratic bill to manage.
Because all positions process in parallel, transformers train efficiently on very large text corpora, and quality tends to keep improving with more data, more parameters, and more compute. That scalability, more than any single trick, is why the design took over.
The catch is the quadratic cost. One line of work changes the math with sparse, low-rank, or linear-attention variants that approximate full attention more cheaply, surveyed in Tay et al. (2020). A different line keeps the math exact but rewrites the computation to fit hardware better: FlashAttention (Dao et al., 2022) reorders the work to avoid slow memory traffic on the accelerator, making exact attention faster and able to handle longer sequences without changing the result.
4 tokens
4 × 4 = 16 pairs.
8 tokens
8 × 8 = 64 pairs.
What one architecture unlocked.
At the field level: reading and understanding text at scale, fluent generation, adapting a general model to a specific domain, the same idea applied beyond text, and grounding answers in retrieved material.
The enterprise lines below are examples of where the general technique shows up, framed as how the field does it. They are not claims about any one product, and none of them describe attention itself. Data isolation and per-person permissions are organizational practice built around a model, not properties of the mechanism.
Reading text at scale.
One pretrained model, adapted with a light touch, handles classifying, extracting fields, and answering from a passage. In a sales setting that is the kind of machinery behind reading a call transcript and pulling out who, what, and the next step.
Fluent generation.
Next-word prediction at scale produces summaries, drafts, and replies. Drafting a follow-up from a meeting, or rephrasing notes into a clean record, sits here.
Adapting to a way of working.
A model pretrained on broad text can be fine-tuned or steered toward one organization's vocabulary and patterns, for example a company's particular way of qualifying a deal, without retraining from scratch. This is the general technique, not a description of any internals.
Grounding and retrieval.
Transformers both produce the contextual vectors used to search a knowledge base and consume retrieved passages to answer from them. As a design pattern, that is also the basis for answering only from material a given person is allowed to see, and for keeping one client's data isolated from another. See Retrieval-augmented generation.
On the product itself we keep it simple: Prism is our own model, built for sales conversations. Everything else here describes the field, not our plumbing.
The direction the field is heading.
We describe the direction, not a roadmap of our own. The honest open problems sit alongside the progress.
From paragraphs to whole histories.
Pushing the window from a few paragraphs to whole books or long call histories, which runs straight into the quadratic cost and drives the efficient-attention work above.
Trade a little exactness for length.
A steady stream of attention variants, sparse, low-rank, linear, and kernel-based, make sequences affordable that full attention could not, surveyed as a family rather than a winner.
Match the memory hierarchy.
Rather than approximate, make the exact computation fit the memory of real accelerators, the FlashAttention line, since the bottleneck is often moving data, not doing math.
Reading in order, revisited.
Newer state-space and recurrent-style designs revisit reading in order to get linear cost on very long inputs. A neutral mention: the field testing whether attention is always the answer.
What it still does not solve.
Cost still grows with length. The model predicts likely text and has no built-in guarantee of truth, which is why grounding matters. And what each attention head actually does is only partly understood.
The papers behind this page.
The last entry is an explainer studied for structure, not cited as a research result.
- Vaswani et al. (2017). Attention Is All You Need. arXiv:1706.03762.
- Devlin et al. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805.
- Dao et al. (2022). FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. arXiv:2205.14135.
- Tay et al. (2020). Efficient Transformers: A Survey. arXiv:2009.06732.
- Alammar, J. The Illustrated Transformer. A visual walkthrough, studied here for plain-then-precise structure, not cited as a primary result.
Keep reading.
Sibling explainers in the Research section.
Talk. Done.
The field is deep, but the promise is plain: you talk, and clean, complete notes are waiting. See it on a real call.
No app required. Just your phone number.