Self-Attention Explained: Queries, Keys, and Values

On Day 12, attention worked. The word it landed on trophy in one sentence and on suitcase in the other. But we cheated: we picked the query vectors by hand. One pointed toward "heavy", the other toward "roomy". A real model doesn't get that gift.

Today we remove the cheat. Every word gets three vectors instead of one, each one computed from the embedding by a small learned matrix. Those three vectors are called query, key and value, and the scoring mechanism from Day 12 sits on top of them.

Why One Vector Is Not Enough

On Day 10 we saw that the embedding table gives each token one fixed row. On Day 12 we used those rows to score attention.

But look at what a single word actually has to do during attention:

• It has to ask: what kind of word am I looking for?
• It has to advertise: here is what I can offer to other words looking for me.
• It has to carry content: if someone picks me, this is the information I send.

Those are three different jobs. And the ask is not the same shape as the content. "It" asks "which object am I?", but the content it sends, once attention resolves it, is the trophy's weight and size. One flat vector sitting in the middle of the space can't point in a search direction and hold an answer at the same time.

import numpy as np

#                    heavy  roomy  object
embedding_it = np.array([0.50, 0.50, 0.50])

# on Day 12 we needed a query like this to find the trophy:
query_we_wanted = np.array([0.95, 0.05, 0.10])

# but the embedding looks like this:
print("embedding:", embedding_it)
print("query we needed:", query_we_wanted)
print("same thing?", np.allclose(embedding_it, query_we_wanted))

embedding: [0.5 0.5 0.5]
query we needed: [0.95 0.05 0.1 ]
same thing? False

That False is the whole problem. We need a way to reshape the embedding into something directional for searching, something different for advertising, and something different again for carrying content.

The embedding sits flat. The query has to point somewhere. They can't be the same thing.

What a Weight Matrix Does

The basic idea

A weight matrix is a small grid of learned numbers. When we multiply the embedding by it, we get a new vector out the other side. The grid decides which dimensions get amplified, which get suppressed, and which get mixed together.

# a weight matrix that amplifies the "heavy" dimension
W_query = np.array([
    [ 1.80, -1.60,  0.20],
    [-1.60,  1.80,  0.20],
    [ 0.10,  0.10,  0.10],
])

q_it = embedding_it @ W_query
print("embedding:", embedding_it)
print("after W_query:", q_it.round(2))

embedding: [0.5 0.5 0.5]
after W_query: [0.15 0.15 0.25]

The flat [0.50, 0.50, 0.50] became [0.15, 0.15, 0.25]. That's a small shift for "it", because "it" genuinely doesn't know what it's looking for yet. But watch what happens when trophy goes through the same matrix.

embedding_trophy = np.array([0.90, 0.10, 0.80])

q_trophy = embedding_trophy @ W_query
print("trophy embedding:", embedding_trophy)
print("trophy's query:  ", q_trophy.round(2))

trophy embedding: [0.9 0.1 0.8]
trophy's query:   [ 1.54 -1.18  0.28]

Trophy's query shoots strongly positive on the first dimension and strongly negative on the second. It's now a sharp, directional vector. The matrix took the raw embedding and reshaped it into a search direction: "find me something heavy, not roomy."

Three Matrices, Three Roles

The fix for our one-vector problem is simple: use three separate matrices. Each one reshapes the embedding for a different job.

W_key = np.array([
    [ 1.00,  0.00,  0.50],
    [ 0.00,  1.00,  0.50],
    [ 0.20,  0.20,  0.80],
])

W_value = np.array([
    [ 1.20,  0.00,  0.30],
    [ 0.00,  1.20,  0.30],
    [ 0.10,  0.10,  0.90],
])

Same input, three different matrices. Each one pulls out what its role needs.

Now every word gets projected three times.

embedding_suitcase = np.array([0.20, 0.90, 0.80])

words = {
    "trophy":   embedding_trophy,
    "suitcase": embedding_suitcase,
    "it":       embedding_it,
}

for name, emb in words.items():
    q = (emb @ W_query).round(2)
    k = (emb @ W_key).round(2)
    v = (emb @ W_value).round(2)
    print(f"{name:10s}  Q={q}  K={k}  V={v}")

trophy      Q=[ 1.54 -1.18  0.28]  K=[1.06 0.26 1.14]  V=[1.16 0.2  1.02]
suitcase    Q=[-1.    1.38  0.3 ]  K=[0.36 1.06 1.19]  V=[0.32 1.16 1.05]
it          Q=[0.15 0.15 0.25]  K=[0.6 0.6 0.9]  V=[0.65 0.65 0.75]

Read the queries column. Trophy's Q points heavily toward the first dimension (heavy). Suitcase's Q points toward the second (roomy). These are the questions each word asks. Now look at the values column. The shapes are softer, rounder, carrying actual content rather than pointing in a search direction.

That difference is the whole reason for having three matrices. The question a word asks and the answer it holds need to be different shapes, and now they are.

Same three matrices, every word. Different embeddings produce different Q, K, V triplets.

How Q, K and V Plug Into Attention

Day 12 showed the mechanism: dot product to score, softmax to weight, weighted sum to blend. That hasn't changed. What changes is what goes into each step.

• Scores come from dot-producting one word's Q against every word's K.
• Weights come from softmax on those scores (Day 12 already showed this).
• The output is a weighted sum of the V vectors.

On Day 12, we used raw embeddings for all three roles. Now each role gets its own projection.

def softmax(x):
    e = np.exp(x - x.max())
    return e / e.sum()

# what "it" is looking for (its query)
q = embedding_it @ W_query

# what each word advertises (their keys)
k_trophy   = embedding_trophy   @ W_key
k_suitcase = embedding_suitcase @ W_key

# score = Q dot K
scores = np.array([q @ k_trophy, q @ k_suitcase])
weights = softmax(scores * 5)

# what each word sends if chosen (their values)
v_trophy   = embedding_trophy   @ W_value
v_suitcase = embedding_suitcase @ W_value

# output = weighted sum of values
output = weights[0] * v_trophy + weights[1] * v_suitcase

print("scores :", scores.round(3))
print("weights:", weights.round(3))
print("output :", output.round(2))

scores : [0.483 0.51 ]
weights: [0.466 0.534]
output : [0.71 0.71 1.04]

The weights are nearly even. That's correct, not a bug. "It" on its own, without context from "big" or "small", genuinely doesn't know which candidate to choose. In a real transformer, earlier layers would have already mixed context from the surrounding words into the embedding, sharpening the query before it ever reaches this step.

Q asks the question. K answers it. The dot product measures the match.

Scores pick using Q and K. The output is built from V. Each vector has its own job.

The Ask Is Not the Offer

This is the point worth sitting with. Look at what trophy produces as its query versus what it produces as its value.

print("trophy asks  (Q):", (embedding_trophy @ W_query).round(2))
print("trophy offers (V):", (embedding_trophy @ W_value).round(2))

trophy asks  (Q): [ 1.54 -1.18  0.28]
trophy offers (V): [1.16 0.2  1.02]

The query is sharp: strongly positive on one axis, strongly negative on another. It's a search beam. The value is softer: all positive, spread across the dimensions, carrying the actual properties of the trophy.

If we forced one vector to do both, we would get a compromise that searches badly and carries content badly. The three matrices let each word be a good searcher (Q), a good advertisement (K), and a good answer (V) all at the same time.

The query says what the word needs. The value says what the word has. Deliberately different shapes.

The Matrices Are Learned, Not Designed

Nobody sat down and decided that W_query should amplify the "heavy" dimension for trophy. These three matrices start as small random numbers, identical to the embedding table on Day 10.

rng = np.random.default_rng(42)
W_random = rng.standard_normal((3, 3)) * 0.1
print("W_query at the start of training:")
print(W_random.round(3))

W_query at the start of training:
[[ 0.03  -0.104  0.075]
 [ 0.094 -0.195 -0.13 ]
 [ 0.013 -0.032 -0.002]]

No pattern. During training, the model predicts the next word, gets it wrong, and the error adjusts these numbers. After billions of rounds, the matrices end up arranged so that queries point toward the right keys and values carry the right information. The arrangement was never programmed. It was found.

Random at the start. Shaped by predicting text billions of times.

The Full Self-Attention Step

Here it is in one function. This is the complete step that every position in a transformer runs.

def self_attention(query_emb, all_embeddings, W_q, W_k, W_v, scale=5):
    q = query_emb @ W_q
    keys   = {name: emb @ W_k for name, emb in all_embeddings.items()}
    values = {name: emb @ W_v for name, emb in all_embeddings.items()}

    names  = list(all_embeddings.keys())
    scores = np.array([q @ keys[n] for n in names])
    weights = softmax(scores * scale)

    output = sum(w * values[n] for w, n in zip(weights, names))
    return output, {n: round(float(w), 3) for n, w in zip(names, weights)}

output, w = self_attention(embedding_it, words, W_query, W_key, W_value)
print("output for 'it':", output.round(2))
print("weights        :", w)

output for 'it': [0.7  0.7  0.97]
weights        : {'trophy': 0.354, 'suitcase': 0.406, 'it': 0.24}

Eight lines. Project to get Q, K, V. Dot Q against every K. Softmax. Weighted sum of V. That's self-attention.

Here, we can see that "it" also scores against its own key and takes part of the weight. In self-attention, every word looks at every position in the sentence, including its own. That is why the output differs slightly from the two-candidate version above.

Every word in the sentence runs this same step, with its own Q, and gets its own output. In parallel, not one after another.

Embedding in, three matrices, one output. Every word runs this in parallel.

What Comes Next

Two things were left unexplained today on purpose.

First, the * 5 scaling factor. On Day 12 we used it to sharpen the weights so they're easy to read. Real transformers divide the scores by the square root of the vector dimension instead. That's the attention formula, and it's the subject of Day 14.

Second, we used one set of Q, K, V matrices. Real transformers use several sets running in parallel, each one looking for a different pattern. That's multi-head attention, coming on Day 15.

One set of matrices sees one pattern. Multiple sets see several at once.

Recap

A single embedding can't ask a question and hold an answer at the same time, because the two jobs need different shapes. Three learned weight matrices fix this by projecting every embedding into three separate vectors: query (what the word is searching for), key (what the word advertises to others), and value (what the word sends if chosen).

The attention mechanism from Day 12 sits on top of these projections. Scores come from Q dot K. Weights come from softmax. The output is a weighted sum of V. The matrices start random and are shaped by training until queries point at the right keys and values carry the right content.

In the next article, we look at the attention formula itself: what the square root of d actually does, why it matters, and what happens without it.