Transformers
Master the Transformer architecture end to end: tokenization, embeddings, the attention family, encoder and decoder families, hands-on fine-tuning, and fast inference with scaling laws.
Transformers Tutorials (13)
Browse, search, and work through all available articles for this category.
Self-Attention Explained: Queries, Keys, and Values
Every word in a transformer asks a question, advertises what it knows, and holds an answer. Those are query, key and value, and this is how they work.
Read Tutorial
Why Transformers Need Attention: The Trophy Problem
One sentence breaks static embeddings: the trophy did not fit in the suitcase because it was too big. Here is why attention was invented to fix it.
Read Tutorial
The Geometry of Meaning: Similarity and Distance
Embeddings are points in space. How cosine similarity measures meaning, why direction beats distance, and what king minus man plus woman really proves.
Read Tutorial
What Are Embeddings? From Token IDs to Meaning
How a token ID becomes a vector: the embedding table, why one-hot encoding fails, what the dimensions mean, and how embeddings learn meaning during training.
Read Tutorial
Why LLMs Can't Count the R's in Strawberry: Tokenization Quirks
The famous strawberry problem explained: why models that write essays fail at counting letters, plus the arithmetic, spelling, and rhyming quirks tokenization causes.
Read Tutorial
Vocabulary Size vs Context Window: The Core Tokenization Trade-Off
Why a bigger vocabulary makes sequences shorter but the model heavier, how tokens fill a context window, and how the two pull against each other.
Read Tutorial![Special Tokens Explained: [UNK], [PAD], [CLS], [SEP], BOS, and EOS](/_next/image?url=%2Fimages%2Fthumbs%2Fspecial-tokens.jpg&w=3840&q=75)
Special Tokens Explained: [UNK], [PAD], [CLS], [SEP], BOS, and EOS
What the square-bracket tokens mean: [UNK], [PAD], [CLS], [SEP], [MASK], and the BOS and EOS markers, and why a model cannot work without them.
Read Tutorial
BPE vs WordPiece vs SentencePiece: Which One Does Your Model Use
A side-by-side comparison of BPE, WordPiece, and SentencePiece: how each picks its merges, how to spot them in the tokens, and which models use which.
Read Tutorial
SentencePiece: Tokenization for Multilingual Text and Code
How SentencePiece tokenizes raw text with no spaces to split on, the meaning of the underscore marker, the Unigram model, and why T5 and Llama use it.
Read Tutorial
WordPiece: How BERT Splits Words
How WordPiece tokenizes words for BERT, the ## continuation marker, greedy longest-match encoding, and how it differs from BPE by merging on likelihood.
Read Tutorial
Byte-Pair Encoding (BPE) Explained Step by Step
How Byte-Pair Encoding builds a subword vocabulary by merging the most frequent character pairs, how it encodes new words, and why GPT uses byte-level BPE.
Read Tutorial
The Four Ways to Split Text: Word, Character, Subword, and Byte-Level
The four ways to tokenize text, word, character, subword, and byte-level, and the trade-offs in vocabulary size, sequence length, and out-of-vocabulary words.
Read Tutorial
Transformer Architecture and Tokenization Explained
Foundational module on Transformer mechanics and tokenization: embeddings, the attention family, encoder/decoder architectures, and subword tokenization.
Read Tutorial