LLM — Knowledge Map

This map organizes an IIT Madras DA5004-style course for Weeks 1–4, scoped by the supplied ebook. Later-week material is archived; the ebook guides scope and the primary papers adjudicate technical claims.

Curriculum Spine

Weekly Topics

Topic names follow sources/topics.json; links identify related concept notes. Some topics also require the weekly notes for full coverage.

Week 1

• Seq2seq and the RNN bottleneck — Sequence-to-Sequence Bottleneck
• Bahdanau (additive) attention — Bahdanau Attention
• Self-attention and Q/K/V projections — Self-Attention
• Scaled dot-product attention and the sqrt(d_k) factor — Scaled Dot-Product Attention
• Multi-head attention and W^O — Multi-Head Attention
• Position-wise FFN and the encoder layer/stack — Position-Wise Feed-Forward Network
• Parameter counting for the encoder — Encoder Layer Parameter Count

Week 2

• Teacher forcing — Teacher Forcing
• Masked (causal) self-attention — Masked Self-Attention
• Cross-attention (encoder-decoder attention) — Cross-Attention
• Decoder output layer and decoder parameter counts — Cross-Attention · Encoder Layer Parameter Count
• Sinusoidal positional encoding — Sinusoidal Positional Encoding
• Residual connections, LayerNorm vs BatchNorm — Residual Connections · Layer Normalization
• Training tricks: label smoothing, warm-up schedule, dropout — Label Smoothing · Learning-Rate Warmup
• The full Transformer data flow — Masked Self-Attention · Cross-Attention · Position-Wise Feed-Forward Network

Week 3

• Language-modelling objective, chain rule and perplexity — Language-Modelling Objective · Perplexity
• Ways to parameterise a language model — Language-Modelling Objective
• Causal language modelling with a decoder-only stack — Causal Language Modelling
• GPT-1 architecture and pre-train/fine-tune paradigm — GPT-1 Architecture · Pretrain–Fine-tune Paradigm
• GPT-1 parameter count (117M) — GPT-1 Architecture
• Pre-training objective and teacher forcing — Language-Modelling Objective · Teacher Forcing
• Fine-tuning GPT and the fine-tuning loss — Pretrain–Fine-tune Paradigm · GPT-1 Architecture
• Task-specific input transformations — Task-Specific Input Transformations

Week 4

• Exhaustive and greedy search — Greedy and Beam Search
• Beam search and length normalisation — Greedy and Beam Search
• Top-k sampling and temperature — Top-k and Top-p Sampling · Temperature Scaling
• Top-p (nucleus) sampling and repetition issues — Top-k and Top-p Sampling
• BERT masked language modelling (80/10/10) — BERT Masked Language Modelling
• NSP, [CLS]/[SEP] and the three input embeddings — Next Sentence Prediction · BERT Input Embeddings
• BERT sizes and parameter counts — BERT Parameter Count
• BERT fine-tuning heads and GPT vs BERT — Pretrain–Fine-tune Paradigm · BERT Masked Language Modelling · GPT-1 Architecture

Concept Taxonomy

Attention family

Sequence-to-Sequence Bottleneck · Bahdanau Attention · Self-Attention · Scaled Dot-Product Attention · Multi-Head Attention · Masked Self-Attention · Cross-Attention

Transformer components & training

Position-Wise Feed-Forward Network · Encoder Layer Parameter Count · Sinusoidal Positional Encoding · Residual Connections · Layer Normalization · Label Smoothing · Learning-Rate Warmup · Teacher Forcing

Language modelling & GPT

Language-Modelling Objective · Perplexity · Causal Language Modelling · GPT-1 Architecture · Pretrain–Fine-tune Paradigm · Task-Specific Input Transformations

Decoding

Greedy and Beam Search · Top-k and Top-p Sampling · Temperature Scaling

BERT

BERT Masked Language Modelling · BERT Input Embeddings · Next Sentence Prediction · BERT Parameter Count

Central Governing Paradigm

seq2seq → attention → self-attention → full Transformer
                                          ├→ decoder-only (GPT) → decoding
                                          └→ encoder-only (BERT) → task predictions

Exam Resources and Sources

• LLM Exam Question Index — Weeks 1–4 lists every paper and how each question was routed; Archive: questions that belong to later weeks holds the questions that belong to later weeks; LLM Exam Study Guide — Weeks 1–4 ranks topics by how often they were tested. Each week has its own question bank.
• How this was checked summarises the verification of routing, banks and links. The course ebook is a third-party document and is not republished; the notes cite its page numbers. The extracted exam papers and their scans are in the paper archive.

Answer provenance: All answers are derived; none are official answer keys.