Archive: questions that belong to later weeks
Navigation: Question index · Exam study guide · Knowledge map
The DA5004 papers also test material from later weeks of the course (tokenisation, encoder-decoder pre-training, prompting and more). Those 247 question records are outside the Weeks 1 to 4 notes, so they have no worked answer here. Each id below links to the question as extracted from its paper, with the scanned stem.
Archived questions by likely later week and reason
Archive records have no Week 1–4 solution bank entry. Their reasons and IDs are retained below.
Likely later week 5 — BERT preprocessing combines subword tokenization with special tokens (1)
Likely later week 5 — BPE tokenization (5)
da5004_2024T1_ET_AN--q147, da5004_2024T1_ET_FN--q204, da5004_2024T1_ET_FN--q205, da5004_2026T1_Q2_NA--q15, da5004_2026T2_Q2_NA--q6
Likely later week 5 — BPE tokenizer coverage (1)
Likely later week 5 — BPE vocabulary and merge operations (7)
da5004_2025T1_Q2_NA--q130, da5004_2025T1_Q2_NA--q131, da5004_2025T1_Q2_NA--q132, da5004_2025T1_Q2_NA--q133, da5004_2025T1_Q2_NA--q134, da5004_2025T1_Q2_NA--q135, da5004_2025T1_Q2_NA--q136
Likely later week 5 — BPE vocabulary construction (2)
da5004_2024T1_Q2_NA--q134, da5004_2026T1_Q2_NA--q16
Likely later week 5 — Character-level tokenization (1)
Likely later week 5 — SentencePiece subword tokenization (2)
da5004_2025T2_ET_AN--q192, da5004_2026T1_Q2_NA--q20
Likely later week 5 — Subword tokenization and tokenizer preprocessing (3)
da5004_2025T2_ET_FN--q138, da5004_2025T2_ET_FN--q140, da5004_2025T2_ET_FN--q144
Likely later week 5 — Subword tokenization, vocabulary design or BPE/WordPiece (9)
da5004_2025T2_Q2_NA--q269, da5004_2025T2_Q2_NA--q271, da5004_2025T2_Q2_NA--q274, da5004_2025T2_Q2_NA--q277, da5004_2025T2_Q2_NA--q281, da5004_2025T2_Q2_NA--q282, da5004_2025T2_Q2_NA--q283, da5004_2025T2_Q2_NA--q284, da5004_2025T2_Q2_NA--q285
Likely later week 5 — Tokenization and BPE vocabulary construction (9)
da5004_2024T3_Q2_NA--q140, da5004_2024T3_Q2_NA--q141, da5004_2024T3_Q2_NA--q142, da5004_2024T3_Q2_NA--q143, da5004_2024T3_Q2_NA--q144, da5004_2024T3_Q2_NA--q145, da5004_2024T3_Q2_NA--q146, da5004_2024T3_Q2_NA--q147, da5004_2024T3_Q2_NA--q148
Likely later week 5 — Tokenizer and subword vocabulary methods (10)
da5004_2025T3_ET_AN--q4, da5004_2025T3_ET_FN--q5, da5004_2025T3_Q2_NA--q107, da5004_2025T3_Q2_NA--q108, da5004_2025T3_Q2_NA--q109, da5004_2025T3_Q2_NA--q111, da5004_2025T3_Q2_NA--q112, da5004_2025T3_Q2_NA--q125, da5004_2025T3_Q2_NA--q126, da5004_2026T1_ET_AN--q6
Likely later week 5 — Tokenizer vocabulary tradeoffs or tokenization pipeline (2)
da5004_2025T1_Q2_NA--q119, da5004_2025T1_Q2_NA--q121
Likely later week 5 — Unigram tokenization and Viterbi segmentation (1)
Likely later week 5 — Unigram tokenization and segmentation (1)
Likely later week 5 — Vocabulary-based token segmentation (1)
Likely later week 5 — WordPiece tokenization (1)
Likely later week 5 — WordPiece vocabulary and merging (10)
da5004_2024T3_ET_AN--q185, da5004_2024T3_ET_AN--q186, da5004_2024T3_ET_AN--q187, da5004_2024T3_ET_AN--q188, da5004_2024T3_ET_AN--q189, da5004_2024T3_ET_FN--q202, da5004_2024T3_ET_FN--q203, da5004_2024T3_ET_FN--q204, da5004_2024T3_ET_FN--q205, da5004_2024T3_ET_FN--q206
Likely later week 5 — interchanging BPE tokenizers (1)
Likely later week 5 — tokenizer and model pipeline (2)
da5004_2024T3_ET_AN--q180, da5004_2024T3_ET_FN--q197
Likely later week 5 — vocabulary size and tokenization (2)
da5004_2024T3_ET_AN--q190, da5004_2024T3_ET_FN--q191
Likely later week 7 — BART architecture (1)
Likely later week 7 — BART denoising architecture (1)
Likely later week 7 — BART, T5, or prefix language-model objectives and comparisons (13)
da5004_2025T3_ET_AN--q21, da5004_2025T3_ET_FN--q7, da5004_2025T3_ET_FN--q19, da5004_2025T3_Q2_NA--q118, da5004_2025T3_Q2_NA--q120, da5004_2025T3_Q2_NA--q121, da5004_2025T3_Q2_NA--q122, da5004_2025T3_Q2_NA--q123, da5004_2025T3_Q2_NA--q124, da5004_2026T1_ET_AN--q3, da5004_2026T1_ET_AN--q19, da5004_2026T1_ET_FN--q13, da5004_2026T1_ET_FN--q18
Likely later week 7 — Comparison includes T5 span corruption and prefix language modeling (1)
Likely later week 7 — Comparison of prefix LM and T5-style denoising objectives (1)
Likely later week 7 — Denoising objectives across BERT, BART and T5 (1)
Likely later week 7 — GPT-2 positional shift evaluation (2)
da5004_2024T3_ET_AN--q183, da5004_2024T3_ET_FN--q200
Likely later week 7 — GPT-2 zero-shot transfer (1)
Likely later week 7 — GPT-2 zero-shot transfer and prompting (1)
Likely later week 7 — Multitask dataset sampling during pretraining (1)
Likely later week 7 — Prefix language modeling and its bidirectional-prefix mask (2)
da5004_2024T3_Q2_NA--q151, da5004_2025T1_Q2_NA--q129
Likely later week 7 — Prefix language modelling objective (1)
Likely later week 7 — T5 or BART model-specific adaptation or denoising (4)
da5004_2025T1_Q2_NA--q122, da5004_2025T1_Q2_NA--q123, da5004_2025T1_Q2_NA--q124, da5004_2025T1_Q2_NA--q128
Likely later week 7 — T5 pretraining objective experiments (1)
Likely later week 7 — T5 span corruption and sentinel tokens (1)
Likely later week 7 — T5 span corruption objective (1)
Likely later week 7 — T5 span-corruption denoising objective (1)
Likely later week 7 — T5 text-to-text fine-tuning (2)
da5004_2024T3_ET_FN--q195, da5004_2026T2_Q2_NA--q12
Likely later week 7 — T5 text-to-text or span-corruption objective (2)
da5004_2025T2_Q2_NA--q272, da5004_2025T2_Q2_NA--q273
Likely later week 7 — T5 text-to-text task formulation (1)
Likely later week 7 — T5-era architecture comparison experiments and prefix LM (1)
Likely later week 7 — comparison of BERT with BART and T5 denoising objectives (1)
Likely later week 7 — text-to-text regression fine-tuning (1)
Likely later week 8 — C4 dataset filtering (2)
da5004_2024T3_ET_AN--q192, da5004_2024T3_ET_FN--q192
Likely later week 8 — C4 training data filtering (1)
Likely later week 8 — Compute-optimal scaling laws (1)
Likely later week 8 — GPT-3 architecture and pretraining data (1)
Likely later week 8 — GPT-3 model-specific architecture and pretraining data (1)
Likely later week 8 — Multitask pretraining data mixing strategy (1)
Likely later week 8 — Pretraining data duplication and filtering (2)
da5004_2025T1_Q2_NA--q120, da5004_2025T1_Q2_NA--q126
Likely later week 8 — Pretraining data quality, filtering or scaling laws (4)
da5004_2024T3_Q2_NA--q149, da5004_2024T3_Q2_NA--q152, da5004_2024T3_Q2_NA--q153, da5004_2024T3_Q2_NA--q155
Likely later week 8 — Pretraining data, datasets, or scaling (6)
da5004_2025T3_Q2_NA--q114, da5004_2025T3_Q2_NA--q116, da5004_2025T3_Q2_NA--q117, da5004_2025T3_Q2_NA--q119, da5004_2026T1_ET_AN--q9, da5004_2026T1_ET_FN--q7
Likely later week 8 — Scaling laws and training data (1)
Likely later week 8 — Scaling laws or pretraining data filtering (4)
da5004_2025T2_Q2_NA--q270, da5004_2025T2_Q2_NA--q275, da5004_2025T2_Q2_NA--q276, da5004_2025T2_Q2_NA--q280
Likely later week 8 — Training data deduplication (1)
Likely later week 8 — Training data duplication (1)
Likely later week 8 — Training data preprocessing and filtering (1)
Likely later week 8 — Training data quality, diversity and scale (1)
Likely later week 8 — dataset size and repeated training effects (2)
da5004_2024T1_Q2_NA--q141, da5004_2024T1_Q2_NA--q142
Likely later week 8 — pretraining data duplication (2)
da5004_2024T3_ET_AN--q181, da5004_2024T3_ET_FN--q198
Likely later week 8 — pretraining data filtering (3)
da5004_2024T1_Q2_NA--q145, da5004_2024T1_Q2_NA--q146, da5004_2024T1_Q2_NA--q147
Likely later week 8 — scaling laws (1)
Likely later week 11 — Efficient or sparse attention and inference caching (20)
da5004_2025T3_ET_AN--q7, da5004_2025T3_ET_AN--q18, da5004_2025T3_ET_AN--q19, da5004_2025T3_ET_AN--q22, da5004_2025T3_ET_FN--q9, da5004_2025T3_ET_FN--q10, da5004_2025T3_ET_FN--q20, da5004_2025T3_ET_FN--q21, da5004_2025T3_ET_FN--q22, da5004_2026T1_ET_AN--q4, da5004_2026T1_ET_AN--q12, da5004_2026T1_ET_AN--q15, da5004_2026T1_ET_AN--q16, da5004_2026T1_ET_AN--q20, da5004_2026T1_ET_AN--q21, da5004_2026T1_ET_FN--q5, da5004_2026T1_ET_FN--q8, da5004_2026T1_ET_FN--q9, da5004_2026T1_ET_FN--q14, da5004_2026T1_ET_FN--q19
Likely later week 11 — FlashAttention (1)
Likely later week 11 — Grouped-query attention and KV cache (1)
Likely later week 11 — KV cache memory (1)
Likely later week 11 — KV caching (2)
da5004_2026T2_ET_AN--q15, da5004_2026T2_ET_FN--q14
Likely later week 11 — Kernel-based linear attention (1)
Likely later week 11 — PagedAttention and KV cache serving (1)
Likely later week 11 — Sparse or block attention for efficient long sequences (9)
da5004_2025T2_ET_AN--q194, da5004_2025T2_ET_AN--q196, da5004_2025T2_ET_AN--q201, da5004_2025T2_ET_AN--q202, da5004_2025T2_ET_AN--q203, da5004_2025T2_ET_FN--q139, da5004_2025T2_ET_FN--q148, da5004_2025T2_ET_FN--q149, da5004_2025T2_ET_FN--q150
Likely later week 11 — Strided local attention complexity (1)
Likely later week 11 — block and strided attention (3)
da5004_2024T3_ET_FN--q207, da5004_2024T3_ET_FN--q208, da5004_2024T3_ET_FN--q209
Likely later week 11 — efficient attention variants (1)
Likely later week 11 — sparse attention complexity (2)
da5004_2024T1_ET_AN--q145, da5004_2024T1_ET_FN--q200
Likely later week 11 — strided and block attention (2)
da5004_2024T3_ET_AN--q194, da5004_2024T3_ET_AN--q195
Likely later week 11 — strided local attention complexity (1)
Likely later week 12 — ALiBi positional bias (1)
Likely later week 12 — Clipped relative positional encoding (1)
Likely later week 12 — Comparison of RoPE, ALiBi and relative position encodings (1)
Likely later week 12 — Long-context length generalization (1)
Likely later week 12 — Positional methods beyond sinusoidal encoding (22)
da5004_2025T3_ET_AN--q8, da5004_2025T3_ET_AN--q9, da5004_2025T3_ET_AN--q10, da5004_2025T3_ET_AN--q11, da5004_2025T3_ET_AN--q12, da5004_2025T3_ET_AN--q13, da5004_2025T3_ET_AN--q14, da5004_2025T3_ET_AN--q16, da5004_2025T3_ET_FN--q11, da5004_2025T3_ET_FN--q12, da5004_2025T3_ET_FN--q13, da5004_2025T3_ET_FN--q14, da5004_2025T3_ET_FN--q15, da5004_2025T3_ET_FN--q16, da5004_2025T3_ET_FN--q17, da5004_2026T1_ET_AN--q5, da5004_2026T1_ET_AN--q17, da5004_2026T1_ET_AN--q18, da5004_2026T1_ET_FN--q10, da5004_2026T1_ET_FN--q11, da5004_2026T1_ET_FN--q20, da5004_2026T1_ET_FN--q21
Likely later week 12 — Relative positional embedding clipping (1)
Likely later week 12 — Relative positional embeddings (4)
da5004_2025T2_ET_FN--q151, da5004_2025T2_ET_FN--q152, da5004_2025T2_ET_FN--q153, da5004_2025T2_ET_FN--q154
Likely later week 12 — Relative positional embeddings or ALiBi (5)
da5004_2025T2_ET_AN--q204, da5004_2025T2_ET_AN--q205, da5004_2025T2_ET_AN--q206, da5004_2025T2_ET_AN--q207, da5004_2025T2_ET_AN--q208
Likely later week 12 — Relative positional encoding (1)
Likely later week 12 — RoPE-modified query-key dot product (1)
Likely later week 12 — Rotary positional encoding (2)
da5004_2026T2_ET_AN--q20, da5004_2026T2_ET_FN--q10
Likely later week 12 — Transformer-XL relative position attention (1)
Likely later week 12 — learned positional embedding during generation (1)
Likely later week 12 — learned positional embedding parameter count (1)
Likely later week 12 — position information injected into attention (1)
Likely later week other/unknown — Broad LLM design choices spanning later data and position methods (1)
Likely later week other/unknown — Parameter-efficient adapter fine-tuning (1)
Likely later week other/unknown — Prompting or parameter-efficient fine-tuning beyond Weeks 1–4 (2)
da5004_2026T1_ET_AN--q7, da5004_2026T1_ET_AN--q13
Likely later week other/unknown — Question stem and options absent from source (1)
Likely later week other/unknown — prefix masking and architecture choices for summarization (1)
Likely later week other/unknown — zero-shot and few-shot transfer approaches (1)
Likely later week other/unknown — zero-shot summarization and transfer (1)
Total reconciliation: routed W1 73 + W2 65 + W3 36 + W4 67 + archived 247 = 241 + 247 = 488.