Week 3: Exam Questions and Worked Answers

2024 T1 Quiz 1 NA, Q193

Paper: 2024T1_Q1_NA · Type: MSQ · Marks: 4 · Topic(s): w3-causal-lm

A team trained a Language model using the GPT architecture using a large corpus of text. The model configurations are given below:

• Vocabulary size = 30000
• \(d_{model} = 512\)
• length of context window \(T = 1024\)
• \(d_{ff} = 4 * d_{model}\)
• \(d_q = d_k = d_v = 64\)

They trained the model for millions of steps and stopped the training after the loss had decreased significantly. Then, they used the model to generate the text by giving the special starting token [BOS] as an input to the model. Select the length of the text sequence that the model could generate.

A. 512

Label added for presentation.

B. 1024

Label added for presentation.

C. 1536

Label added for presentation.

D. 2048

Label added for presentation.

Worked answer

Final answer: A if BOS counts within a 1024-token full sequence; A, B if 1024 newly predicted tokens are counted; A–D with sliding

Steps: A capped full sequence leaves 1023 positions after BOS; a final prediction can instead be emitted from a full input context, allowing 1024 new tokens. Sliding or resetting positional IDs permits still longer text. The exam’s fixed-window treatment is the closest course convention; its wording does not settle output counting.

Status: ambiguous

Ambiguity note: Implied course/exam convention: A if BOS counts within a 1024-token full sequence; A, B if 1024 newly predicted tokens are counted; A–D with sliding. A capped full sequence leaves 1023 positions after BOS; a final prediction can instead be emitted from a full input context, allowing 1024 new tokens. Sliding or resetting positional IDs permits still longer text. The exam’s fixed-window treatment is the closest course convention; its wording does not settle output counting.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2024 T1 Quiz 1 NA, Q194

Paper: 2024T1_Q1_NA · Type: MSQ · Marks: 4 · Topic(s): w3-finetuning, w3-pretraining-objective

Choose the correct statements

A. All the parameters of the GPT model are randomly initialized during pre-training

Label added for presentation.

B. The model minimizes the CLM objective during fine-tuning to improve the performance

Label added for presentation.

C. All parameters of the GPT model are randomly initialized during fine-tuning

Label added for presentation.

D. In general, fine-tuning requires a dataset with labels

Label added for presentation.

Worked answer

Final answer: A, B, D for the original GPT-1 supervised experiments; A, D for ordinary supervised fine-tuning without the optional auxiliary CLM loss

Steps: GPT-1 pretraining starts from random weights; supervised fine-tuning reuses them and typically uses labels. Radford et al. included an auxiliary language-model objective in their experiments, while fine-tuning in general need not. The paper-specific and generic conventions differ, and the course paper implies the GPT-1 experiment.

Status: ambiguous

Ambiguity note: Implied course/exam convention: A, B, D for the original GPT-1 supervised experiments; A, D for ordinary supervised fine-tuning without the optional auxiliary CLM loss. GPT-1 pretraining starts from random weights; supervised fine-tuning reuses them and typically uses labels. Radford et al. included an auxiliary language-model objective in their experiments, while fine-tuning in general need not. The paper-specific and generic conventions differ, and the course paper implies the GPT-1 experiment.

verified_by_code: false — No numeric checker path; answer remains explicitly ambiguous.

Shared context

Question ID: 640653993611

Type: Comprehension
PDF pages: 8

Consider a vocabulary \(V\),

\(V = (\text{[start]}, \text{breaking}, \text{ground}, \text{a}, \text{is}, \text{seismology}, \text{study}, \text{experience}, \text{earth quake}, \text{[end]})\).

Assume that we have a pre-trained GPT model for text generation and the first input token to the model is always the special token [start]. The prediction probabilities outputted by the model are given below.

The columns of the matrix are in the same order as the words listed in the vocabulary set above.

The zeroth row of the matrix is the output probability distribution by the model given the special [Start] token as input.

Following the Greedy Search decoding strategy, the subsequent rows give the conditional probability distribution conditioned over the previous tokens. If you think the given information is insufficient for any sub-questions, then enter -1 as your answer.

!Pre-trained GPT vocabulary and prediction probability distribution matrix for greedy decoding

Figure 6. Pre-trained GPT vocabulary and prediction probability distribution matrix for greedy decoding

Based on the above data, answer the given subquestions.

Question 285

Corrected shared-context text: Vocabulary V=([start], breaking, ground, a, is, seismology, study, experience, earth quake, [end]). Rows of the probability matrix correspond to greedy prefixes, beginning with [start]; columns follow V. Matrix: [[.04,.01,.14,.07,.09,.41,.08,.05,.08,.01],[.10,.03,.06,.16,.43,.01,.06,.09,.01,.04],[.06,.08,.28,.29,.03,.08,.04,.08,.04,.04],[.11,.03,.48,.02,.01,.07,.06,.14,.07,.01],[.09,.29,.04,.01,.04,.03,.25,.05,.15,.06],[.14,.22,.13,.08,.01,.03,.06,.23,.08,.03]]. Enter -1 if information is insufficient.

Dependent question

2024 T3 Quiz 1 NA, Q285

Paper: 2024T3_Q1_NA · Type: SA · Marks: 1 · Topic(s): w3-language-modelling

What is the probability of the predicted word equal to “Seismology” given the “[Start]” token as the initial input \(P(y_1 = \text{Seismology} \mid y_0 = \text{[Start]})\)?

Editorial correction: the extracted text differed from the scan (shared_context); see evidence LLM/sources/exam-papers/assets/da5004_2024T3_Q1_NA/ctx-285-288-passage-01-p008.png shows vocabulary, probability matrix, greedy-prefix conditioning and insufficient-data instruction..

Worked answer

Final answer: 0.41.

Steps: The initial row gives P(seismology|[start])=0.41.

Status: derived

verified_by_code: false — depends on figure, no code path

Shared context

Question ID: 640653993611

Type: Comprehension
PDF pages: 8

Consider a vocabulary \(V\),

\(V = (\text{[start]}, \text{breaking}, \text{ground}, \text{a}, \text{is}, \text{seismology}, \text{study}, \text{experience}, \text{earth quake}, \text{[end]})\).

Assume that we have a pre-trained GPT model for text generation and the first input token to the model is always the special token [start]. The prediction probabilities outputted by the model are given below.

The columns of the matrix are in the same order as the words listed in the vocabulary set above.

The zeroth row of the matrix is the output probability distribution by the model given the special [Start] token as input.

Following the Greedy Search decoding strategy, the subsequent rows give the conditional probability distribution conditioned over the previous tokens. If you think the given information is insufficient for any sub-questions, then enter -1 as your answer.

!Pre-trained GPT vocabulary and prediction probability distribution matrix for greedy decoding

Figure 6. Pre-trained GPT vocabulary and prediction probability distribution matrix for greedy decoding

Based on the above data, answer the given subquestions.

Question 285

Corrected shared-context text: Vocabulary V=([start], breaking, ground, a, is, seismology, study, experience, earth quake, [end]). Rows of the probability matrix correspond to greedy prefixes, beginning with [start]; columns follow V. Matrix: [[.04,.01,.14,.07,.09,.41,.08,.05,.08,.01],[.10,.03,.06,.16,.43,.01,.06,.09,.01,.04],[.06,.08,.28,.29,.03,.08,.04,.08,.04,.04],[.11,.03,.48,.02,.01,.07,.06,.14,.07,.01],[.09,.29,.04,.01,.04,.03,.25,.05,.15,.06],[.14,.22,.13,.08,.01,.03,.06,.23,.08,.03]]. Enter -1 if information is insufficient.

Dependent question

2024 T3 Quiz 1 NA, Q286

Paper: 2024T3_Q1_NA · Type: SA · Marks: 2 · Topic(s): w3-language-modelling

Say the probability of the sequence “Seismology is a ground breaking experience” is p, then what is \(1000 \times p\)?

Editorial correction: the extracted text differed from the scan (shared_context); see evidence LLM/sources/exam-papers/assets/da5004_2024T3_Q1_NA/ctx-285-288-passage-01-p008.png shows vocabulary, probability matrix, greedy-prefix conditioning and insufficient-data instruction..

Worked answer

Final answer: Approximately 1.637.

Steps: The six successive tokens follow the greedy path, so 1000 × 0.41 × 0.43 × 0.29 × 0.48 × 0.29 × 0.23 = 1.636882032.

Status: derived

verified_by_code: false — depends on figure, no code path

Shared context

Question ID: 640653993611

Type: Comprehension
PDF pages: 8

Consider a vocabulary \(V\),

\(V = (\text{[start]}, \text{breaking}, \text{ground}, \text{a}, \text{is}, \text{seismology}, \text{study}, \text{experience}, \text{earth quake}, \text{[end]})\).

Assume that we have a pre-trained GPT model for text generation and the first input token to the model is always the special token [start]. The prediction probabilities outputted by the model are given below.

The columns of the matrix are in the same order as the words listed in the vocabulary set above.

The zeroth row of the matrix is the output probability distribution by the model given the special [Start] token as input.

Following the Greedy Search decoding strategy, the subsequent rows give the conditional probability distribution conditioned over the previous tokens. If you think the given information is insufficient for any sub-questions, then enter -1 as your answer.

!Pre-trained GPT vocabulary and prediction probability distribution matrix for greedy decoding

Figure 6. Pre-trained GPT vocabulary and prediction probability distribution matrix for greedy decoding

Based on the above data, answer the given subquestions.

Question 285

Corrected shared-context text: Vocabulary V=([start], breaking, ground, a, is, seismology, study, experience, earth quake, [end]). Rows of the probability matrix correspond to greedy prefixes, beginning with [start]; columns follow V. Matrix: [[.04,.01,.14,.07,.09,.41,.08,.05,.08,.01],[.10,.03,.06,.16,.43,.01,.06,.09,.01,.04],[.06,.08,.28,.29,.03,.08,.04,.08,.04,.04],[.11,.03,.48,.02,.01,.07,.06,.14,.07,.01],[.09,.29,.04,.01,.04,.03,.25,.05,.15,.06],[.14,.22,.13,.08,.01,.03,.06,.23,.08,.03]]. Enter -1 if information is insufficient.

Dependent question

2024 T3 Quiz 1 NA, Q287

Paper: 2024T3_Q1_NA · Type: SA · Marks: 2 · Topic(s): w3-language-modelling

What is the probability of the sequence “Seismology is earthquake study ”?

Editorial correction: the extracted text differed from the scan (shared_context); see evidence LLM/sources/exam-papers/assets/da5004_2024T3_Q1_NA/ctx-285-288-passage-01-p008.png shows vocabulary, probability matrix, greedy-prefix conditioning and insufficient-data instruction..

Worked answer

Final answer: -1

Steps: The pictured passage explicitly instructs entry of -1 for insufficient information. The requested earthquake path deviates from greedy at the third token; P(study | seismology,is,earthquake) is absent. The unknown sequence probability therefore makes -1 the uniquely prescribed answer; the old note identifies the missing conditional correctly but overlooks the resulting resolvability.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2024 T3 Quiz 1 NA, Q289

Paper: 2024T3_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w3-gpt1-parameter-count

What will be the shape of positional embedding?

A. 64 × 128

Label added for presentation.

B. 64 × 4

Label added for presentation.

C. 12 × 128

Label added for presentation.

D. 512 × 768

Label added for presentation.

E. None of these.

Label added for presentation.

Worked answer

Final answer: A: 64 × 128.

Steps: Learned positional embeddings have one 128-dimensional row per position, giving 64 × 128. B uses head count as width, C uses 12 positions, and D is an unrelated configuration. E is false.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2025 T1 Quiz 1 NA, Q108

Paper: 2025T1_Q1_NA · Type: MSQ · Marks: 2 · Topic(s): w3-finetuning, w3-gpt1-architecture

Choose the correct statements

A. All the parameters of the GPT model are randomly initialized during pre-
training

B. The model minimizes the CLM objective during fine-tuning to improve the
performance

C. All parameters of the GPT model are randomly initialized during fine-tuning

D. In general, fine-tuning requires a dataset with labels

Worked answer

Final answer: A, B, D for the original GPT-1 supervised experiments; A, D for ordinary supervised fine-tuning without the optional auxiliary CLM loss

Steps: GPT-1 pretraining starts from random weights; supervised fine-tuning reuses them and typically uses labels. Radford et al. included an auxiliary language-model objective in their experiments, while fine-tuning in general need not. The paper-specific and generic conventions differ, and the course paper implies the GPT-1 experiment.

Status: ambiguous

Ambiguity note: Implied course/exam convention: A, B, D for the original GPT-1 supervised experiments; A, D for ordinary supervised fine-tuning without the optional auxiliary CLM loss. GPT-1 pretraining starts from random weights; supervised fine-tuning reuses them and typically uses labels. Radford et al. included an auxiliary language-model objective in their experiments, while fine-tuning in general need not. The paper-specific and generic conventions differ, and the course paper implies the GPT-1 experiment.

verified_by_code: false — No numeric checker path; answer remains explicitly ambiguous.

2025 T1 Quiz 1 NA, Q118

Paper: 2025T1_Q1_NA · Type: SA · Marks: 2 · Topic(s): w3-language-modelling

What is the value of \(10 \times P(\text{Sachin})\)?

Note: \(x_1 = \text{Sachin}\)

Worked answer

Final answer: 60/41 ≈ 1.4634146

Steps: The passage asks for an estimate from the given corpus. Ignoring punctuation leaves 41 tokens, six equal to Sachin; empirical P(Sachin)=6/41, so 10P=60/41. Uniform empirical token counting is the natural data-based estimate, not a missing parameter.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2025 T1 Quiz 1 NA, Q119

Paper: 2025T1_Q1_NA · Type: SA · Marks: 2 · Topic(s): w3-language-modelling

What is the value of \(100 \times P(\text{Sachin}, \text{Tendulkar})\)?

Note: \(x_1 = \text{Sachin}\) and \(x_2 = \text{Tendulkar}\)

Worked answer

Final answer: 100/41 ≈ 2.4390244

Steps: After punctuation removal the 41-token corpus contains the consecutive pair Sachin Tendulkar once. Empirical chain-rule estimation gives P(Sachin)P(Tendulkar|Sachin)=(6/41)(1/6)=1/41, and 100P=100/41. The printed upper bound 2.43 excludes this exact result by about 0.009, apparently a key tolerance issue.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2025 T1 Quiz 1 NA, Q120

Paper: 2025T1_Q1_NA · Type: SA · Marks: 3 · Topic(s): w3-language-modelling

What is the value of the following expression \(100 \times P(\text{Sachin}, \text{has}, \text{broken}, \text{many}, \text{records})\)?

Note: \(x_1 = \text{Sachin}\), \(x_2 = \text{has}\), and so on.

Worked answer

Final answer: 100/41 ≈ 2.4390244

Steps: The 41-token corpus contains the requested five-token phrase once. Chain-rule empirical counts telescope: (6/41)(4/6)(2/4)(2/2)(1/2)=1/41; multiply by 100. The printed 2.43 upper bound narrowly excludes the result, a key tolerance issue.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2025 T1 Quiz 1 NA, Q121

Paper: 2025T1_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w3-language-modelling

Once the model is sufficiently trained, which of the following tokens will have the highest
probability to be the next token, if the input is “Sachin has”:

A. broken

B. acted

C. highest

D. None of these

Worked answer

Final answer: A: broken

Steps: The requested corpus-based estimate counts broken twice after “Sachin has”, versus acted once and highest once. An adequately fitted model is intended to reflect the empirical ranking; the old caveat is mathematically true for arbitrary training but ignores the stated “sufficiently trained” teaching context.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T2 Quiz 1 NA, Q140

Paper: 2025T2_Q1_NA · Type: MSQ · Marks: 3 · Topic(s): w3-causal-lm

Which of the following statements are true regarding causal language modeling (CLM)?

A. The model only attends to past and current tokens during training.

B. The model is trained by predicting the next token in a sequence.

C. The model uses bidirectional context.

D. The CLM objective is commonly used for encoder-only models.

Worked answer

Final answer: A and B.

Steps: Causal attention admits current and earlier positions (A), and predicts the next token (B). C requires future context and D describes the usual encoder-only MLM setting.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T2 Quiz 1 NA, Q141

Paper: 2025T2_Q1_NA · Type: MSQ · Marks: 3 · Topic(s): w3-gpt1-architecture

Which of the following are valid reasons why transformer-based large language models are widely used in natural language processing?

A. Transformers process input sequences in parallel, enabling faster training.

B. They use recurrence to remember long-term dependencies more effectively than LSTMs.

C. Transformers are limited to short text inputs due to their architecture.

D. Large transformer models can be fine-tuned for various NLP tasks using a single pre-trained model.

Worked answer

Final answer: A and D.

Steps: A: masked attention parallelizes training across positions; D: pretrained weights transfer to many tasks. B incorrectly introduces recurrence; C overstates the finite context limit.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 Quiz 1 NA, Q189

Paper: 2025T3_Q1_NA · Type: SA · Marks: 3 · Topic(s): w3-pretraining-objective

How many next-token prediction targets are generated during training a full batch?

Worked answer

Final answer: 32736 targets.

Steps: Without BOS, each length-1024 sequence supplies 1023 next-token labels; 32 × 1023 = 32736.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T1 Quiz 1 NA, Q16

Paper: 2026T1_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w3-gpt1-parameter-count

What is the number of parameters in the token embedding matrix?

A. 30,720,000

B. 40,768

C. 393,216

D. 15,728,640,000

Worked answer

Final answer: A

Steps: Token matrix [40000, 768]: 40000 × 768 = 30720000; B adds dimensions, C is position count and D multiplies unrelated dimensions.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T1 Quiz 1 NA, Q17

Paper: 2026T1_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w3-gpt1-parameter-count

What is the total number of parameters in the positional embedding matrix?

A. 512,768

B. 393,216

C. 3,932,160

D. 39,321

Worked answer

Final answer: B

Steps: Position matrix [512, 768]: 512 × 768 = 393216; A adds dimensions, C and D have incorrect decimal scale.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T1 Quiz 1 NA, Q18

Paper: 2026T1_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w3-gpt1-parameter-count

For ONE complete attention head, what is the total number of parameters (Q + K + V projections combined)?

A. 49,152

B. 196,608

C. 589,824

D. 147,456

Worked answer

Final answer: D: 147,456 weights; 147,648 including Q/K/V biases

Steps: With width 768 and 12 heads, each head width is 64. The three 768×64 projection matrices contain 147,456 weights; three bias vectors add 192. D and the course-style matrix count imply bias-free projections, whereas the track’s GPT parameter convention includes biases.

Status: ambiguous

Ambiguity note: Implied course/exam convention: D: 147,456 weights; 147,648 including Q/K/V biases. With width 768 and 12 heads, each head width is 64. The three 768×64 projection matrices contain 147,456 weights; three bias vectors add 192. D and the course-style matrix count imply bias-free projections, whereas the track’s GPT parameter convention includes biases.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T1 Quiz 1 NA, Q19

Paper: 2026T1_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w3-gpt1-parameter-count

After the multi-head attention computation, all head outputs are concatenated and projected back to the model dimension.

How many parameters are in the output projection matrix (weights only, excluding bias)?

A. 768

B. 49,152

C. 589,824

D. 294,912

Worked answer

Final answer: C

Steps: The concatenated heads have width 768; output weights [768, 768] contain 589824. A, B and D use smaller incorrect dimensions.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T1 Quiz 1 NA, Q20

Paper: 2026T1_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w3-gpt1-parameter-count

What is the total number of parameters in the complete FFN (including both weight matrices and bias vectors)?

A. 2,359,296

B. 4,718,592

C. 4,722,432

D. 9,437,184

Worked answer

Final answer: C

Steps: Two weight matrices plus biases: 2 × 768 × 3072 + 3072 + 768 = 4722432. A omits a matrix; B omits biases; D doubles the count.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T2 Quiz 1 NA, Q10

Paper: 2026T2_Q1_NA · Type: SA · Marks: 3 · Topic(s): w3-language-modelling

Compute the joint probability of the sentence : "You eat apples"

(Provide answer correct upto 2 digits after the decimal)

Worked answer

Final answer: 0.18.

Steps: From the tree: P(you)P(eat|you)P(apples|you,eat)=0.4×0.5×0.9=0.18.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T2 Quiz 1 NA, Q11

Paper: 2026T2_Q1_NA · Type: MCQ · Marks: 3 · Topic(s): w3-language-modelling

Choose the option corresponding to the sentence with the highest joint probability under the given language model.

A. "i eat apples"

B. "apples like You"

C. "i eat bananas"

D. "you like bananas

E. "you like apples

Worked answer

Final answer: C if the omitted B path is below 0.09; B if it exceeds 0.09; tie at 0.09

Steps: The pictured partial tree gives A=.5×.3×.4=.06, C=.5×.3×.6=.09, D=.4×.4×.5=.08, and E=.4×.4×.4=.064. B starts with an unpictured apples branch whose mass may be as high as .1; its continuation probabilities are not given. Both C and B can win.

Status: ambiguous

Ambiguity note: The pictured partial tree gives A=.5×.3×.4=.06, C=.5×.3×.6=.09, D=.4×.4×.5=.08, and E=.4×.4×.4=.064. B starts with an unpictured apples branch whose mass may be as high as .1; its continuation probabilities are not given. Both C and B can win.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T2 Quiz 1 NA, Q12

Paper: 2026T2_Q1_NA · Type: SA · Marks: 2 · Topic(s): w3-language-modelling

Using the given probability tree calculate the conditional probability of predicting "apples" given "like" [i.e P(apples | like)]. (Submit -1 if the provided information is insufficient)

Worked answer

Final answer: -1

Steps: The pictured tree is explicitly partial, and the question explicitly prescribes -1 if the information is insufficient. P(apples|like) marginalizes over all first-token contexts; omitted first tokens may also precede like. The requested ratio is not determined, so -1 is the unique prescribed response.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T2 Quiz 1 NA, Q13

Paper: 2026T2_Q1_NA · Type: MSQ · Marks: 3 · Topic(s): w3-causal-lm

Which of the following are components found within a GPT decoder layer?

A. Causal (Masked) Multi-Head Self-Attention

B. Position-wise Feed-Forward Network

C. Multi Head (Cross) Attention

D. Recurrent Neural Network (RNN) Layer

E. Add & Norm Layer

Worked answer

Final answer: A, B and E.

Steps: GPT decoder blocks contain causal self-attention, positionwise FFN, residual additions and layer normalization. C is encoder–decoder cross-attention, absent in GPT-1; D is recurrent, not a GPT block.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2024 T1 Quiz 2 NA, Q135

Paper: 2024T1_Q2_NA · Type: MSQ · Marks: 4 · Topic(s): w3-gpt1-parameter-count

Choose the correct statements about the number of parameters in the model (excluding the
embedding and output layer parameters)

A. Team A model has less number of parameters than the team B

B. Team B model has less number of parameters than the team C

C. Team C model has more number of parameters than the team A

D. All models have the same number of parameters

Worked answer

Final answer: D: all have the same decoder parameter count.

Steps: All share two layers and identical widths, heads and FFN dimensions; vocabulary changes only excluded embedding/output parameters. Thus A, B and C are false.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T1 Quiz 2 NA, Q13

Paper: 2026T1_Q2_NA · Type: MCQ · Marks: 2 · Topic(s): w3-causal-lm

Is the given attention matrix valid for a causal language modelling task?

A. True

B. False

C. Insufficient information

Worked answer

Final answer: B: false.

Steps: All future entries are zero, but row t=5 sums to 0.90, not 1. A is false for an attention probability matrix; C is false because the supplied entries suffice.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T2 Quiz 2 NA, Q8

Paper: 2026T2_Q2_NA · Type: MCQ · Marks: 3 · Topic(s): w3-task-input-formats

During the fine-tuning of GPT-1 for Textual Entailment tasks, what is the purpose of the delimiter token ($)?

A. To trigger the softmax function.

B. To separate premise and hypothesis.

C. To mark the end of the sentence.

D. To initialize the linear head.

Worked answer

Final answer: B: separate premise and hypothesis.

Steps: GPT-1 places a delimiter between the two text segments. A softmax is applied by the classification head; C uses a separate end token; D the delimiter does not initialize parameters.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T2 Quiz 2 NA, Q9

Paper: 2026T2_Q2_NA · Type: MCQ · Marks: 3 · Topic(s): w3-finetuning

During the fine-tuning of a decoder-only Transformer for sequence classification, why is the hidden representation of the final input token used as the input to the classification layer?

A. The first token's representation relies exclusively on absolute positional
encodings and lacks semantic context for classification.

B. The final token's representation is computed independently of self-attention,
making its gradients more stable during the fine-tuning process.

C. Selecting the final token aligns with the autoregressive objective, eliminating
the need for teacher forcing during the classification forward pass.

D. Causal self-attention ensures that only the final token's representation aggregates information from all preceding tokens in the sequence.

Worked answer

Final answer: D.

Steps: Causal attention permits the final representation to incorporate all earlier tokens. A overstates positional embeddings, B denies attention, and C confuses classification with teacher forcing.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2024 T1 End Term AN, Q144

Paper: 2024T1_ET_AN · Type: MCQ · Marks: 2 · Topic(s): w3-gpt1-architecture

GPT stands for

A. Generative Pre-trained Transformer

Label added for presentation.

B. Generative Pre-trained Text

Label added for presentation.

C. Generalized Pre-trained Transformer

Label added for presentation.

D. Generative Pre-trained Transfer-text

Label added for presentation.

Worked answer

Final answer: A: Generative Pre-trained Transformer.

Steps: GPT abbreviates Generative Pre-trained Transformer. B substitutes Text, C Generalized, and D Transfer-text.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2024 T1 End Term AN, Q149

Paper: 2024T1_ET_AN · Type: MSQ · Marks: 3 · Topic(s): w3-causal-lm

A user runs inference on the model N times with an input text that contains 712 tokens (including the special [start] token), then which of the following could be the possible length of summarized text

A. 256

Label added for presentation.

B. 16

Label added for presentation.

C. 1024

Label added for presentation.

D. 512

Label added for presentation.

E. 2048

Label added for presentation.

Worked answer

Final answer: A, B, C, D with a fixed complete-sequence window; A–E with sliding and reset position IDs

Steps: The shared prompt gives T=2048 and 712 prompt tokens, leaving 1336 slots under a complete-sequence cap, so 256, 16, 1024 and 512 fit. A sliding context can generate 2048; the supplied prompt does not specify inference policy. The exam key implies the fixed-window reading.

Status: ambiguous

Ambiguity note: Implied course/exam convention: A, B, C, D with a fixed complete-sequence window; A–E with sliding and reset position IDs. The shared prompt gives T=2048 and 712 prompt tokens, leaving 1336 slots under a complete-sequence cap, so 256, 16, 1024 and 512 fit. A sliding context can generate 2048; the supplied prompt does not specify inference policy. The exam key implies the fixed-window reading.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2024 T1 End Term FN, Q202

Paper: 2024T1_ET_FN · Type: MSQ · Marks: 3 · Topic(s): w3-causal-lm

A user runs inference on the model N times with an input text that contains 712 tokens (including the special [start] token), then which of the following could be the possible length of summarized text

A. 256

Label added for presentation.

B. 16

Label added for presentation.

C. 1024

Label added for presentation.

D. 512

Label added for presentation.

Worked answer

Final answer: A, B with a fixed complete-sequence window; A–D with sliding and reset position IDs

Steps: The shared prompt gives T=1024 and 712 prompt tokens, leaving 312 slots under a complete-sequence cap. Only 256 and 16 fit. Sliding can generate all listed lengths; the exam key implies the fixed-window reading.

Status: ambiguous

Ambiguity note: Implied course/exam convention: A, B with a fixed complete-sequence window; A–D with sliding and reset position IDs. The shared prompt gives T=1024 and 712 prompt tokens, leaving 312 slots under a complete-sequence cap. Only 256 and 16 fit. Sliding can generate all listed lengths; the exam key implies the fixed-window reading.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2024 T3 End Term AN, Q177

Paper: 2024T3_ET_AN · Type: MCQ · Marks: 3 · Topic(s): w3-pretraining-objective

Assume that we have a large corpus of text. The vocabulary constructed from the text contains
10000 words. Of these, 100 words occurred only once in the entire corpus of text. The parameters
of the embedding layer and the output layer of the model are shared. Suppose we create a batch
of 256 samples (each sample is a sentence from the corpus). None of these samples contains any
of the 100 rare words. Suppose we pre-train the model for one iteration using the batch of
samples, then:

A. it is certain that the embeddings of none of these 100 rare words will get
updated.

B. there is a chance that the embeddings of all or some of these 100 rare words
will get updated

C. the embeddings of all these 100 rare words will defintely get updated

D. None of these

Worked answer

Final answer: B: rare embeddings can be updated.

Steps: Tied output weights receive softmax gradients even for absent targets. A is false; C is not guaranteed since gradients may cancel or learning rates may be zero; D ignores this mechanism.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2024 T3 End Term AN, Q193

Paper: 2024T3_ET_AN · Type: MSQ · Marks: 2 · Topic(s): w3-gpt1-architecture

Which of the following are correct about GPT-1 model:

A. It is a decoder only model.

B. It uses absolute positional encoding.

C. It uses a composite dataset for pre-training.

D. None of these.

Worked answer

Final answer: A and B.

Steps: GPT-1 uses a causal decoder and learned absolute positions. C is false: Radford et al. pretrained on BooksCorpus, rather than the later composite WebText-style mixtures; D is false.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2024 T3 End Term FN, Q194

Paper: 2024T3_ET_FN · Type: MSQ · Marks: 3 · Topic(s): w3-finetuning, w3-pretraining-objective

Choose all the correct statements about the GPT model:

A. All the parameters of are randomly initialized during pre-training

Label added for presentation.

B. The model minimizes the CLM objective during fine-tuning for downstream tasks like sentiment classification to improve the performance

Label added for presentation.

C. All parameters are randomly initialized during fine-tuning

Label added for presentation.

D. In general, fine-tuning requires a dataset with labels

Label added for presentation.

E. None of these

Label added for presentation.

Worked answer

Final answer: A, B, D

Steps: In the GPT-1 setting, random initialization applies at pretraining, downstream fine-tuning reuses pretrained weights and typically uses labels, and the original supervised-task formulation includes an auxiliary CLM loss. The qualifier “for downstream tasks like sentiment classification” supplies that context; CLM is not a universal requirement for every adaptation.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 End Term AN, Q23

Paper: 2025T3_ET_AN · Type: MCQ · Marks: 1 · Topic(s): w3-causal-lm

Which of the following describes the fundamental training mechanism of this model?

A. The model predicts the entire sequence \(x\) simultaneously without considering dependencies.

B. The model predicts the token \(x_{t+1}\) based only on the tokens \(x_1\) up to \(x_t\).

C. The model predicts a masked token \(x_t\) based on both the left and right context tokens.

Worked answer

Final answer: B.

Steps: Autoregressive training models P(x[t+1]|x[1:t]). A denies dependencies; C describes bidirectional masked-token prediction.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 End Term AN, Q24

Paper: 2025T3_ET_AN · Type: MCQ · Marks: 1 · Topic(s): w3-pretraining-objective

During the training phase on a sequence \(x\) of length \(T\), how does the teacher forcing mechanism specifically relate to the calculation of the loss function \(\mathcal{L}(\theta)\)?

A. It means the model uses its own previous predictions \((\hat{x}_1, \dots, \hat{x}_t)\) as input to predict \(x_{t+1}\).

B. It dictates that the loss is only calculated for tokens that were incorrectly
predicted.

C. It ensures that for every prediction \(P(x_{t+1} \mid \dots)\), the model is conditioned on the ground-truth sequence \((x_1, \dots, x_t)\).

D. It allows the model to see future tokens \((x_{t+2}, \dots, x_T)\) when predicting \(x_{t+1}\).

Worked answer

Final answer: C.

Steps: Teacher forcing conditions each prediction on the observed prefix. A instead feeds generated tokens; B ignores correctly predicted tokens; D violates causal masking.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 End Term FN, Q23

Paper: 2025T3_ET_FN · Type: MSQ · Marks: 3 · Topic(s): w3-causal-lm

A GPT-style causal language model is trained using the next-token prediction objective:
$\(\mathcal{L} = -\sum_{t=1}^T \log p_\theta(x_t \mid x_{<t})\)$
During inference, the model must generate tokens autoregressively from left to right using only past context.
Consider the following statements about GPT-style causal models.

A. GPT cannot condition on future tokens during training because the causal mask zeros out all attention to positions \(> t\)

B. If two prefixes \(x_{<t}\) and \(x'_{<t}\) have identical embeddings at all positions, GPT must assign identical next-token distributions for both contexts

C. Removing positional encodings would force GPT to treat all permutations of the same set of tokens as equivalent contexts

D. Causal masking ensures that the computational cost of training scales linearly with sequence length

Worked answer

Final answer: A and B.

Steps: A: the mask blocks future keys; B: identical position-wise input embeddings and deterministic computation give identical distributions. C is false: the triangular mask itself conveys order through prefix structure across layers. D is false: dense attention remains quadratic.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.