Week 2: Exam Questions and Worked Answers

Shared context

Question ID: 640653739756

Type: Comprehension
PDF pages: 3, 4

Consider a vocabulary \(V = (A, T, C, G)\) and the corresponding embedding matrix \(E = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 1 & -1 \\ 1 & 1 \end{bmatrix}\). For example, the 0-th row of \(E\) is the embedding for 0-th token of \(V\) and so on.

The following sequence \(X = (G, C, C, T, A, G)\) is to be processed by the encoder layer of the transformer architecture by appropriately adding the positional information using the following encoding scheme:

Assume the index for the position starts from zero. Add positional information to all the tokens in the sequence. Let the resultant sequence be denoted by \(H = E(X) + P\), (\(H\) will be used in the subsequent questions, please write it down). Here, \(E(X)\) denotes the embedding for each token in \(X\).

!Vocabulary V, embedding matrix E, sequence X, and positional encoding scheme equations for transformer encoder

Figure 2. Vocabulary V, embedding matrix E, sequence X, and positional encoding scheme equations for transformer encoder

Based on the above data, answer the given subquestions.

Question 184

Corrected shared-context text: Vocabulary V=(A,T,C,G), embeddings E=[[1,0],[0,1],[1,-1],[1,1]], and X=(G,C,C,T,A,G). Position indices start at zero; H=E(X)+P. The scan defines P(pos,2i)=sin(pos/10^(2i/d_model)) and P(pos,2i+1)=cos(pos/10^(2i/d_model)).

Dependent question

2024 T1 Quiz 1 NA, Q184

Paper: 2024T1_Q1_NA · Type: SA · Marks: 3 · Topic(s): w2-positional-encoding

What is the sum of the elements in the embedding vector \(P_5\)?

Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2024T1_Q1_NA/q184-q184-stem-01-p004.png asks for P_5..
Editorial correction: the extracted text differed from the scan (shared_context); see evidence LLM/sources/exam-papers/assets/da5004_2024T1_Q1_NA/ctx-184-187-passage-01-p004.png shows vocabulary, embedding matrix, sequence, positional formula and zero indexing..

Worked answer

Final answer: −0.675262 approximately.

Steps: The original image asks for P_5, despite the inventory transcription P_3. With dimension 2, P_5 = [sin(5), cos(5)], whose sum is sin(5)+cos(5).

Status: derived

verified_by_code: false — depends on figure, no code path

Shared context

Question ID: 640653739756

Type: Comprehension
PDF pages: 3, 4

Consider a vocabulary \(V = (A, T, C, G)\) and the corresponding embedding matrix \(E = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 1 & -1 \\ 1 & 1 \end{bmatrix}\). For example, the 0-th row of \(E\) is the embedding for 0-th token of \(V\) and so on.

The following sequence \(X = (G, C, C, T, A, G)\) is to be processed by the encoder layer of the transformer architecture by appropriately adding the positional information using the following encoding scheme:

Assume the index for the position starts from zero. Add positional information to all the tokens in the sequence. Let the resultant sequence be denoted by \(H = E(X) + P\), (\(H\) will be used in the subsequent questions, please write it down). Here, \(E(X)\) denotes the embedding for each token in \(X\).

!Vocabulary V, embedding matrix E, sequence X, and positional encoding scheme equations for transformer encoder

Figure 2. Vocabulary V, embedding matrix E, sequence X, and positional encoding scheme equations for transformer encoder

Based on the above data, answer the given subquestions.

Question 184

Corrected shared-context text: Vocabulary V=(A,T,C,G), embeddings E=[[1,0],[0,1],[1,-1],[1,1]], and X=(G,C,C,T,A,G). Position indices start at zero; H=E(X)+P. The scan defines P(pos,2i)=sin(pos/10^(2i/d_model)) and P(pos,2i+1)=cos(pos/10^(2i/d_model)).

Dependent question

2024 T1 Quiz 1 NA, Q185

Paper: 2024T1_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-positional-encoding

What is the sum of the elements in the 3-rd token, \(h_3 \in H\)

Editorial correction: the extracted text differed from the scan (shared_context); see evidence LLM/sources/exam-papers/assets/da5004_2024T1_Q1_NA/ctx-184-187-passage-01-p004.png shows vocabulary, embedding matrix, sequence, positional formula and zero indexing..

Worked answer

Final answer: 0.151128 approximately.

Steps: Zero-based position 3 is T, whose embedding is [0, 1]; add [sin(3), cos(3)]. The components sum to 1+sin(3)+cos(3).

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T1 Quiz 1 NA, Q190

Paper: 2024T1_Q1_NA · Type: MCQ · Marks: 4 · Topic(s): w2-cross-attention

Let \(R_E \in \mathbb{R}^{T_1 \times d_{model}}\) be the representation from the topmost encoder layer of the encoder and \(Z_D \in \mathbb{R}^{T_2 \times d_{model}}\) be the output of Multi-head Masked Attention layer of one of the decoder layers. What will be the size of the attention matrix in the Multi-head Cross Attention layer of the same decoder layer? Assume \(d_q = d_k = d_v = d\) if required.

A. \(T_1 \times T_2\)

Label added for presentation.

B. \(T_2 \times T_2\)

Label added for presentation.

C. \(T_1 \times T_1\)

Label added for presentation.

D. \(T_2 \times T_1\)

Label added for presentation.

Worked answer

Final answer: D: T₂ × T₁.

Steps: Decoder queries have T₂ rows and encoder keys T₁ rows, hence QKᵀ has shape [T₂, T₁] per head. A reverses query/key axes; B and C are self-attention sizes.

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T1 Quiz 1 NA, Q192

Paper: 2024T1_Q1_NA · Type: MSQ · Marks: 3 · Topic(s): w2-teacher-forcing

Suppose we have a dataset for machine translation tasks with thousands of samples. Suppose a team considers training the transformer model. The model could be trained in two approaches

A Autoregressive training
B Teacher forcing

Choose the correct statements

A. Approach A helps the model to converge faster than approach B

Label added for presentation.

B. Approach B helps the model converge faster than approach A

Label added for presentation.

C. One can start the training with approach B first and then switch to approach A after some training steps

Label added for presentation.

D. Once the training starts with approach B and then switching to approach A after some training steps can not be done

Label added for presentation.

Worked answer

Final answer: C is defensible; B is the intended empirical claim but not guaranteed

Steps: Teacher forcing supplies gold previous tokens; free-running training uses model outputs. Either can be scheduled after the other (C). Faster convergence is an empirical tendency, not a theorem for every dataset, so B is unproved and D false.

Status: ambiguous

Ambiguity note: Autoregressive factorization is compatible with teacher forcing, so the two named approaches are not mutually defined. A switch to generated-prefix training is possible (C), whereas an unconditional faster-convergence claim (B) depends on setup and convergence metric; the item does not specify these.

verified_by_code: false — No numeric checker path; answer remains explicitly ambiguous.

2024 T1 Quiz 1 NA, Q196

Paper: 2024T1_Q1_NA · Type: MCQ · Marks: 4 · Topic(s): w2-masked-self-attention

The attention score matrix given in Table 1 is appropriate for the causal language modelling task

A. True

Label added for presentation.

B. False

Label added for presentation.

C. Insufficient information

Label added for presentation.

Worked answer

Final answer: A: True.

Steps: The source table is lower triangular with nonnegative rows summing to 1, so no future key receives weight. B incorrectly claims incompatibility; C overlooks the supplied matrix. Exact zeros for allowed entries can arise as rounded values.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2024 T1 Quiz 1 NA, Q197

Paper: 2024T1_Q1_NA · Type: SA · Marks: 4 · Topic(s): w2-masked-self-attention

Assume the time step starts from t = 0 and ends at t = 6. Suppose the model is at time step t = 4, what is the attention value assigned for the word “is”?

Worked answer

Final answer: 0.

Steps: The sequence indexes [BOS], This, is, a, cool, idea, [EOS] from 0; the source table entry at row 4, column 2 is 0.

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T3 Quiz 1 NA, Q275

Paper: 2024T3_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w2-teacher-forcing

Teacher forcing for training an transformer model is:

A. mandatory.

Label added for presentation.

B. optional.

Label added for presentation.

Worked answer

Final answer: B: optional.

Steps: Teacher forcing is a training choice; generated-prefix training is possible. A incorrectly makes it mandatory.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2024 T3 Quiz 1 NA, Q276

Paper: 2024T3_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w2-residual-layernorm, w2-positional-encoding

Choose the correct statements regarding transformer architecture:

A. Residual connections are practically optional in a transformer model with 10 encoder layers and 10 decoder layers.

Label added for presentation.

B. Having multiple heads helps in capturing different relationships between input tokens.

Label added for presentation.

C. One hot encoding is a good choice for position encoding.

Label added for presentation.

D. Batch normalization can be used instead of layer normalization.

Label added for presentation.

E. None of these.

Label added for presentation.

Worked answer

Final answer: B and D

Steps: Multiple heads can capture different relationships (B); Transformer variants can use batch normalization (D). Both are defensible literal claims, although layer normalization is standard, and only one choice is allowed.

Status: ambiguous

Ambiguity note: Multiple heads can capture different relationships (B); Transformer variants can use batch normalization (D). Both are defensible literal claims, although layer normalization is standard, and only one choice is allowed.

verified_by_code: false — No numeric checker path; answer remains explicitly ambiguous.

2024 T3 Quiz 1 NA, Q277

Paper: 2024T3_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-decoder-output-and-params

Assume the model has two layers (N = 2). Calculate the total number of parameters in the model (excluding the embedding layer and output layer). Moreover, no bias was added to the neuron in the FFNN layers.

Worked answer

Final answer: 230,656 with bias-free attention and affine LayerNorm; 232,192 with attention biases

Steps: The shared passage explicitly makes encoder and decoder configurations equal, so N=2 applies to both. Six attention blocks contribute 6×4×64²=98,304, four bias-free FFNs 4×2×64×256=131,072, and ten affine norms 1,280, total 230,656. Attention biases add 6×4×64=1,536. The printed answer favors bias-free attention; the old note incorrectly suggests independent stack depths are unspecified.

Status: ambiguous

Ambiguity note: Implied course/exam convention: 230,656 with bias-free attention and affine LayerNorm; 232,192 with attention biases. The shared passage explicitly makes encoder and decoder configurations equal, so N=2 applies to both. Six attention blocks contribute 6×4×64²=98,304, four bias-free FFNs 4×2×64×256=131,072, and ten affine norms 1,280, total 230,656. Attention biases add 6×4×64=1,536. The printed answer favors bias-free attention; the old note incorrectly suggests independent stack depths are unspecified.

verified_by_code: false — Override numeric derivation not code-verified: complete numeric inputs are absent from the editable extraction; manual scan adjudication retained.

2024 T3 Quiz 1 NA, Q278

Paper: 2024T3_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-decoder-output-and-params

How many parameters does the output layer have?

Worked answer

Final answer: 48,000 weights; 48,750 including an output bias

Steps: A 64-to-750 output projection contains 64×750=48,000 weights; an output bias contributes 750. Whether tied output weights are counted as independent parameters is another convention. The paper key implies a weights-only count.

Status: ambiguous

Ambiguity note: Implied course/exam convention: 48,000 weights; 48,750 including an output bias. A 64-to-750 output projection contains 64×750=48,000 weights; an output bias contributes 750. Whether tied output weights are counted as independent parameters is another convention. The paper key implies a weights-only count.

verified_by_code: false — Override numeric derivation not code-verified: complete numeric inputs are absent from the editable extraction; manual scan adjudication retained.

2024 T3 Quiz 1 NA, Q279

Paper: 2024T3_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-decoder-output-and-params

How many parameters does the embedding layer have?

Worked answer

Final answer: 32,000 source embedding weights; 48,000 target embedding weights; 80,000 for both

Steps: The shared encoder–decoder has source vocabulary 500, target vocabulary 750, and width 64, yet asks for “the embedding layer” without identifying source, target, or both. Their untied counts are 500×64, 750×64, and their sum; sharing would need further specification.

Status: ambiguous

Ambiguity note: The shared encoder–decoder has source vocabulary 500, target vocabulary 750, and width 64, yet asks for “the embedding layer” without identifying source, target, or both. Their untied counts are 500×64, 750×64, and their sum; sharing would need further specification.

verified_by_code: false — Override numeric derivation not code-verified: complete numeric inputs are absent from the editable extraction; manual scan adjudication retained.

2024 T3 Quiz 1 NA, Q280

Paper: 2024T3_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-decoder-output-and-params

At time step t = 1, the prediction probability for the words “Naan”, “transformer” and “padaththai” are 0.55,0.15 and 0.2, respectively. What is the probability for the word “rasithen”? If you think, the given information is insufficient enter -1

Worked answer

Final answer: −1: insufficient information.

Steps: The named three probabilities sum to 0.90. The remaining 0.10 is distributed among rasithen and the other target vocabulary entries, so its individual probability cannot be inferred.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

Shared context

Question ID: 640653993617

Type: Comprehension
PDF pages: 6

Consider following input matrix to a single neural network layer:

It has 3 samples with 3 features each i.e. batch size is 3.
Each column represents a sample. Left most column represents the first sample and topmost row represents the first feature vector.
Note: assume \(\gamma = 1, \beta = 0\) and \(\epsilon = 10^{-5}\) for both batch and layer normalization, if not mentioned explicitly. Use standard scaling.

!Input matrix X and batch/layer normalization configuration specifications

Figure 3. Input matrix X and batch/layer normalization configuration specifications

Based on the above data, answer the given subquestions.

Question 281

Corrected shared-context text: X=[[1,2,3],[4,7,11],[-3,5,-2]] has three samples as columns and three features as rows; gamma=1, beta=0, epsilon=10^-6, with standard scaling.

Dependent question

2024 T3 Quiz 1 NA, Q281

Paper: 2024T3_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-residual-layernorm

Say matrix F is computed by applying batch normalization on X, what will be sum of every element in first/topmost row of F?

Editorial correction: the extracted text differed from the scan (shared_context); see evidence LLM/sources/exam-papers/assets/da5004_2024T3_Q1_NA/ctx-281-284-passage-01-p006.png shows input matrix, sample orientation and epsilon = 10^-6..

Worked answer

Final answer: 0.

Steps: Batch normalization centers each feature across its sample row. Therefore the centered first row sums to zero, and division by its common standard deviation preserves the zero sum.

Status: derived

verified_by_code: false — depends on figure, no code path

Shared context

Question ID: 640653993617

Type: Comprehension
PDF pages: 6

Consider following input matrix to a single neural network layer:

It has 3 samples with 3 features each i.e. batch size is 3.
Each column represents a sample. Left most column represents the first sample and topmost row represents the first feature vector.
Note: assume \(\gamma = 1, \beta = 0\) and \(\epsilon = 10^{-5}\) for both batch and layer normalization, if not mentioned explicitly. Use standard scaling.

!Input matrix X and batch/layer normalization configuration specifications

Figure 3. Input matrix X and batch/layer normalization configuration specifications

Based on the above data, answer the given subquestions.

Question 281

Corrected shared-context text: X=[[1,2,3],[4,7,11],[-3,5,-2]] has three samples as columns and three features as rows; gamma=1, beta=0, epsilon=10^-6, with standard scaling.

Dependent question

2024 T3 Quiz 1 NA, Q282

Paper: 2024T3_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-residual-layernorm

Say matrix F is computed by applying layer normalization on X, what will be sum of every element in first/leftmost column of F?

Editorial correction: the extracted text differed from the scan (shared_context); see evidence LLM/sources/exam-papers/assets/da5004_2024T3_Q1_NA/ctx-281-284-passage-01-p006.png shows input matrix, sample orientation and epsilon = 10^-6..

Worked answer

Final answer: 0.

Steps: Layer normalization centers each sample across its feature column; the common denominator preserves the zero sum of its centered first column.

Status: derived

verified_by_code: false — depends on figure, no code path

Shared context

Question ID: 640653993617

Type: Comprehension
PDF pages: 6

Consider following input matrix to a single neural network layer:

It has 3 samples with 3 features each i.e. batch size is 3.
Each column represents a sample. Left most column represents the first sample and topmost row represents the first feature vector.
Note: assume \(\gamma = 1, \beta = 0\) and \(\epsilon = 10^{-5}\) for both batch and layer normalization, if not mentioned explicitly. Use standard scaling.

!Input matrix X and batch/layer normalization configuration specifications

Figure 3. Input matrix X and batch/layer normalization configuration specifications

Based on the above data, answer the given subquestions.

Question 281

Corrected shared-context text: X=[[1,2,3],[4,7,11],[-3,5,-2]] has three samples as columns and three features as rows; gamma=1, beta=0, epsilon=10^-6, with standard scaling.

Dependent question

2024 T3 Quiz 1 NA, Q283

Paper: 2024T3_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-residual-layernorm

Say \(\sigma_1, \sigma_2\) and \(\sigma_3\) were the standard deviations used to compute batch normalization of X. What is \(\sum_{i=1}^3 \sigma_i\)?

Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2024T3_Q1_NA/q283-q283-stem-01-p007.png shows the sum of σ_i, with no square..
Editorial correction: the extracted text differed from the scan (shared_context); see evidence LLM/sources/exam-papers/assets/da5004_2024T3_Q1_NA/ctx-281-284-passage-01-p006.png shows input matrix, sample orientation and epsilon = 10^-6..

Worked answer

Final answer: Approximately 7.243 using population standard deviations.

Steps: The image asks for the sum of standard deviations, not their squares. For the three feature rows, population variances are 2/3, 74/9 and 38/3; their square roots sum to 7.242964. Epsilon 10⁻⁶ affects normalization denominators negligibly.

Status: derived

verified_by_code: false — depends on figure, no code path

Shared context

Question ID: 640653993617

Type: Comprehension
PDF pages: 6

Consider following input matrix to a single neural network layer:

It has 3 samples with 3 features each i.e. batch size is 3.
Each column represents a sample. Left most column represents the first sample and topmost row represents the first feature vector.
Note: assume \(\gamma = 1, \beta = 0\) and \(\epsilon = 10^{-5}\) for both batch and layer normalization, if not mentioned explicitly. Use standard scaling.

!Input matrix X and batch/layer normalization configuration specifications

Figure 3. Input matrix X and batch/layer normalization configuration specifications

Based on the above data, answer the given subquestions.

Question 281

Corrected shared-context text: X=[[1,2,3],[4,7,11],[-3,5,-2]] has three samples as columns and three features as rows; gamma=1, beta=0, epsilon=10^-6, with standard scaling.

Dependent question

2024 T3 Quiz 1 NA, Q284

Paper: 2024T3_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-residual-layernorm

Say \(\sigma_1, \sigma_2\) and \(\sigma_3\) were the standard deviations used to compute layer normalization of X. What is \(\sum_{i=1}^3 \sigma_i\)?

Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2024T3_Q1_NA/q284-q284-stem-01-p007.png shows the sum of σ_i, with no square..
Editorial correction: the extracted text differed from the scan (shared_context); see evidence LLM/sources/exam-papers/assets/da5004_2024T3_Q1_NA/ctx-281-284-passage-01-p006.png shows input matrix, sample orientation and epsilon = 10^-6..

Worked answer

Final answer: Approximately 10.276 using population standard deviations.

Steps: The image asks for the sum of standard deviations, not their squares. Across the three sample columns the population variances are 74/9, 38/9 and 86/3, whose square roots sum to 10.276373.

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T3 Quiz 1 NA, Q291

Paper: 2024T3_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w2-masked-self-attention

What will be shape of the mask M?

A. 16 × 16

Label added for presentation.

B. 32 × 32

Label added for presentation.

C. 64 × 64

Label added for presentation.

D. 128 × 128

Label added for presentation.

E. None of these.

Label added for presentation.

Worked answer

Final answer: C: 64 × 64.

Steps: The causal mask has one row per query and one column per key; the sequence length is 64. A, B and D use incorrect sequence lengths; E is false since C fits.

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T3 Quiz 1 NA, Q293

Paper: 2024T3_Q1_NA · Type: SA · Marks: 3 · Topic(s): w2-positional-encoding

Consider the embedding vector for a word, \(x = [0.1, 0.2, -0.3, 0.4]\). Suppose the word is at position 2 in the given sentence. Add the corresponding position embedding \(p\) to the word embedding to get \(h\), i.e. the sum of the elements in \(h = x + p\). Use the fixed-sinusoidal position embedding vector calculated using the formula given below

What is \(h[0] + h[1]\) i.e. sum of first two elements of \(h\)?

2025 T1 Quiz 1 NA, Q104

Paper: 2025T1_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w2-cross-attention

Let \(H_e \in \mathbb{R}^{T_1 \times d_{model}}\) be the representation from the topmost encoder layer of the encoder and \(Z_d \in \mathbb{R}^{T_2 \times d_{model}}\) be the output of Multihead Masked Attention layer of one of the decoder layers. What will be the size of the attention matrix in the Multi-head Cross Attention layer of the same decoder layer? Assume \(d_k = d_q = d_v = d\) if required.

A. \(T_1 \times T_1\)

B. \(T_1 \times T_2\)

C. \(T_2 \times T_2\)

D. \(T_2 \times T_1\)

Worked answer

Final answer: D: T₂ × T₁.

Steps: Decoder query length is T₂; encoder key length is T₁. Therefore QKᵀ is [T₂, T₁] per head. A and C are square self-attention sizes; B reverses axes.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T1 Quiz 1 NA, Q105

Paper: 2025T1_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w2-teacher-forcing

Consider following assertion and reason pair:

Assertion: A transformer model cannot be trained in autogregressive mode for machine
translation task.
Reason: Teacher forcing improves the convergence rate.

Choose the correct statements

A. Assertion and Reason are both true and Reason is a correct explanation of
Assertion.

B. Assertion and Reason are both true and Reason is not a correct explanation of
Assertion.

C. Assertion is true but Reason is false.

D. Assertion is false but Reason is true.

Worked answer

Final answer: D

Steps: An autoregressive Transformer can be trained for translation, so the assertion is false. Teacher forcing generally accelerates practical training and convergence by supplying reference prefixes and enabling parallel positions; the reason is the standard qualitative course statement, not an all-datasets theorem. D is the unique intended pair.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T1 Quiz 1 NA, Q111

Paper: 2025T1_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-positional-encoding

Suppose the number of learnable parameters in the source input-embedding layer is 3200, how
many parameters are there in the positional embedding layer of the source language?

Worked answer

Final answer: 0 for fixed sinusoidal positions; 1,024 for a learned table

Steps: The source embedding condition gives d=3200/100=32 and the context has 32 positions. A learned position table contains 32×32=1,024 parameters; fixed sinusoids contain zero. The original Transformer used sinusoids, while the paper key implies a learned table; “vanilla” alone does not settle this conflict.

Status: ambiguous

Ambiguity note: Implied course/exam convention: 0 for fixed sinusoidal positions; 1,024 for a learned table. The source embedding condition gives d=3200/100=32 and the context has 32 positions. A learned position table contains 32×32=1,024 parameters; fixed sinusoids contain zero. The original Transformer used sinusoids, while the paper key implies a learned table; “vanilla” alone does not settle this conflict.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2025 T1 Quiz 1 NA, Q114

Paper: 2025T1_Q1_NA · Type: SA · Marks: 1 · Topic(s): w2-residual-layernorm

Say matrix F is computed by applying batch normalization on X, what will be sum of every element
in first / topmost row of F?

Worked answer

Final answer: 0.

Steps: Batch normalization centers each feature row over four samples; all entries have the same denominator and their sum is zero.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T1 Quiz 1 NA, Q115

Paper: 2025T1_Q1_NA · Type: SA · Marks: 1 · Topic(s): w2-residual-layernorm

Say matrix F is computed by applying layer normalization on X, what will be sum of every element
in first / leftmost column of F?

Worked answer

Final answer: 0.

Steps: Layer normalization centers the two features within the first sample; their normalized values sum to zero.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T1 Quiz 1 NA, Q116

Paper: 2025T1_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-residual-layernorm

Say \(\sigma_i\)'s were the standard deviations used to compute batch normalization of \(X\). What is \(\sum_{i=1}^2 \sigma_i\)?

Worked answer

Final answer: Approximately 6.50047.

Steps: Standard population variances of the two feature rows are 7.25 and 14.5; the sum of their standard deviations is √7.25+√14.5. The paper image specifies epsilon 10⁻⁶.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T1 Quiz 1 NA, Q117

Paper: 2025T1_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-residual-layernorm

Say \(\sigma_i\)'s were the standard deviations used to compute layer normalization of \(X\). What is \(\sum_{i=1}^4 \sigma_i\)?

Worked answer

Final answer: 11.

Steps: For each two-feature sample, population standard deviation is half the absolute feature difference: 1.5, 2.5, 4 and 3. Their sum is 11.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T2 Quiz 1 NA, Q139

Paper: 2025T2_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w2-residual-layernorm

Why are residual connections important in transformer architectures?

A. They reduce memory consumption.

B. They add extra cost of compute by adding batch normalization.

C. They help in training deep networks by enabling gradient flow.

D. They remove the need for layer normalization.

Worked answer

Final answer: C.

Steps: The identity path in a residual connection facilitates gradient propagation through deep networks. A is not its general effect; B confuses residuals with batch normalization; D is false because residuals do not eliminate the role of layer normalization.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T2 Quiz 1 NA, Q144

Paper: 2025T2_Q1_NA · Type: SA · Marks: 3 · Topic(s): w2-positional-encoding

Consider the embedding vector for a word, \(x = [0.5, -0.4, 0.3, -0.2]^T\). Suppose the word is at position 3 in the given sentence. Add the corresponding position embedding \(p\) to the word embedding to get \(h\), i.e. the sum of the elements in \(h = x + p\). Use the fixed sinusoidal position embedding vector calculated using the formula given below:

What is the value of \(h[0] - h[1]\) i.e. substract second element from the first element of \(h\) vector ? (Answer correct up to 2 values after the decimal)

Worked answer

Final answer: 2.03.

Steps: At position 3, p₀=sin(3), p₁=cos(3), so h₀−h₁=(0.5−(−0.4))+sin(3)−cos(3)=2.0311125, rounded to two decimals.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T2 Quiz 1 NA, Q150

Paper: 2025T2_Q1_NA · Type: SA · Marks: 1 · Topic(s): w2-residual-layernorm

For the given input matrix X and multihead attention output MHA(X), apply a residual connection and store the result in matrix R, and finally compute the sum of all elements in matrix R.

Worked answer

Final answer: 32

Steps: The pictured shared passage gives X=[[1,1],[2,3],[1,2],[3,1]] and MHA(X)=[[1,2],[3,3],[2,3],[3,1]]. Their sum is [[2,3],[5,6],[3,5],[6,2]], whose entries total 32. A transcription or answer-key discrepancy does not make the legible original image ambiguous.

Status: derived

verified_by_code: false — Override numeric derivation not code-verified: complete numeric inputs are absent from the editable extraction; manual scan adjudication retained.

2025 T2 Quiz 1 NA, Q151

Paper: 2025T2_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-residual-layernorm

To apply the layer normalization on the matrix R. How many learning parameters (\(\theta\)) will be required ?

Worked answer

Final answer: 8 trainable parameters

Steps: The passage states d_model=4, and its 4 × 2 image represents two tokens with four features each. LayerNorm has one gain and one bias per feature, shared across tokens: 4+4=8.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T2 Quiz 1 NA, Q152

Paper: 2025T2_Q1_NA · Type: SA · Marks: 3 · Topic(s): w2-residual-layernorm

Initialize \(\gamma = 2, \beta = 1\) and use \(\epsilon = 0\) and apply the layer normalization on the matrix R and sum of all the elements of the final output.

Worked answer

Final answer: 8

Steps: The image specifies gamma=2σ_l, beta=1, epsilon=0, not gamma=2. For either token column, centering makes the four normalized features sum to zero; multiplying all by the same gamma for that column preserves this. Adding 1 to each of 8 entries yields sum 8.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T3 Quiz 1 NA, Q174

Paper: 2025T3_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w2-positional-encoding

Which of the following is true regarding sinusoidal encoding?

A. The encoding vector of words present at even positions in the given text sequence uses sine, and the encoding vector of words present at odd positions uses cosine.

Label added for presentation.

B. The encoding vector of words present at odd positions in the given text sequence uses sine, and the encoding vector of words present at even positions uses cosine.

Label added for presentation.

C. The encoding vector for each word position contains values computed using both sine (for even dimensions) and cosine (for odd dimensions).

Label added for presentation.

Worked answer

Final answer: C — each position uses sine in even dimensions and cosine in odd dimensions

Steps: The original encoding alternates the trigonometric function by dimension, not by token-position parity. Therefore A and B incorrectly assign a single function by word position.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 Quiz 1 NA, Q175

Paper: 2025T3_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w2-teacher-forcing

We have a transformer model translating English to French. Suppose the target sequence is:

Target (French): ['je', 'suis', 'heureux']

Suppose the model predicted at time step \(t\):
\(t = 1\): 'je'
\(t = 2\): 'content'
\(t = 3\): 'heureux'

If teacher forcing is used, what will be the decoder input at time step 3?

A. ‘content’

Label added for presentation.

B. ‘je’

Label added for presentation.

C. ‘suis’

Label added for presentation.

D. ‘heureux’

Label added for presentation.

Worked answer

Final answer: C — suis

Steps: At step 3 teacher forcing supplies the ground-truth token at step 2, suis. A is the wrong model prediction at step 2; B is the step-1 target; D is the step-3 target rather than the preceding input.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 Quiz 1 NA, Q176

Paper: 2025T3_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w2-masked-self-attention

We have a decoder input of length 4 tokens: [x1, x2, x3, x4].
We use masked self-attention with the following pre-softmax attention scores:

For token x3, the pre-softmax row is [1, 2, 0, 2]. After masking, what does it become?

A. [1, 2, 0, 2]

Label added for presentation.

B. [1, 2, 0, 0]

Label added for presentation.

C. [1, 2, 0, -\infty]

Label added for presentation.

D. [-\infty, 2, 0, 2]

Label added for presentation.

E. [-\infty, -\infty, -\infty, 2]

Label added for presentation.

Worked answer

Final answer: C — [1, 2, 0, −∞]

Steps: Query x3 can attend to positions 1, 2 and 3; future x4 receives −∞ before softmax. A leaves the future visible; B sets its logit to 0, which still has positive softmax weight; D masks a past token; E masks the allowed prefix and retains the future.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 Quiz 1 NA, Q177

Paper: 2025T3_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w2-teacher-forcing

In the context of language model training, teacher forcing is a technique where:

A. The correct target tokens from the training data are used as input for the next time step, regardless of the model’s prediction.

Label added for presentation.

B. The model’s predictions are used as the input for the next time step, even if they are incorrect.

Label added for presentation.

C. The model is forced to learn without any pre-trained weights.

Label added for presentation.

D. The model is forced to correct the input data.

Label added for presentation.

Worked answer

Final answer: A — feed the correct preceding target token

Steps: Teacher forcing conditions on the reference prefix, independent of predicted tokens. B instead feeds predictions; C concerns initialization; D is not the procedure.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 Quiz 1 NA, Q179

Paper: 2025T3_Q1_NA · Type: MCQ · Marks: 3 · Topic(s): w2-residual-layernorm

You have a mini-batch of 2 samples, each with 2 features:

• Rows represent samples
• Columns represent features

If Batch Normalization is applied across the batch for each feature (assume \(\gamma = 1, \beta = 0\) and \(\epsilon = 0\)), what is the resulting normalized mini-batch?

A. \(\begin{bmatrix} -1 & -1 \\ 1 & 1 \end{bmatrix}\)

Label added for presentation.

B. \(\begin{bmatrix} -0.707 & -0.707 \\ 0.707 & 0.707 \end{bmatrix}\)

Label added for presentation.

C. \(\begin{bmatrix} -0.707 & 0.707 \\ -0.707 & 0.707 \end{bmatrix}\)

Label added for presentation.

D. \(\begin{bmatrix} 1 & 1 \\ -1 & -1 \end{bmatrix}\)

Label added for presentation.

Worked answer

Final answer: A — [[−1, −1], [1, 1]]

Steps: Per-feature batch means are [2,3] and population variances are [1,1], so standardizing each column gives A. B uses a sample-variance convention instead of the standard BN batch variance; C normalizes along the wrong axis and also uses that convention; D reverses the signs.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T3 Quiz 1 NA, Q184

Paper: 2025T3_Q1_NA · Type: MSQ · Marks: 4 · Topic(s): w2-full-transformer

Which of the following statements about the Transformer architecture are true?

A. Masking (look-ahead mask) is applied in every decoder self-attention layer, not just the first one.

Label added for presentation.

B. Token embeddings are used in the encoder but not in the decoder.

Label added for presentation.

C. Positional encoding is required only in the encoder, not in the decoder.

Label added for presentation.

D. Token embeddings are used in both the encoder and the decoder.

Label added for presentation.

E. Positional encodings are added to the input embeddings in both the encoder and the decoder.

Label added for presentation.

Worked answer

Final answer: A, D and E

Steps: Every decoder self-attention layer is causally masked (A); original encoder and decoder both use token embeddings (D) and positional encodings (E). B denies decoder embeddings, while C denies decoder positional encoding.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 Quiz 1 NA, Q190

Paper: 2025T3_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-masked-self-attention

For one sequence, how many non-zero attention scores remain in the masked attention score matrix (per head)?

Worked answer

Final answer: 524,800 unmasked or finite-softmax positive entries; raw nonzero logits undetermined

Steps: The inclusive causal triangle contains 1024×1025/2=524,800 allowed positions. With finite logits, post-softmax allowed weights are positive and masked weights zero; pre-softmax raw allowed scores can be zero and additive-masked scores are −∞. The course convention counts available causal weights, but “score matrix” does not specify the stage.

Status: ambiguous

Ambiguity note: Implied course/exam convention: 524,800 unmasked or finite-softmax positive entries; raw nonzero logits undetermined. The inclusive causal triangle contains 1024×1025/2=524,800 allowed positions. With finite logits, post-softmax allowed weights are positive and masked weights zero; pre-softmax raw allowed scores can be zero and additive-masked scores are −∞. The course convention counts available causal weights, but “score matrix” does not specify the stage.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T1 Quiz 1 NA, Q5

Paper: 2026T1_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w2-cross-attention

In a standard Transformer encoder-decoder architecture, is masking typically applied in the cross-attention layer?

A. Yes, both causal masking and padding masking are applied to prevent the decoder from attending to future encoder positions

B. Yes, only causal masking is applied to maintain the autoregressive property

C. No causal masking is needed, but padding masking may be applied to ignore padded positions in the source sequence

D. No masking is ever applied in cross-attention since the encoder processes the complete input sequence

Worked answer

Final answer: C — no source causal mask; source padding may be masked

Steps: The complete source is available to cross-attention, but padding keys can be excluded. A and B incorrectly impose target-style causality on source positions; D ignores source padding masks.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T1 Quiz 1 NA, Q11

Paper: 2026T1_Q1_NA · Type: MSQ · Marks: 2 · Topic(s): w2-cross-attention

In the context of the Transformer architecture, which of the following statements correctly describe the source of the Query (\(Q\)), Key (\(K\)), and Value (\(V\)) vectors during the Cross-Attention (Encoder-Decoder Attention) mechanism?

A. The Keys (\(K\)) and Values (\(V\)) are derived from the output of the Decoder.

B. The Queries (\(Q\)) are generated from the previous hidden states of the Decoder.

C. The Queries (\(Q\)), Keys (\(K\)), and Values (\(V\)) all originate from the same input sequence.

D. This mechanism allows the Decoder to focus on relevant parts of the input sequence processed by the Encoder.

Worked answer

Final answer: B and D

Steps: Decoder hidden states provide Q (B), and decoder queries use encoder keys/values to select source information (D). A incorrectly sources K and V from the decoder; C describes self-attention rather than encoder–decoder attention.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T1 Quiz 1 NA, Q12

Paper: 2026T1_Q1_NA · Type: SA · Marks: 2 · Topic(s): w2-positional-encoding

Consider the positional encoding for a given position \(pos\) and dimension \(i\) is defined by:

Consider a Transformer model with a hidden state dimension \(d_{model} = 128\). You are calculating the positional embedding for the 4th word in a sequence (index \(pos = 3\), using 0-based indexing). Calculate the value of the 6th element (index \(i = 5\)) of the positional embedding vector for this word. (use radians for angle)

Worked answer

Final answer: cos(3/10000^(4/128)) ≈ −0.62792665

Steps: The explicitly requested sixth element is zero-based coordinate 5=2×2+1; the cosine formula therefore uses pair index 2. The reuse of i for coordinate and pair index is a notation slip, but “sixth element” fixes the requested coordinate unambiguously.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T2 Quiz 1 NA, Q7

Paper: 2026T2_Q1_NA · Type: SA · Marks: 4 · Topic(s): w2-masked-self-attention

Consider a masked multi-head attention layer in a transformer decoder. The source vocabulary is of size \(V_s = 1000\), \(d_{model} = 64\) and context length \(T = 128\). To enforce autoregressive generation, a causal mask is applied to the raw attention score before applying softmax. How many entries in the causal mask are set to \(-\infty\) ?

Worked answer

Final answer: 8,128 entries per 128 × 128 causal mask

Steps: Only the strict upper triangle is masked, giving T(T−1)/2=128×127/2=8128; vocabulary size and hidden width do not change this count. A shared mask is counted once, not once per head.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T2 Quiz 1 NA, Q8

Paper: 2026T2_Q1_NA · Type: SA · Marks: 4 · Topic(s): w2-positional-encoding

Consider a Transformer model with a hidden state dimension \(d_{model} = 512\). The positional encoding for a given position, \(pos\) and dimension \(i\) is defined by:

Compute the squared euclidean norm for the vector corresponding to \(pos = 3\) and enter the value.

Worked answer

Final answer: 256

Steps: There are 512/2=256 sine–cosine pairs, each with squared norm sin²θ+cos²θ=1. Their total squared Euclidean norm is 256, independent of position.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T2 Quiz 1 NA, Q9

Paper: 2026T2_Q1_NA · Type: MCQ · Marks: 4 · Topic(s): w2-teacher-forcing

Consider the following statements regarding the use of Teacher Forcing while training
autoregressive sequence-to-sequence models and select the appropriate option:

Statement 1: Teacher Forcing involves passing the input token to a larger Teacher model whose
output is then input as the next token

Statement 2: Teacher Forcing helps to prevent compounding of early incorrect predictions that
can destabilize training

A. Statement 1 is True but Statement 2 is False

B. Statement 1 is False but Statement 2 is True

C. Both the statements are True

D. Both the statements are False

Worked answer

Final answer: B — statement 1 false; statement 2 true

Steps: Teacher forcing feeds preceding ground-truth target tokens, not output from a larger teacher model. It prevents feeding a wrong prediction into the next training step, reducing within-prefix compounding. Thus A, C and D all misclassify at least one statement.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 Quiz 2 NA, Q113

Paper: 2025T3_Q2_NA · Type: SA · Marks: 2 · Topic(s): w2-masked-self-attention

In a 4-token sequence, a Transformer is calculating the attention for the third Token. Its Query vector (\(q_3\)) must be compared against the Key vectors (\(k_1, k_2, k_3, k_4\)) of all tokens. The Causal LM mask is applied to the raw scores before the softmax function. The raw dot-product scores are:

• \(q_3 \cdot k_1 = 1.8\)
• \(q_3 \cdot k_2 = 0.5\)
• \(q_3 \cdot k_3 = 2.0\)
• \(q_3 \cdot k_4 = 1.2\)

After the Causal LM mask and the subsequent softmax function are applied, what will be the final attention probability assigned to Token 4?

Worked answer

Final answer: 0

Steps: Token 4 is future relative to query 3. The causal mask changes its logit to −∞ and exp(−∞)=0, so the softmax probability is exactly 0 regardless of the other finite scores.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T3 Quiz 2 NA, Q127

Paper: 2025T3_Q2_NA · Type: MCQ · Marks: 2 · Topic(s): w2-residual-layernorm

For a layer with 512 features, how many learnable parameters do BatchNorm and LayerNorm have?

A. BatchNorm: 512, LayerNorm: 512

B. BatchNorm: 1024, LayerNorm: 512

C. BatchNorm: 512, LayerNorm: 1024

D. BatchNorm: 1024, LayerNorm: 1024

Worked answer

Final answer: D — BatchNorm 1,024 and LayerNorm 1,024

Steps: Each standard affine normalization has 512 learned scale and 512 learned shift values, totaling 1024. A omits one set for each; B omits LayerNorm bias or scale; C omits the corresponding BatchNorm set.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2025 T3 Quiz 2 NA, Q128

Paper: 2025T3_Q2_NA · Type: MCQ · Marks: 2 · Topic(s): w2-cross-attention

In a standard Encoder-Decoder Transformer, where do the Query \(Q\), Key \(K\), and Value \(V\) inputs for the Decoder's Cross-Attention sub-layer come from?

A. \(Q\) comes from the decoder's previous sub-layer; \(K\) and \(V\) come from the final hidden states of the encoder.

B. \(Q\) comes from the final hidden states of the encoder; \(K\) and \(V\) come from the decoder's previous sub-layer.

Worked answer

Final answer: A — Q from decoder, K and V from encoder

Steps: The decoder sublayer produces queries, while the final encoder representations supply keys and values. B reverses these origins.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T1 Quiz 2 NA, Q12

Paper: 2026T1_Q2_NA · Type: SA · Marks: 2 · Topic(s): w2-residual-layernorm

Let \(x_{ij}\) denote the activation of the \(j\)-th neuron for the \(i\)-th training sample.

In Layer Normalization, normalization is performed across features for each individual sample.

Let \(H\) be the number of neurons (features) in the hidden layer.

For a given sample, the layer normalization statistics are computed as:
$\(\mu_i = \frac{1}{H} \sum_{j=1}^H x_{ij}\)$
$\(\sigma_i^2 = \frac{1}{H} \sum_{j=1}^H (x_{ij} - \mu_i)^2\)$
$\(\hat{x}_{ij} = \frac{x_{ij} - \mu_i}{\sqrt{\sigma_i^2 + \epsilon}}\)$
where \(\epsilon\) is a small constant for numerical stability.

Layer Normalization is applied independently to each sample in the given data above.

For Sample 2, after applying Layer Normalization, what is the maximum value in the normalized output vector?

(ignore the learnable parameters \(\gamma\) and \(\beta\), and assume \(\epsilon = 0\)):

Worked answer

Final answer: 3/√5 ≈ 1.341641

Steps: Sample 2 is [1,3,5,7], with mean 4 and population variance (9+1+1+9)/4=5. Its largest centered value is 3, so its maximum normalized value is 3/√5. The zero variance of sample 3 is irrelevant to this question.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T2 Quiz 2 NA, Q2

Paper: 2026T2_Q2_NA · Type: SA · Marks: 2 · Topic(s): w2-residual-layernorm

In the given Transformer architecture for one Encoder Decoder block, identify every Add & Norm layer. Submit the corresponding component alphabets in alphabetical (ascending) order as a single uppercase string without spaces, commas, or quotation marks.

Example: If the correct components are Q, G, and S, your answer should be GQS

Worked answer

Final answer: FHMPR

Steps: In the diagram, F and H are the two encoder residual-plus-normalization blocks; M, P and R follow the decoder masked-attention, cross-attention and feed-forward blocks respectively. Sorting gives FHMPR.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T2 Quiz 2 NA, Q3

Paper: 2026T2_Q2_NA · Type: MCQ · Marks: 1 · Topic(s): w2-masked-self-attention

Identify the function of component "L" in the given Transformer architecture.

A. Positional Encoding

B. Feed-Forward Network

C. Masked Multi-Head Self-Attention

D. Multi-Head Cross-Attention

E. Linear Output Projection

Worked answer

Final answer: C — masked multi-head self-attention

Steps: L is the first decoder sublayer receiving shifted target representations as Q, K and V; its residual path joins M. A is positional encoding at D, B is the later feed-forward block Q, D is cross-attention O, and E is the output projection S.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T2 Quiz 2 NA, Q4

Paper: 2026T2_Q2_NA · Type: MSQ · Marks: 2 · Topic(s): w2-full-transformer

Based on the architecture shown in the given image, identify all components that appear more than once. Assume the architecture contains exactly one Encoder block and one Decoder block.

A. Positional Encoding

B. Feed-Forward Network

C. Masked Multi-Head Self-Attention

D. Multi-Head Cross-Attention

E. tokenizer

F. Add & Norm

Worked answer

Final answer: A, B and F

Steps: The diagram has two positional encodings B and D, two feed-forward blocks G and Q, and five Add & Norm blocks F,H,M,P,R. C appears once at L; D once at O. E is not depicted as a tokenizer (A and C are embedding blocks).

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T2 Quiz 2 NA, Q5

Paper: 2026T2_Q2_NA · Type: MSQ · Marks: 3 · Topic(s): w2-masked-self-attention

A researcher is exploring an alternate approach to implement masked attention in the transformer decoder. The causal mask considered is a binary lower-triangular matrix \(M\) (where \(M_{ij} = 1\) for \(j \le i\) and \(0\) otherwise). This mask (\(M\)) is applied as an element-wise multiplication (\(\odot\)) after the softmax operation on Attention Matrix (\(A\)).

Identify the correct statements from the following:

A. The approach violates the autoregressive property

B. The approach ensures autoregressive property is not violated by ensuring that
the final attention weights of future tokens are 0

C. The final attention weights in the new approach will be greater than or equal
to that of the original approach

D. The attention weights for the final token of the sequence will always be the
same for both the approaches

Worked answer

Final answer: A and D

Steps: Post-softmax multiplication zeros future weights but leaves allowed weights divided by the sum over all logits, including future logits. Changing future keys therefore changes earlier outputs, violating autoregressive independence (A); B overlooks this indirect dependence. Allowed weights are at most, not at least, their pre-softmax-masked counterparts, so C is false. In the final row all positions are allowed, so both methods coincide (D).

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2024 T1 End Term AN, Q157

Paper: 2024T1_ET_AN · Type: SA · Marks: 4 · Topic(s): w2-masked-self-attention

Suppose we pass this \(QK^T\) through the softmax function to compute the final attention score. Following zero-based indexing, at time step \(t_3\), the attention score for the query \(q_3\) is highest for the \(i\)-th key \(k_i\). Enter the value of \(i\).

Worked answer

Final answer: 3

Steps: For query 3 the causal mask admits keys 0–3, with source-image scores 0.94, 0.41, 2.26, 2.29. Softmax preserves their order; the largest is key 3.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2024 T1 End Term FN, Q212

Paper: 2024T1_ET_FN · Type: SA · Marks: 4 · Topic(s): w2-masked-self-attention

Suppose we pass this \(QK^T\) through the softmax function to compute the final attention score. Following zero-based indexing, at time step \(t_3\), the attention score for the query \(q_3\) is highest for the \(i\)-th key \(k_i\). Enter the value of \(i\).

2024 T3 End Term AN, Q191

Paper: 2024T3_ET_AN · Type: MSQ · Marks: 2 · Topic(s): w2-full-transformer, w2-masked-self-attention

Choose the correct statements regarding an encoder-decoder transformer model:

A. Multi head masked attention block in decoder makes use of a mask.

B. Transformer model is an improvement over its RNN based predecessor for
machine translation.

C. For machine tranlation task the input and output embedding layers have to be
identical.

D. None of these.

Worked answer

Final answer: A and B.

Steps: A: causal decoder self-attention masks future positions. B: the Transformer replaces recurrent sequence computation and improved translation benchmarks. C is false: source and target embeddings need not be identical. D is false because A and B hold.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2024 T3 End Term FN, Q190

Paper: 2024T3_ET_FN · Type: MSQ · Marks: 2 · Topic(s): w2-full-transformer, w2-teacher-forcing

Choose the correct statements regarding an encoder-decoder transformer model:

A. A self attention block makes use of a mask matrix.

Label added for presentation.

B. Multi head self attention block is part of the encoder.

Label added for presentation.

C. Teacher forcing can help speed up the training process for tasks such as machine translation.

Label added for presentation.

D. None of these.

Label added for presentation.

Worked answer

Final answer: A, B, C

Steps: An encoder–decoder Transformer has masked decoder self-attention, encoder multi-head self-attention, and parallel teacher-forced training. A states that a self-attention block uses a mask; it does not assert every self-attention block uses a causal mask. D is false.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T2 End Term AN, Q193

Paper: 2025T2_ET_AN · Type: MCQ · Marks: 2 · Topic(s): w2-masked-self-attention

A GPT model is trained using causal language modeling. During training, for a sequence of T = 4
tokens, which of the following correctly represents the attention mask matrix applied to the
attention logits?

A. $\(\begin{bmatrix} 0 & -\infty & -\infty & -\infty \\ 0 & 0 & -\infty & -\infty \\ 0 & 0 & 0 & -\infty \\ 0 & 0 & 0 & 0 \end{bmatrix}\)$

Label added for presentation.

B. $\(\begin{bmatrix} 0 & \infty & \infty & \infty \\ 0 & 0 & \infty & \infty \\ 0 & 0 & 0 & \infty \\ 0 & 0 & 0 & 0 \end{bmatrix}\)$

Label added for presentation.

C. $\(\begin{bmatrix} 1 & 0 & 0 & 0 \\ 1 & 1 & 0 & 0 \\ 1 & 1 & 1 & 0 \\ 1 & 1 & 1 & 1 \end{bmatrix}\)$

Label added for presentation.

D. $\(\begin{bmatrix} 0 & 1 & 1 & 1 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end{bmatrix}\)$

Label added for presentation.

Worked answer

Final answer: A.

Steps: An additive causal logit mask is 0 at allowed positions and −∞ above the diagonal, making future softmax weights zero. B favors future positions; C and D are finite and cannot suppress logits when added.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T2 End Term FN, Q142

Paper: 2025T2_ET_FN · Type: MSQ · Marks: 3 · Topic(s): w2-residual-layernorm, w2-training-tricks

Which of the following are valid design choices when building a transformer-based language
model?

A. Number of transformer layers

Label added for presentation.

B. Learning rate schedule

Label added for presentation.

C. Font style of the source code

Label added for presentation.

D. Vocabulary size

Label added for presentation.

E. Type of normalization (e.g., LayerNorm)

Label added for presentation.

Worked answer

Final answer: A, B, D and E.

Steps: Depth, learning-rate schedule, vocabulary size and normalization are model or training design choices. C, source-code font, does not change the model or learning rule.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 End Term AN, Q3

Paper: 2025T3_ET_AN · Type: MCQ · Marks: 2 · Topic(s): w2-residual-layernorm

A neural network layer produces activations of hidden dimension \(H = 512\).
You consider two normalization strategies:

1. Batch Normalization
2. Layer Normalization
   Which option correctly states the number of trainable parameters added by each normalization method?

A. BatchNorm adds 512 parameters, LayerNorm adds 512 parameters

B. BatchNorm adds 1024 parameters, LayerNorm adds 1024 parameters

C. BatchNorm adds 1024 parameters, LayerNorm adds 512 parameters

D. BatchNorm adds 512 parameters, LayerNorm adds 1024 parameters

E. Both add 0 trainable parameters

Worked answer

Final answer: B — 1,024 parameters for each

Steps: Standard affine BatchNorm and LayerNorm each have 512 scales and 512 biases, so 2×512=1024. A omits one set for both; C and D omit one set for one layer; E assumes no affine parameters.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2025 T3 End Term AN, Q25

Paper: 2025T3_ET_AN · Type: MCQ · Marks: 1 · Topic(s): w2-teacher-forcing

In the inference phase (when the model is used for generation), why is the teacher forcing
technique not used, and what is the primary challenge this introduces?

A. Teacher forcing is used, but only with a small probability.

B. It's not used because the ground-truth sequence is unknown; this introduces
the exposure bias problem, where the model is unfamiliar with its own prediction errors.

C. It's not used because it drastically slows down the inference speed.

D. It's not used; the challenge is the need for a separate encoder-decoder
architecture.

Worked answer

Final answer: B — ground truth is unavailable during generation, producing exposure bias

Steps: B describes the train/inference prefix mismatch. A incorrectly claims ordinary teacher forcing remains; C gives an unsupported speed reason; D confuses the mismatch with an architecture requirement.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 End Term AN, Q26

Paper: 2025T3_ET_AN · Type: MCQ · Marks: 1 · Topic(s): w2-masked-self-attention

Which component of the Transformer decoder is essential for enforcing the Causal Language
Modeling constraint?

A. The Feed-Forward Network layer.

B. The Positional Encoding layer.

C. The masked Self-Attention layer.

D. The Cross-Attention layer.

Worked answer

Final answer: C — masked self-attention

Steps: C suppresses attention to future decoder positions. A is a pointwise feed-forward layer, B encodes positions without forbidding future access, and D attends to source states rather than enforcing target causality.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 End Term FN, Q2

Paper: 2025T3_ET_FN · Type: MCQ · Marks: 1 · Topic(s): w2-residual-layernorm

What happens when you use Batch Normalization with a batch size of 1?

A. It works perfectly fine

B. The statistics become meaningless (mean=0, std=0)

C. It automatically switches to Layer Normalization

D. It uses the running statistics from training

Worked answer

Final answer: No universally correct option; D applies in evaluation with tracked running statistics

Steps: For one observation per normalized channel during training, the mean is that observation, not necessarily 0, and variance is 0; some implementations reject the case. Evaluation with tracked running statistics uses those statistics (D), but other spatial dimensions and evaluation modes are possible. The prompt specifies neither mode nor reduction axes.

Status: ambiguous

Ambiguity note: For one observation per normalized channel during training, the mean is that observation, not necessarily 0, and variance is 0; some implementations reject the case. Evaluation with tracked running statistics uses those statistics (D), but other spatial dimensions and evaluation modes are possible. The prompt specifies neither mode nor reduction axes.

verified_by_code: false — No numeric checker path; answer remains explicitly ambiguous.

2026 T1 End Term AN, Q2

Paper: 2026T1_ET_AN · Type: MCQ · Marks: 2 · Topic(s): w2-masked-self-attention

In the standard Transformer Decoder, the Multi-Head Attention layer is "Masked". What is the specific purpose of this mask during training?

A. To filter out padding tokens to save computation.

B. To prevent the model from attending to the [CLS] and [SEP] special tokens.

C. To preserve the auto-regressive property by ensuring that the prediction for position \(t\) can only depend on known outputs at positions less than \(t\) (preventing "peeking" into the future).

D. To force the model to focus on the Encoder output rather than the Decoder input.

Worked answer

Final answer: C — prevent target-future leakage

Steps: The causal mask permits the current shifted decoder input and earlier ones; when predicting target y_t those inputs contain only y_<t. A describes padding masking, B special-token exclusion, and D is unrelated to the causal constraint.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T1 End Term AN, Q14

Paper: 2026T1_ET_AN · Type: MCQ · Marks: 3 · Topic(s): w2-positional-encoding

Why do standard LLM architectures include explicit Positional Encodings (like RoPE or Sinusoidal) despite the causal mask providing some implicit order?

A. To reduce the memory footprint of the \(K\) and \(V\) matrices.

B. To ensure that the model can only attend to future tokens.

C. To provide a strong inductive bias that allows the model to learn structural and distance-based relationships more efficiently than relying on masking alone.

D. To fix a mathematical error in the original Dot-Product Attention formula.

Worked answer

Final answer: C — position provides a structural and distance-sensitive inductive bias

Steps: C is the intended rationale: explicit encodings make relative and absolute position information accessible. A is unrelated to KV memory reduction; B reverses causality; D invents an error in dot-product attention.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T2 End Term AN, Q3

Paper: 2026T2_ET_AN · Type: MCQ · Marks: 2 · Topic(s): w2-positional-encoding

Which of the following statements regarding one-hot positional encoding and sinusoidal positional encoding is incorrect?

A. The dot product between two sinusoidal positional encoding vectors strictly increases as the relative distance between their positions increases, taking values in the range \([0, \infty)\)

B. The squared Euclidean norm of a sinusoidal positional encoding vector is a constant value independent of the position index

C. For any two distinct positions, the dot product of their corresponding one-hot encoded vectors is zero, and the Euclidean distance between them is strictly constant

D. One-hot positional encoding is incapable of mathematically representing relative distance between tokens

Worked answer

Final answer: A is false; D is also false as an unrestricted mathematical claim

Steps: Sinusoidal dot products sum cosines of relative offsets and do not strictly increase with separation (A false). One-hot vectors have equal pairwise distances, but their active indices can be decoded and subtracted, so one-hot positions are not mathematically incapable of representing relative distance (D false). A single-choice question has two false statements under literal wording.

Status: ambiguous

Ambiguity note: Sinusoidal dot products sum cosines of relative offsets and do not strictly increase with separation (A false). One-hot vectors have equal pairwise distances, but their active indices can be decoded and subtracted, so one-hot positions are not mathematically incapable of representing relative distance (D false). A single-choice question has two false statements under literal wording.

verified_by_code: false — No numeric checker path; answer remains explicitly ambiguous.

2026 T2 End Term AN, Q6

Paper: 2026T2_ET_AN · Type: SA · Marks: 3 · Topic(s): w2-residual-layernorm

Consider a mini-batch of 3 samples, each with 2 features. The input matrix X is given below, where

rows represent individual samples and columns represent features:

Layer Normalization is applied on this with a learned scale parameter \(\gamma = 2\) and a shift parameter \(\beta = 1\).

What is the final output value of the first feature for the third sample?

Shift parameter \(\beta = 1\):

Worked answer

Final answer: 3 when ε=0; 1+2/√(1+ε) otherwise

Steps: The third pictured row is [7,5], with mean 6 and population variance 1. LayerNorm gives 2(7−6)/√(1+ε)+1. Elementary hand calculations conventionally omit ε and obtain 3; implementations use a small positive ε and the problem does not state it.

Status: ambiguous

Ambiguity note: Implied course/exam convention: 3 when ε=0; 1+2/√(1+ε) otherwise. The third pictured row is [7,5], with mean 6 and population variance 1. LayerNorm gives 2(7−6)/√(1+ε)+1. Elementary hand calculations conventionally omit ε and obtain 3; implementations use a small positive ε and the problem does not state it.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.