Week 1: Exam Questions and Worked Answers

Shared context

Question ID: 640653739756

Type: Comprehension
PDF pages: 3, 4

Consider a vocabulary \(V = (A, T, C, G)\) and the corresponding embedding matrix \(E = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 1 & -1 \\ 1 & 1 \end{bmatrix}\). For example, the 0-th row of \(E\) is the embedding for 0-th token of \(V\) and so on.

The following sequence \(X = (G, C, C, T, A, G)\) is to be processed by the encoder layer of the transformer architecture by appropriately adding the positional information using the following encoding scheme:

Assume the index for the position starts from zero. Add positional information to all the tokens in the sequence. Let the resultant sequence be denoted by \(H = E(X) + P\), (\(H\) will be used in the subsequent questions, please write it down). Here, \(E(X)\) denotes the embedding for each token in \(X\).

!Vocabulary V, embedding matrix E, sequence X, and positional encoding scheme equations for transformer encoder

Figure 2. Vocabulary V, embedding matrix E, sequence X, and positional encoding scheme equations for transformer encoder

Based on the above data, answer the given subquestions.

Question 184

Corrected shared-context text: Vocabulary V=(A,T,C,G), embeddings E=[[1,0],[0,1],[1,-1],[1,1]], and X=(G,C,C,T,A,G). Position indices start at zero; H=E(X)+P. The scan defines P(pos,2i)=sin(pos/10^(2i/d_model)) and P(pos,2i+1)=cos(pos/10^(2i/d_model)).

Dependent question

2024 T1 Quiz 1 NA, Q186

Paper: 2024T1_Q1_NA · Type: SA · Marks: 3 · Topic(s): w1-self-attention-qkv

The bottommost encoder layer of the transformer contains the following parameter matrices:

given that the embeddings are row vectors, compute the attention score to produce the new representation for the token (T) at the 3-rd position in the sequence X.

What is the un-normalized attention weight given for the token (A) at the 4th position?

Note: unnormalized here implies the raw score before applying softmax. Avoid scaling the dot-product by \(\sqrt{d_k}\)

Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2024T1_Q1_NA/q186-q186-stem-01-p005.png shows W_Q = diag(1,−1) and W_K = [[1,0],[1,0]]..
Editorial correction: the extracted text differed from the scan (shared_context); see evidence LLM/sources/exam-papers/assets/da5004_2024T1_Q1_NA/ctx-184-187-passage-01-p004.png shows vocabulary, embedding matrix, sequence, positional formula and zero indexing..

Worked answer

Final answer: Approximately -0.057922.

Steps: Using zero-based positions, T is at 3 and A at 4. From the image W_Q=diag(1,-1), W_K=[[1,0],[1,0]]; score = sin(3) × (1+sin(4)+cos(4)).

Status: derived

verified_by_code: false — depends on figure, no code path

Shared context

Question ID: 640653739756

Type: Comprehension
PDF pages: 3, 4

Consider a vocabulary \(V = (A, T, C, G)\) and the corresponding embedding matrix \(E = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 1 & -1 \\ 1 & 1 \end{bmatrix}\). For example, the 0-th row of \(E\) is the embedding for 0-th token of \(V\) and so on.

The following sequence \(X = (G, C, C, T, A, G)\) is to be processed by the encoder layer of the transformer architecture by appropriately adding the positional information using the following encoding scheme:

Assume the index for the position starts from zero. Add positional information to all the tokens in the sequence. Let the resultant sequence be denoted by \(H = E(X) + P\), (\(H\) will be used in the subsequent questions, please write it down). Here, \(E(X)\) denotes the embedding for each token in \(X\).

!Vocabulary V, embedding matrix E, sequence X, and positional encoding scheme equations for transformer encoder

Figure 2. Vocabulary V, embedding matrix E, sequence X, and positional encoding scheme equations for transformer encoder

Based on the above data, answer the given subquestions.

Question 184

Corrected shared-context text: Vocabulary V=(A,T,C,G), embeddings E=[[1,0],[0,1],[1,-1],[1,1]], and X=(G,C,C,T,A,G). Position indices start at zero; H=E(X)+P. The scan defines P(pos,2i)=sin(pos/10^(2i/d_model)) and P(pos,2i+1)=cos(pos/10^(2i/d_model)).

Dependent question

2024 T1 Quiz 1 NA, Q187

Paper: 2024T1_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w1-self-attention-qkv

The bottommost encoder layer of the transformer contains the following parameter matrices:

given that the embeddings are row vectors, compute the attention score to produce the new representation for the token (T) at the 3-rd position in the sequence X.

Which token in the sequence was given the highest score?

A. G

Label added for presentation.

B. C

Label added for presentation.

C. T

Label added for presentation.

D. A

Label added for presentation.

Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2024T1_Q1_NA/q187-q187-stem-01-p006.png shows W_Q = diag(1,−1) and W_K = [[1,0],[1,0]]..
Editorial correction: the extracted text differed from the scan (shared_context); see evidence LLM/sources/exam-papers/assets/da5004_2024T1_Q1_NA/ctx-184-187-passage-01-p004.png shows vocabulary, embedding matrix, sequence, positional formula and zero indexing..

Worked answer

Final answer: A: G at position 0.

Steps: For query T at zero-based index 3 the unscaled scores are approximately [0.42336, 0.19500, 0.06959, 0.02133, -0.05792, 0.18695]. G at 0 is highest; C (B), T (C), and A (D) score lower.

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T1 Quiz 1 NA, Q189

Paper: 2024T1_Q1_NA · Type: SA · Marks: 3 · Topic(s): w1-encoder-parameter-count

How many parameters are there in the multi-head attention layer of the encoder (exclude the parameters in the WO matrix used for linear transformation)?

Worked answer

Final answer: 768 weights if the preceding subquestion’s 1600 source embedding parameters apply; otherwise undetermined

Steps: The shared passage fixes vocabulary 100 and two heads but does not give model width. The preceding subquestion introduces 1600 embedding parameters only as a supposition: if carried forward, d=1600/100=16 and 3d²=768 weights; otherwise d remains free. Projection bias is also unspecified.

Status: ambiguous

Ambiguity note: The shared passage fixes vocabulary 100 and two heads but does not give model width. The preceding subquestion introduces 1600 embedding parameters only as a supposition: if carried forward, d=1600/100=16 and 3d²=768 weights; otherwise d remains free. Projection bias is also unspecified.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2024 T3 Quiz 1 NA, Q290

Paper: 2024T3_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w1-multi-head-attention

What will be the shape of \(W_K\)?

A. 32 × 32

Label added for presentation.

B. 64 × 32

Label added for presentation.

C. 128 × 16

Label added for presentation.

D. 128 × 32

Label added for presentation.

E. 8 × 32

Label added for presentation.

F. None of these.

Label added for presentation.

Worked answer

Final answer: D: 128 × 32 for one head.

Steps: The image asks for W_Q in one head: d_model=128 and 4 heads give d_q=32. Thus W_Q is 128 × 32; A, B, C, E have incorrect dimensions and F is false.

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T3 Quiz 1 NA, Q292

Paper: 2024T3_Q1_NA · Type: SA · Marks: 4 · Topic(s): w1-self-attention-qkv

The input embeddings for the words “learning”, “brings” and “joy” are \(h_1 = [0.5, 0.25, 1]\), \(h_2 = [0.1, 0.25, 0]\), and \(h_3 = [0.1, 0.1, 0.9]\), respectively. Note that the embeddings are row vectors. The projection matrices are as follows

The following quantities are computed as
$\(Q = H W_Q \quad K = H W_K \quad V = H W_V\)$

Let \(e_{ij}\) denote the unnormalized attention score, \(\alpha_{ij}\) denote the normalized attention score (ignore the scaling by \(\sqrt{d_k}\)) and \(z_j\) denote the linear combination of the value vectors for the \(j\)-th word.

Suppose the gradient vector \(\frac{\partial L}{\partial z_1} = [1, 2]\), then what is the gradient vector \(\frac{\partial L}{\partial h_1}\)? Enter the sum of gradients.

Worked answer

Final answer: 0.

Steps: The image asks for sum of ∂L/∂e₃, not ∂L/∂h₁. Since a₃=softmax(e₃), ∂L/∂e₃=(diag(a₃)−a₃a₃ᵀ)V[1,2]ᵀ; every column of the softmax Jacobian sums to zero, so the gradient sum is zero.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T1 Quiz 1 NA, Q107

Paper: 2025T1_Q1_NA · Type: MSQ · Marks: 2 · Topic(s): w1-seq2seq-bottleneck

Which of the following models, in general, struggle to encode a context when translating long
sentences?

A. An RNN model

B. An RNN model with attention mechanism

C. A transformer model

Worked answer

Final answer: A

Steps: A fixed-vector encoder–decoder RNN has a long-sequence information bottleneck; attention supplies direct access to source states, as do Transformer attention mechanisms. The phrase “in general” asks for this comparative course principle, not a claim that the other models have unlimited context.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T1 Quiz 1 NA, Q112

Paper: 2025T1_Q1_NA · Type: SA · Marks: 3 · Topic(s): w1-encoder-parameter-count, w1-multi-head-attention

How many parameters are there in the multi-head attention layer of the encoder (exclude the
parameters in the WO matrix used for linear transformation and FFN layer)?

Worked answer

Final answer: 3,072 Q/K/V matrix weights, or 3,168 including Q/K/V biases, if d=32

Steps: Carrying the preceding conditional source table to the common model gives d=3200/100=32; 3d²=3072 weights, plus 3d=96 biases if present. The course key implies shared context and weights only, but the earlier supposition is not expressly repeated and biases are not fixed.

Status: ambiguous

Ambiguity note: Implied course/exam convention: 3,072 Q/K/V matrix weights, or 3,168 including Q/K/V biases, if d=32. Carrying the preceding conditional source table to the common model gives d=3200/100=32; 3d²=3072 weights, plus 3d=96 biases if present. The course key implies shared context and weights only, but the earlier supposition is not expressly repeated and biases are not fixed.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2025 T1 Quiz 1 NA, Q113

Paper: 2025T1_Q1_NA · Type: SA · Marks: 3 · Topic(s): w1-encoder-parameter-count, w1-multi-head-attention

How many parameters does the matrix WQ have in anyone head of the encoder?

Worked answer

Final answer: 256 matrix weights if d=32; otherwise d²/4

Steps: If the preceding subquestion’s embedding condition applies, d=3200/100=32 and one of four heads has W_Q of size 32×8=256. Without carrying the conditional value forward the common passage leaves d unknown. The matrix itself has no separate bias; the course key implies carrying the width forward.

Status: ambiguous

Ambiguity note: Implied course/exam convention: 256 matrix weights if d=32; otherwise d²/4. If the preceding subquestion’s embedding condition applies, d=3200/100=32 and one of four heads has W_Q of size 32×8=256. Without carrying the conditional value forward the common passage leaves d unknown. The matrix itself has no separate bias; the course key implies carrying the width forward.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2025 T2 Quiz 1 NA, Q135

Paper: 2025T2_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w1-ffn-encoder-stack

Transformers process input tokens:

A. One at a time (sequentially)

B. In reverse order

C. All at once (in parallel)

D. Only after seeing the full input

Worked answer

Final answer: C: all input positions can be processed in parallel.

Steps: Self-attention and feed-forward operations allow parallel processing of available positions during a forward pass. A and B prescribe a recurrence or reverse order, neither required; D is not generally true for causal generation, which processes a prefix.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T2 Quiz 1 NA, Q136

Paper: 2025T2_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w1-scaled-dot-product

What is the purpose of the softmax function in the attention mechanism?

A. Normalize attention scores to a probability distribution

B. Add non-linearity to the model

C. To predict the correct class for the loss function

D. Remove redundant features from the input

Worked answer

Final answer: A: normalize attention scores into a probability distribution.

Steps: Row-wise softmax yields nonnegative weights summing to one, used to average values. B is a generic consequence rather than the attention-specific purpose; C concerns an output classifier; D is not what softmax does.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T2 Quiz 1 NA, Q146

Paper: 2025T2_Q1_NA · Type: SA · Marks: 3 · Topic(s): w1-scaled-dot-product

A transformer generates the following attention weights for a token over four positions: (0.1, 0.2, 0.5, 0.2). The corresponding value vectors are:

Compute the output vector from the attention mechanism by taking the weighted sum of the value vectors, and return the first element of the resulting vector. (Round to 2 decimal places)

Worked answer

Final answer: 1.00.

Steps: The weighted vector is 0.1[1,0]+0.2[0,1]+0.5[1,1]+0.2[2,1]=[1.0,0.9]; the first component is 1.00.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T2 Quiz 1 NA, Q147

Paper: 2025T2_Q1_NA · Type: MCQ · Marks: 3 · Topic(s): w1-scaled-dot-product, w1-multi-head-attention

Select the scaled dot-product attention for the first head:

A. $\(\begin{bmatrix} 0 & 0 \\ 1 & 1 \end{bmatrix}\)$

B. $\(\begin{bmatrix} 0 & 0 \\ 6 & 12 \end{bmatrix}\)$

C. $\(\begin{bmatrix} 1.46 & 2.92 \\ 4.53 & 9.08 \end{bmatrix}\)$

D. $\(\begin{bmatrix} 1.98 & 3.76 \\ 4.02 & 8.31 \end{bmatrix}\)$

Worked answer

Final answer: No listed option; approximately [[4,8],[4,8]]

Steps: The original shared image has X=[[1,0],[0,2],[1,0],[0,2]] and W¹=[[1,1,1,1],[2,2,2,2]], with column-oriented tokens. Hence Q=K=V=[[2,4],[4,8]], and the displayed softmax(QᵀK/√2)Vᵀ is approximately [[4,8],[4,8]]. No option matches. The Markdown transcription is different.

Status: ambiguous

Ambiguity note: The original shared image has X=[[1,0],[0,2],[1,0],[0,2]] and W¹=[[1,1,1,1],[2,2,2,2]], with column-oriented tokens. Hence Q=K=V=[[2,4],[4,8]], and the displayed softmax(QᵀK/√2)Vᵀ is approximately [[4,8],[4,8]]. No option matches. The Markdown transcription is different.

verified_by_code: false — Override numeric derivation not code-verified: complete numeric inputs are absent from the editable extraction; manual scan adjudication retained.

2025 T2 Quiz 1 NA, Q148

Paper: 2025T2_Q1_NA · Type: MCQ · Marks: 3 · Topic(s): w1-scaled-dot-product, w1-multi-head-attention

Select the scaled dot-product attention for the second head:

A. $\(\begin{bmatrix} 1.34 & 2.23 \\ 0.66 & 4.77 \end{bmatrix}\)$

B. $\(\begin{bmatrix} 2.33 & 1.32 \\ 2.94 & 3.76 \end{bmatrix}\)$

C. $\(\begin{bmatrix} 1.96 & 1.68 \\ 3.04 & 2.32 \end{bmatrix}\)$

D. $\(\begin{bmatrix} 2.65 & 2.60 \\ 2.58 & 2.32 \end{bmatrix}\)$

Worked answer

Final answer: No listed option; approximately [[1.33952,3.33024],[0.11161,3.94419]]

Steps: From the pictured shared X and W², Q=K=V=[[2,0],[3,4]]. Row-softmax(QᵀK/√2)Vᵀ gives the stated matrix; no pictured option agrees. The Markdown transcription differs from the shared image.

Status: ambiguous

Ambiguity note: From the pictured shared X and W², Q=K=V=[[2,0],[3,4]]. Row-softmax(QᵀK/√2)Vᵀ gives the stated matrix; no pictured option agrees. The Markdown transcription differs from the shared image.

verified_by_code: false — Override numeric derivation not code-verified: complete numeric inputs are absent from the editable extraction; manual scan adjudication retained.

2025 T2 Quiz 1 NA, Q149

Paper: 2025T2_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w1-multi-head-attention

Concatenate the outputs from both the attention heads, then apply the output projection matrix Wo to produce the final output of the multi-head attention mechanism. Select the correct result of this operation.

A. $\(\begin{bmatrix} 0.73 & 1.46 \\ 2.26 & 4.54 \\ 2.33 & 1.32 \\ 2.94 & 3.76 \end{bmatrix}\)$

B. $\(\begin{bmatrix} 0.99 & 1.88 \\ 2.01 & 4.16 \\ 1.96 & 1.68 \\ 3.04 & 2.32 \end{bmatrix}\)$

C. $\(\begin{bmatrix} 0.00 & 0.00 \\ 0.50 & 0.50 \\ 2.65 & 2.60 \\ 2.58 & 2.32 \end{bmatrix}\)$

D. $\(\begin{bmatrix} 0.00 & 0.00 \\ 3.00 & 6.00 \\ 1.34 & 2.23 \\ 0.66 & 4.77 \end{bmatrix}\)$

Worked answer

Final answer: No listed option; approximately [[2,2],[4,4],[1.33952,0.11161],[3.33024,3.94419]]

Steps: Transpose the two head outputs into the image’s feature-by-token layout, concatenate and left-multiply diag(0.5,0.5,1,1). The resulting four-by-two matrix matches no pictured option; the shared-image matrices differ from the transcription.

Status: ambiguous

Ambiguity note: Transpose the two head outputs into the image’s feature-by-token layout, concatenate and left-multiply diag(0.5,0.5,1,1). The resulting four-by-two matrix matches no pictured option; the shared-image matrices differ from the transcription.

verified_by_code: false — Override numeric derivation not code-verified: complete numeric inputs are absent from the editable extraction; manual scan adjudication retained.

2025 T3 Quiz 1 NA, Q173

Paper: 2025T3_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w1-scaled-dot-product

What is the primary reason for dividing the dot product of the query and key vectors by the square root of the key vector’s dimension, \(\sqrt{d_k}\)?

A. To increase the numerical precision of the attention scores.

Label added for presentation.

B. To reduce the overall computational cost of the attention mechanism.

Label added for presentation.

C. To ensure that the sum of the attention scores for each query equals 1.

Label added for presentation.

D. To avoid numerical issues and loss of gradients during training.

Label added for presentation.

Worked answer

Final answer: D: scaling limits softmax saturation and vanishing gradients.

Steps: Independent unit-variance components yield dot-product variance proportional to d_k; dividing by √d_k controls scale. A is not the main motivation; B does not reduce asymptotic cost; C is guaranteed by softmax.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 Quiz 1 NA, Q183

Paper: 2025T3_Q1_NA · Type: MSQ · Marks: 2 · Topic(s): w1-multi-head-attention

Which of the following statements about multiple attention heads in a Transformer are true?

A. They allow learning diverse local/global relations.

Label added for presentation.

B. They reduce memory usage compared to one head.

Label added for presentation.

C. Increasing the number of heads while keeping dmodel fixed, decreases the dimensionality handled by each head.

Label added for presentation.

D. All heads necessarily learn completely independent information about the input.

Label added for presentation.

Worked answer

Final answer: A and C.

Steps: A: heads can learn different relations. C: standard equal head width is d_model/h. B: more heads do not generally reduce memory and score matrices grow with head count. D: heads may overlap.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 Quiz 1 NA, Q186

Paper: 2025T3_Q1_NA · Type: SA · Marks: 3 · Topic(s): w1-multi-head-attention

A Transformer with single self-attention layer has batch size \(B = 4\), context length (sequence length) \(T = 12\), and number of heads \(h = 2\). How many scalar attention scores are computed in total across two heads and all sequences for one forward pass?

Worked answer

Final answer: 1152 scalar scores.

Steps: Each head produces a [12, 12] score matrix per sequence, hence 4×2×12² = 1152.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T3 Quiz 1 NA, Q187

Paper: 2025T3_Q1_NA · Type: SA · Marks: 3 · Topic(s): w1-multi-head-attention

A Transformer model processes a sequence of \(T = 1000\) tokens. Within the self-attention layer, the attention scores are stored as 32-bit floating-point numbers (float32). We compare two configurations that differ in the number of attention heads, \(h\):

Model A: \(h = 8\)
Model B: \(h = 16\)

Using the given conversion factors (1 byte = 8 bits, 1 MB = \(10^6\) bytes), calculate the percentage increase in memory (in MB) required to store all attention scores when moving from Model A to Model B.

Worked answer

Final answer: 100% increase.

Steps: Float32 uses 4 bytes; 8×1000²×4/10⁶ = 32 MB and 16×1000²×4/10⁶ = 64 MB. The increase is (64−32)/32×100%.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T3 Quiz 1 NA, Q188

Paper: 2025T3_Q1_NA · Type: SA · Marks: 2 · Topic(s): w1-encoder-parameter-count

In a Transformer model with hidden size \(d_{\text{model}} = 512\), number of heads \(h = 8\), and key/query dimension \(d_k = 64\), how many trainable parameters are there in the projection matrix \(W_Q\) for a given head?

Worked answer

Final answer: 32768 parameters.

Steps: One head has W_Q shape [512, 64], so 512×64 = 32768 matrix entries; bias is not part of the matrix.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T3 Quiz 1 NA, Q191

Paper: 2025T3_Q1_NA · Type: SA · Marks: 2 · Topic(s): w1-multi-head-attention

The output projection matrix \(W_O\) maps the concatenated head outputs dimension back to \(d_{\text{model}}\) dimension. How many trainable parameters does \(W_O\) contain?

Worked answer

Final answer: 589824 matrix parameters.

Steps: In the shared setup d_model=768 and the conventional concatenated value width is 768. W_O has shape [768, 768], hence 768²=589824; an optional bias is not part of W_O.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T1 Quiz 1 NA, Q2

Paper: 2026T1_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w1-self-attention-qkv

In a self-attention mechanism, let the input be a matrix \(X \in \mathbb{R}^{T \times d_{model}}\), where \(T\) is the sequence length. If the dimension of the query projection matrix is \(W_Q \in \mathbb{R}^{d_{model} \times d_k}\), what is the dimension of the Query matrix \(Q\)?

A. \(d_{model} \times T\)

B. \(d_k \times T\)

C. \(T \times d_k\)

D. \(T \times d_{model}\)

Worked answer

Final answer: C: [T, d_k].

Steps: Q=XW_Q has shapes [T,d_model] × [d_model,d_k] = [T,d_k]. A and B transpose dimensions; D incorrectly retains d_model in general.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T1 Quiz 1 NA, Q3

Paper: 2026T1_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w1-scaled-dot-product

In the scaled dot-product attention equation \(\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right) V\), what is the primary reason for dividing the dot product by \(\sqrt{d_k}\)?

A. To ensure the matrix dimensions match for multiplication with \(V\)

B. To reduce the number of trainable parameters in the model

C. To prevent the dot product values from growing too large, which would push the softmax function into regions with extremely small gradients

D. To normalize the embedding vectors to have a unit length

Worked answer

Final answer: C.

Steps: The dot-product variance grows with d_k, and √d_k scaling prevents excessively saturated softmax and tiny gradients. A: scalar scaling changes no shape; B: adds/removes no learned parameters; D: does not normalize vector lengths.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T1 Quiz 1 NA, Q4

Paper: 2026T1_Q1_NA · Type: MCQ · Marks: 2 · Topic(s): w1-ffn-encoder-stack

Consider a Transformer Encoder stack consisting of \(N = 6\) identical layers. If the input to the first layer is a sequence of word embeddings of shape \(T \times d_{model}\) (where \(T = 50\) and \(d_{model} = 512\)), what is the shape of the output of the final (6th) layer?

A. \(1 \times 512\) (A single context vector)

B. \(6 \times 512\) (One vector per layer)

C. \(50 \times 512\) (One vector per token, preserving sequence length)

D. \(300 \times 512\) (Concatenation of all token vectors across layers)

Worked answer

Final answer: C: [50, 512].

Steps: Every standard encoder layer preserves sequence length and model width. A pools tokens, B treats layers as tokens, and D concatenates layer outputs, none of which is specified.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T1 Quiz 1 NA, Q9

Paper: 2026T1_Q1_NA · Type: MSQ · Marks: 3 · Topic(s): w1-multi-head-attention

Select all statements that correctly describe the motivation and behavior of Multi-Head Attention as compared to single-head attention.

A. It allows the model to jointly attend to information from different representation subspaces at different positions.

B. It is mathematically similar to having multiple filters/kernels in a CNN to capture different features.

C. It reduces the total number of parameters required compared to a single head with the same total dimension.

D. Each head can theoretically learn to capture different linguistic relationships (e.g., one head links "it" to "animal", another links "it" to "tired").

Worked answer

Final answer: A, B and D.

Steps: A and D follow from separate learned head projections and attention patterns. B is a useful mathematical analogy: parallel learned transformations extract different features, although heads are dynamic attention operators rather than convolution kernels. C is false: standard fixed total width gives the same projection parameter count as one head.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T1 Quiz 1 NA, Q10

Paper: 2026T1_Q1_NA · Type: MSQ · Marks: 3 · Topic(s): w1-scaled-dot-product

Given the vectorized self-attention calculation \(\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right) V\), select all true statements regarding the matrix dimensions and operations. Assume \(Q, K, V \in \mathbb{R}^{T \times d_k}, T > d_k\).

A. The matrix resulting from \(QK^T\) has dimensions \(T \times T\).

B. The softmax operation is applied to the entire matrix at once (globally), not row-wise.

C. The final output \(Z\) has dimensions \(T \times d_v\).

D. The matrix \(QK^T\) represents the raw affinity/score between every pair of tokens in the sequence.

Worked answer

Final answer: A, C and D, with d_v=d_k under the given assumption.

Steps: A: [T,d_k] times [d_k,T] gives [T,T]. D: each entry is a query–key affinity. Softmax is row-wise, so B is false. C: multiplying [T,T] by V yields [T,d_k], equivalently [T,d_v] here with d_v=d_k.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T1 Quiz 1 NA, Q13

Paper: 2026T1_Q1_NA · Type: SA · Marks: 1 · Topic(s): w1-bahdanau-attention

Compute the attention score for hidden state \(h_1\) using the given function. Enter the final answer correct to two decimal places.

Worked answer

Final answer: 1.93.

Steps: For h₁=(1,0), identity U and W give s+h₁=(2,2), and v=(1,1) gives score 2 tanh(2)≈1.928055, rounded to two decimals.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T1 Quiz 1 NA, Q14

Paper: 2026T1_Q1_NA · Type: SA · Marks: 3 · Topic(s): w1-bahdanau-attention

Normalize the attention scores using the softmax function to obtain the attention weights \(\alpha_t\):

Submit \(\alpha_{t1}\) (i.e., the first element of the \(\alpha_t\) vector). Enter the final answer correct to two decimal places.

Worked answer

Final answer: 0.35.

Steps: The three additive scores are approximately 1.928055, 1.756649, 1.959082. Applying exp(score₁)/Σ exp(scoreⱼ) gives 0.347948, rounded only at the end.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T1 Quiz 1 NA, Q15

Paper: 2026T1_Q1_NA · Type: SA · Marks: 2 · Topic(s): w1-bahdanau-attention

Calculate the context vector \(c_t\):

Provide the sum of all elements of \(c_t\). Enter the final answer correct to two decimal places.

Worked answer

Final answer: 1.36.

Steps: With h₁=(1,0), h₂=(0,1), h₃=(1,1), c=(α₁+α₃, α₂+α₃), whose component sum is 1+α₃=1.358913.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T2 Quiz 1 NA, Q2

Paper: 2026T2_Q1_NA · Type: MSQ · Marks: 4 · Topic(s): w1-bahdanau-attention

Which statements are true about traditional attention used in sequence-to-sequence models?

A. The decoder decides which encoder states are important.

B. Attention weights are computed over all decoder hidden states.

C. The context vector is a weighted sum of encoder states.

D. Encoder hidden states are ignored after encoding.

Worked answer

Final answer: A and C.

Steps: In Bahdanau attention the decoder state scores encoder states and the context is their weighted sum. B is false because weights run over encoder states, not all decoder states; D is false because those states are retrieved throughout decoding.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T2 Quiz 1 NA, Q3

Paper: 2026T2_Q1_NA · Type: SA · Marks: 3 · Topic(s): w1-scaled-dot-product

Compute the \((Q K^T)\) of the final scaled dot-product attention and submit the sum of the diagonal elements of \((Q K^T)\) matrix.

Worked answer

Final answer: 4.

Steps: The original PNG uses QᵀK, consistent with tokens as columns of X; Q=K=X because W_Q=W_K=I. Thus QᵀK=[[1,0,1],[0,1,1],[1,1,2]] and its trace is 1+1+2=4. The transcription incorrectly writes QKᵀ.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T2 Quiz 1 NA, Q4

Paper: 2026T2_Q1_NA · Type: MCQ · Marks: 3 · Topic(s): w1-self-attention-qkv

Choose the token pair with the least attention score.

A. {"learn", "easy"}

B. {"learn", "math"}

C. {"easy", "math"}

D. {"learn", "learn"}

E. {"easy" , "easy"}

Worked answer

Final answer: A: {learn, easy}.

Steps: The source image specifies column-token QᵀK=[[1,0,1],[0,1,1],[1,1,2]]. The indicated pairs have scores A=0; B=C=D=E=1. Scaling by √2 does not change the ranking, and within the learn query A also has the smallest row-softmax weight.

Status: derived

verified_by_code: false — solve_numeric.py has no registered derivation for this ID; the extracted question data do not supply a complete computable input set.

2026 T2 Quiz 1 NA, Q5

Paper: 2026T2_Q1_NA · Type: SA · Marks: 4 · Topic(s): w1-multi-head-attention

A Transformer model processes a sequence containing 6 tokens using Multi-Head Attention. The model initially uses 4 attention heads. If the number of attention heads is doubled to 8, while the sequence length remains unchanged, how many additional attention scores are computed across all heads?

Worked answer

Final answer: 144 additional scores.

Steps: Every head computes 6²=36 scores per sequence; four additional heads therefore add (8−4)×36=144.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T2 Quiz 1 NA, Q6

Paper: 2026T2_Q1_NA · Type: SA · Marks: 4 · Topic(s): w1-ffn-encoder-stack

A batch of 8 sentences is passed through a Transformer Encoder consisting of 6 encoder layers and 8 attention heads. After padding, each sentence has a maximum length of 32 tokens. The model uses an embedding dimension of \(d_{model} = 128\). What is the total number of elements (volume) in the output tensor produced by the final encoder layer for the entire batch?

Worked answer

Final answer: 32768 elements.

Steps: The last encoder output retains shape [8,32,128], independent of the number of layers and heads; 8×32×128=32768.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T3 Quiz 2 NA, Q110

Paper: 2025T3_Q2_NA · Type: SA · Marks: 2 · Topic(s): w1-encoder-parameter-count

A study is comparing three architectures, all trained with the same compute budget:

• Model A (Encoder-Decoder): A 12-layer encoder and a 12-layer decoder with separate parameters. Its total size is 220 million parameters.

• Model B (Decoder-Only): A 24-layer decoder-only model. Its total size is also approximately 220 million parameters.

• Model C (Shared Enc-Dec): A 12-layer encoder and a 12-layer decoder where the parameters are shared (\(W_{\text{enc}} = W_{\text{dec}}\)).

Assuming that the 12-layer encoder and 12-layer decoder in Model A have an equal number of parameters, what is the approximate total parameter count (in millions) of Model C?

Worked answer

Final answer: Approximately 110 million parameters under the stated idealized sharing assumption.

Steps: Equal halves of 220 million each contain 110 million; fully tying corresponding encoder and decoder parameters stores one copy. This assumes all counted parameters can actually be shared.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T1 Quiz 2 NA, Q2

Paper: 2026T1_Q2_NA · Type: MCQ · Marks: 2 · Topic(s): w1-multi-head-attention

In the context of Multi-Head Attention, if the model dimension \(d_{model} = 512\) and we employ \(h = 8\) heads, what is the dimension of the concatenated output of the 8 heads before it is passed through the final linear output projection \(W^O\)?

A. \(T \times 64\)

B. \(T \times 4096\)

C. \(T \times 512\)

D. \(T \times 8\)

Worked answer

Final answer: C: [T, 512].

Steps: Under standard equal-width heads d_v=512/8=64; concatenating eight [T,64] outputs gives [T,512]. A is one head, B assumes each head has width 512, and D confuses head count with concatenated width.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T1 Quiz 2 NA, Q8

Paper: 2026T1_Q2_NA · Type: MSQ · Marks: 3 · Topic(s): w1-ffn-encoder-stack

Which of the following are components found within a standard Transformer Encoder layer?

A. Multi-Head Self-Attention mechanism

B. Position-wise Feed-Forward Networks

C. Cross-Attention mechanism (Encoder-Decoder attention)

D. Masked Multi-Head Self-Attention

Worked answer

Final answer: A and B.

Steps: The original encoder has unmasked multi-head self-attention and position-wise FFNs. C belongs to decoder cross-attention, and D describes the causal masked decoder self-attention, not the standard encoder.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T1 Quiz 2 NA, Q9

Paper: 2026T1_Q2_NA · Type: MSQ · Marks: 3 · Topic(s): w1-seq2seq-bottleneck

Select all correct statements regarding the comparison between RNNs and Transformers.

A. In Transformers, the path length between any two positions in the sequence is constant \(O(1)\), whereas in RNNs it is \(O(T)\) (where \(T\) is the sequence length).

B. Transformers process tokens strictly sequentially during training in the same way as RNNs.

C. Transformers allow for significantly more parallelization during training compared to RNNs.

D. Attention mechanisms in Transformers utilize Query, Key, and Value vectors derived from input embeddings.

Worked answer

Final answer: A, C and D.

Steps: A: full self-attention connects arbitrary positions in one layer, versus O(T) maximum recurrent path. C: token operations can run in parallel at training. D: Q/K/V are projections of layer inputs, initially embeddings. B is false: teacher-forced transformer training parallelizes sequence positions.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T1 Quiz 2 NA, Q14

Paper: 2026T1_Q2_NA · Type: SA · Marks: 2 · Topic(s): w1-self-attention-qkv

At time step t = 5 (word: data), what is the attention weight assigned to the word science? (Provide exact answer)

Worked answer

Final answer: 0.2.

Steps: In the provided attention matrix, row t=5 (data) and column 2 (science) contain 0.2. The row sums to 0.9, so the supplied matrix is not a valid normalized attention distribution, but its stated entry is unambiguous.

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T1 End Term AN, Q151

Paper: 2024T1_ET_AN · Type: SA · Marks: 4 · Topic(s): w1-encoder-parameter-count

Suppose the number of encoder layers N = 3. What is the total number of learnable parameters, including the parameters of the input and position embedding layers, in the model? Enter the final answer in thousands. For example, if the answer is 12,238, then enter 12 as the answer.

Worked answer

Final answer: Not uniquely determinable; floor((2,366,208 + 256V)/1000) under the stated illustrative convention

Steps: The pictured shared passage supplies no vocabulary size V, although token embeddings are requested. With bias-free attention, FFN biases and two affine norms per layer, the three encoder layers have 2,366,208 parameters and embeddings add 256V; other bias assumptions change even the constant. Fixed sinusoids add zero. The printed 2566 cannot establish a missing vocabulary size.

Status: ambiguous

Ambiguity note: The pictured shared passage supplies no vocabulary size V, although token embeddings are requested. With bias-free attention, FFN biases and two affine norms per layer, the three encoder layers have 2,366,208 parameters and embeddings add 256V; other bias assumptions change even the constant. Fixed sinusoids add zero. The printed 2566 cannot establish a missing vocabulary size.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2024 T1 End Term AN, Q152

Paper: 2024T1_ET_AN · Type: SA · Marks: 3 · Topic(s): w1-encoder-parameter-count

Suppose we increase the context length from T = 256 to T = 512 keeping number of layers same (N = 3). How many parameters does the model need to learn additionally compared to the previous setting?

Worked answer

Final answer: 0 additional parameters.

Steps: Fixed sinusoidal positions have no learned parameters and the layer and vocabulary sizes are unchanged.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2024 T1 End Term AN, Q153

Paper: 2024T1_ET_AN · Type: SA · Marks: 4 · Topic(s): w1-self-attention-qkv

Compute the representation for the word “often” and enter the sum of the elements in it.

Worked answer

Final answer: 1.676

Steps: The image supplies an attention matrix and W_V and W_O, so the attention-layer output naturally includes W_O. In sentence order the row [0.04,0.12,0.17,0.67] gives A X W_V W_O=[-0.417,2.093] and sum 1.676 using the printed rounded entries. Calling the preprojection intermediate a representation does not require leaving W_O unused; the source transcription differs from the picture.

Status: derived

verified_by_code: false — Override numeric derivation not code-verified: complete numeric inputs are absent from the editable extraction; manual scan adjudication retained.

2024 T1 End Term AN, Q154

Paper: 2024T1_ET_AN · Type: SA · Marks: 4 · Topic(s): w1-self-attention-qkv

Suppose the input sentence is “coffee i drink often“. What is the attention score between the word “coffee” with “often”?

Worked answer

Final answer: Approximately 0.53.

Steps: Without positional information, reordering permutes attention rows and columns. Coffee is row 3 and often column 4 in the original table; the entry is 0.53.

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T1 End Term AN, Q155

Paper: 2024T1_ET_AN · Type: SA · Marks: 3 · Topic(s): w1-self-attention-qkv

What is the input sequence length that would have generated this matrix?

Worked answer

Final answer: 8 tokens.

Steps: The displayed score matrix has shape 8 × 8, one row and column per token.

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T1 End Term AN, Q156

Paper: 2024T1_ET_AN · Type: MSQ · Marks: 4 · Topic(s): w1-self-attention-qkv

Suppose the dimension of value vector, \(d_v = 32\), then which of the following could be the dimensions of the key and query vectors \((d_k, d_q)\)?

A. (8,8)

Label added for presentation.

B. (32,8)

Label added for presentation.

C. (32,32)

Label added for presentation.

D. (64,64)

Label added for presentation.

E. (8,32)

Label added for presentation.

Worked answer

Final answer: A, C and D.

Steps: The query–key dot product requires d_q = d_k, while d_v can differ. A, C, D have equal query and key dimensions; B and E do not.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2024 T1 End Term FN, Q206

Paper: 2024T1_ET_FN · Type: SA · Marks: 4 · Topic(s): w1-encoder-parameter-count

Suppose the number of encoder layers N = 3. What is the total number of learnable parameters, including the parameters of the input and position embedding layers, in the model? Enter the final answer in thousands. For example, if the answer is 12,238, then enter 12 as the answer.

Worked answer

Final answer: Not uniquely determinable; floor((2,366,208 + 256V)/1000) under the stated illustrative convention

Steps: The pictured shared passage supplies no vocabulary size V, although token embeddings are requested. With bias-free attention, FFN biases and two affine norms per layer, the three encoder layers have 2,366,208 parameters and embeddings add 256V; other bias assumptions change even the constant. Fixed sinusoids add zero. The printed 2566 cannot establish a missing vocabulary size.

Status: ambiguous

Ambiguity note: The pictured shared passage supplies no vocabulary size V, although token embeddings are requested. With bias-free attention, FFN biases and two affine norms per layer, the three encoder layers have 2,366,208 parameters and embeddings add 256V; other bias assumptions change even the constant. Fixed sinusoids add zero. The printed 2566 cannot establish a missing vocabulary size.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2024 T1 End Term FN, Q207

Paper: 2024T1_ET_FN · Type: SA · Marks: 3 · Topic(s): w1-encoder-parameter-count

Suppose we increase the context length from T = 256 to T = 512 keeping number of layers same (N = 3). How many parameters does the model need to learn additionally compared to the previous setting?

Worked answer

Final answer: 0 additional parameters.

Steps: Fixed sinusoidal positions have no learned parameters and the layer and vocabulary sizes are unchanged.

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T1 End Term FN, Q208

Paper: 2024T1_ET_FN · Type: SA · Marks: 4 · Topic(s): w1-self-attention-qkv

Compute the representation for the word “coffee” and enter the sum of the elements in it.

Worked answer

Final answer: 1.592

Steps: Using the pictured coffee row [0.25,0.06,0.17,0.53] and its input embeddings gives A X W_V W_O=[-0.324,1.916], sum 1.592. W_O is supplied as part of the attention layer and is naturally applied; the source Markdown transcription contains different matrices.

Status: derived

verified_by_code: false — Override numeric derivation not code-verified: complete numeric inputs are absent from the editable extraction; manual scan adjudication retained.

2024 T1 End Term FN, Q209

Paper: 2024T1_ET_FN · Type: SA · Marks: 4 · Topic(s): w1-self-attention-qkv

Suppose the input sentence is “coffee i like much“. What is the attention score between the word “coffee” with “much”?

Worked answer

Final answer: Approximately 0.53.

Steps: Without positional information, reordering permutes attention rows and columns. Coffee is row 3 and much column 4 in the original table; the entry is 0.53.

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T1 End Term FN, Q210

Paper: 2024T1_ET_FN · Type: SA · Marks: 3 · Topic(s): w1-self-attention-qkv

What is the input sequence length that would have generated this matrix?

Worked answer

Final answer: 8 tokens.

Steps: The displayed score matrix has shape 8 × 8, one row and column per token.

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T1 End Term FN, Q211

Paper: 2024T1_ET_FN · Type: MSQ · Marks: 4 · Topic(s): w1-self-attention-qkv

Suppose the dimension of value vector, \(d_v = 16\), then which of the following could be the dimensions of the key and query vectors \((d_k, d_q)\)?

A. (8,8)

Label added for presentation.

B. (64,64)

Label added for presentation.

C. (32,32)

Label added for presentation.

D. (16,16)

Label added for presentation.

E. (8,16)

Label added for presentation.

F. (16,8)

Label added for presentation.

Worked answer

Final answer: A, B, C and D.

Steps: The query–key dot product requires d_q = d_k, while d_v can differ. A–D have equal dimensions; E and F do not.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2024 T3 End Term AN, Q176

Paper: 2024T3_ET_AN · Type: SA · Marks: 3 · Topic(s): w1-self-attention-qkv

The input embeddings for the words “learning”, “brings” and “joy” are \(h_1 = [1.0, 0.5, 1]\), \(h_2 = [1, 0.25, 0]\), and \(h_3 = [0.1, 0.1, 0.9]\), respectively. Note that the embeddings are row vectors. The projection matrices are as follows:

The following quantities are computed as:

Let \(e_{ij}\) denote the unnormalized attention score, \(\alpha_{ij}\) denote the normalized attention score (ignore the scaling by \(\sqrt{d_k}\)) and \(z_j\) denote the linear combination of the value vectors for the \(j\)-th word.

Enter the value of first element i.e. with index (0,0) of \(z_1\).

Worked answer

Final answer: Approximately 0.239933.

Steps: The image asks for the (0,0) element of ∂a₃/∂e₃, not z₁. For query 3, scores are [2.2, 1.1, 1.1], giving a₃,₀≈0.600333; its softmax diagonal derivative is a₃,₀(1−a₃,₀)≈0.239933.

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T3 End Term AN, Q184

Paper: 2024T3_ET_AN · Type: MCQ · Marks: 2 · Topic(s): w1-self-attention-qkv

Which of the following is correct expression for \(e_{ij}\)? The symbols have usual meaning.

A. \(x_i W_Q W_K^T x_j^T + x_i W_Q W_K^T p_j^T + p_i W_Q W_K^T x_j^T + p_i W_Q W_K^T p_j^T\)

B. \(x_i W_Q W_K^T x_j^T - x_i W_Q W_K^T p_j^T + p_i W_Q W_K^T x_j^T - p_i W_Q W_K^T p_j^T\)

C. \(x_i W_K W_Q^T x_j^T + x_i W_K W_Q^T p_j^T + p_i W_K W_Q^T x_j^T + p_i W_K W_Q^T p_j^T\)

D. \(x_i W_Q W_K x_j + x_i W_Q W_K p_j + p_i W_Q W_K x_j + p_i W_Q W_K p_j\)

E. None of these

Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2024T3_ET_AN/q184-q184-stem-01-p005.png asks for e_ij, not q_ij..
Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2024T3_ET_AN/q184-q184-option-a-01-p005.png shows the row/column transposes in the corrected expression..
Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2024T3_ET_AN/q184-q184-option-b-01-p005.png shows the row/column transposes in the corrected expression..
Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2024T3_ET_AN/q184-q184-option-c-01-p005.png shows the row/column transposes in the corrected expression..
Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2024T3_ET_AN/q184-q184-option-d-01-p005.png shows the row/column transposes in the corrected expression..

Worked answer

Final answer: A.

Steps: With row vectors, eᵢⱼ=(xᵢ+pᵢ)W_Q W_Kᵀ(xⱼ+pⱼ)ᵀ; expanding gives A. B has incorrect signs; C reverses query and key projections; D omits necessary transposes; E is false.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2024 T3 End Term FN, Q189

Paper: 2024T3_ET_FN · Type: SA · Marks: 3 · Topic(s): w1-self-attention-qkv

The input embeddings for the words “learning”, “brings” and “joy” are \(h_1 = [0.5, 0.25, 1]\), \(h_2 = [0.1, 0.25, 0]\), and \(h_3 = [0.1, 0.1, 0.9]\), respectively. Note that the embeddings are row vectors. The projection matrices are as follows

The following quantities are computed as

Let \(e_{ij}\) denote the unnormalized attention score, \(a_{ij}\) denote the normalized attention score (ignore the scaling by \(\sqrt{d_k}\)) and \(z_i\) denote the linear combination of the value vectors for the \(i\)-th word.

Enter the value of the first element, i.e., with index (0,0) of the matrix containing the partial derivatives, \(\frac{\partial a_{ij}}{\partial e_{ij}}\)

Worked answer

Final answer: Approximately 0.246608.

Steps: The image asks for ∂a₃/∂e₃ at (0,0). For query 3 scores are [1.65, 0.11, 1.1], so a₃,₀≈0.558244 and the corresponding softmax diagonal derivative is a₃,₀(1−a₃,₀)≈0.246608.

Status: derived

verified_by_code: false — depends on figure, no code path

2024 T3 End Term FN, Q199

Paper: 2024T3_ET_FN · Type: MCQ · Marks: 2 · Topic(s): w1-self-attention-qkv

What is the time complexity of the computing attention score matrix A with a batch size of one? The symbols have usual meaning.

A. \(O(T^2 d)\)

Label added for presentation.

B. \(O(T d^2)\)

Label added for presentation.

C. \(O(T d)\)

Label added for presentation.

D. \(O(T + d)\)

Label added for presentation.

E. None of these.

Label added for presentation.

Worked answer

Final answer: A: O(T²d).

Steps: Multiplying Q [T,d] by Kᵀ [d,T] produces T² dot products of length d. B is projection cost, C and D omit pairwise work, E is false.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T2 End Term AN, Q199

Paper: 2025T2_ET_AN · Type: SA · Marks: 3 · Topic(s): w1-self-attention-qkv, w1-scaled-dot-product, w1-multi-head-attention

For the input “you enjoy tea often”, compute the final representation of the word “enjoy” after the
attention layer (i.e., after applying WQ, WK, WV , attention weights, and WO). Enter the sum of the
elements in the resulting vector.

Worked answer

Final answer: Approximately 0.950 using the supplied rounded attention matrix.

Steps: The enjoy row is [0.43,0.25,0.21,0.10]. Multiply this row by H W_V W_O to get [1.331,−0.381], summing to 0.950. Recomputing full precision from the matrices gives approximately 0.95216.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T2 End Term AN, Q200

Paper: 2025T2_ET_AN · Type: SA · Marks: 2 · Topic(s): w1-self-attention-qkv, w1-scaled-dot-product, w1-multi-head-attention

Suppose the input sentence is “tea you enjoy often”. Using the same matrices and processing
method, what is the attention score (i.e., softmax entry from A) for the query word “tea” attending
to key word “often”?

Worked answer

Final answer: Approximately 0.07 from the supplied table.

Steps: Without positional encodings, reordering permutes attention rows and columns; tea is row 3 and often column 4 of the supplied original matrix, so the entry is 0.07. Recomputing from the matrices gives approximately 0.06607.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T2 End Term FN, Q146

Paper: 2025T2_ET_FN · Type: SA · Marks: 3 · Topic(s): w1-self-attention-qkv, w1-scaled-dot-product, w1-multi-head-attention

For the input “you enjoy tea often”, compute the final representation of the word “tea” after the
attention layer (i.e., after applying WQ, WK, WV , attention weights, and WO). Enter the sum of the
elements in the resulting vector.

Worked answer

Final answer: Approximately 1.020 using the supplied rounded attention matrix.

Steps: The tea row is [0.39,0.39,0.16,0.07]. Multiplication by H W_V W_O yields [1.272,−0.252] with sum 1.020. Recomputing full precision from the stated matrices gives approximately 1.01445.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T2 End Term FN, Q147

Paper: 2025T2_ET_FN · Type: SA · Marks: 2 · Topic(s): w1-self-attention-qkv, w1-scaled-dot-product, w1-multi-head-attention

Suppose the input sentence is “tea you enjoy often”. Using the same matrices and processing
method, what is the attention score (i.e., softmax entry from A) for the query word “tea” attending
to key word “often”?

Worked answer

Final answer: Approximately 0.07 from the supplied table.

Steps: With no positional encodings reordering permutes rows and columns; the original tea-query, often-key table entry is 0.07. Recomputing the matrices gives approximately 0.06607.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T3 End Term AN, Q17

Paper: 2025T3_ET_AN · Type: MSQ · Marks: 3 · Topic(s): w1-self-attention-qkv, w1-scaled-dot-product, w1-ffn-encoder-stack

Consider a BART encoder with sequence length \(L\) and hidden size \(d_{model}\) (fixed).
Now suppose the sequence length is doubled from \(L\) to \(2L\), keeping all other parameters unchanged.
Which of the following statements are true?

A. Self-attention FLOPs increase by a factor of 2.

B. Self-attention FLOPs increase by a factor of 4.

C. FFN FLOPs increase by a factor of 2.

D. FFN FLOPs increase by a factor of 4.

Worked answer

Final answer: B and C when “self-attention” means the score and value products; C alone when projection FLOPs are included

Steps: For fixed width, QKᵀ and AV cost cL², so L→2L multiplies these products by 4 (B); the position-wise FFN costs fL and doubles (C). Including Q/K/V and output projections gives aL²+bL and ratio (4aL²+2bL)/(aL²+bL), generally neither 2 nor 4.

Status: ambiguous

Ambiguity note: Implied course/exam convention: B and C when “self-attention” means the score and value products; C alone when projection FLOPs are included. The two attention matrix products scale as L²d and quadruple, while the positionwise FFN scales as L and doubles. Including the Q/K/V/output projections adds a linear-in-L term, making the exact total factor between 2 and 4. The usual textbook attention-complexity convention implies B and C; the old note correctly identifies the scope choice.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2025 T3 End Term AN, Q20

Paper: 2025T3_ET_AN · Type: SA · Marks: 2 · Topic(s): w1-encoder-parameter-count

In the standard Transformer with 6 Encoder Layers, how many distinct sets of transformation weight matrices (\(W_Q, W_K, W_V\)) are learned in the entire Encoder stack, assuming 8 heads per layer?

Worked answer

Final answer: 48 sets (144 individual matrices)

Steps: Each of six encoder layers has eight heads; each head has one set (W_Q,W_K,W_V). Thus there are 6×8=48 sets, containing 144 individual matrices. The question explicitly asks for sets, so the old alternative counts a different object.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2025 T3 End Term FN, Q8

Paper: 2025T3_ET_FN · Type: MCQ · Marks: 1 · Topic(s): w1-self-attention-qkv

Computing the full self-attention matrix has a time complexity of
$\(\mathcal{O}(T^2 d)\)$
where \(T\) is the sequence length and \(d\) is the embedding dimension.

Which step in the self-attention mechanism is primarily responsible for this quadratic scaling?

A. Computing the softmax for each row

B. Computing the matrix product \(QK^T\)

C. Multiplying the attention matrix \(A\) with the value matrix \(V\)

D. Performing layer normalization

Editorial correction: the extracted text differed from the scan (ambiguity_note); see evidence LLM/sources/exam-papers/assets/da5004_2025T3_ET_FN/q8-q8-option-b-01-p004.png and q8-q8-option-c-01-p004.png show portions of different option lines..

Worked answer

Final answer: B: QKᵀ

Steps: QKᵀ multiplies [T,d] by [d,T], constructing the T×T score matrix in O(T²d), so B answers the stated matrix-construction question. AV is also O(T²d) but consumes the matrix; broad wording about the entire mechanism admits C. The supplied B/C image crops are misaligned with their text and cannot verify the option text.

Status: derived

Ambiguity note: The source crops attached to options B and C are vertically misaligned: the B crop shows part of the attention-times-value line and the C crop shows part of the layer-normalization line. Consult the source text or original page for option evidence.

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2025 T3 End Term FN, Q18

Paper: 2025T3_ET_FN · Type: SA · Marks: 3 · Topic(s): w1-self-attention-qkv

A mini-batch has 10 sentences, each padded to a length 100. Self-attention (single head) computes one score matrix per sentence. How many scalar scores are computed in total for a single head across the batch?

Worked answer

Final answer: 100,000 scalar scores.

Steps: Each of 10 sentences has a 100 × 100 score matrix, so 10×100×100=100,000 scores before masking.

Status: derived

verified_by_code: false — depends on figure, no code path

2025 T3 End Term FN, Q25

Paper: 2025T3_ET_FN · Type: MSQ · Marks: 2 · Topic(s): w1-self-attention-qkv, w1-scaled-dot-product, w1-ffn-encoder-stack

Consider a BART encoder with sequence length \(L\) and hidden size \(d_{\text{model}}\) (fixed).
Now suppose the sequence length is doubled from \(L\) to \(2L\), keeping all other parameters unchanged.
Which of the following statements are true?

A. Self-attention FLOPs increase by a factor of 2

B. Self-attention FLOPs increase by a factor of 4

C. FFN FLOPs increase by a factor of 2

D. FFN FLOPs increase by a factor of 4

Worked answer

Final answer: B and C when “self-attention” means the score and value products; C alone when projection FLOPs are included

Steps: For fixed width, QKᵀ and AV cost cL², so L→2L multiplies these products by 4 (B); the position-wise FFN costs fL and doubles (C). Including Q/K/V and output projections gives aL²+bL and ratio (4aL²+2bL)/(aL²+bL), generally neither 2 nor 4.

Status: ambiguous

Ambiguity note: Implied course/exam convention: B and C when “self-attention” means the score and value products; C alone when projection FLOPs are included. The two attention matrix products scale as L²d and quadruple, while the positionwise FFN scales as L and doubles. Including the Q/K/V/output projections adds a linear-in-L term, making the exact total factor between 2 and 4. The usual textbook attention-complexity convention implies B and C; the old note correctly identifies the scope choice.

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T1 End Term AN, Q10

Paper: 2026T1_ET_AN · Type: SA · Marks: 2 · Topic(s): w1-self-attention-qkv

The computational complexity and memory requirement of the Self-Attention mechanism is quadratic with respect to the sequence length \(T\).

Specifically, the Attention Score matrix has a size of \(T \times T\).

If you double the input sequence length \(T\) (e.g., from 512 to 1024) while keeping the embedding dimension (\(d_{model}\)) constant, by what factor does the memory required to store the Attention Score matrix increase? Enter the integer value.

Worked answer

Final answer: Factor 4.

Steps: A score matrix has T² elements, so (2T)²/T²=4 at fixed element size and head count.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T1 End Term FN, Q2

Paper: 2026T1_ET_FN · Type: SA · Marks: 2 · Topic(s): w1-encoder-parameter-count

Consider a Multi-Head Attention sub-layer with the following hyper-parameters:

• Embedding dimension (\(d_{model}\)) = 256
• Number of heads (\(h\)) = 4
• key/value dimension per head (\(d_k = d_v\)) = \(d_{model} / h = 64\).

Calculate the total number of trainable parameters contained only in the projection matrices \(W_i^Q\), \(W_i^K\), \(W_i^V\) and \(W^O\). (Ignore biases). Enter the integer value.

Worked answer

Final answer: 262144 parameters.

Steps: For Q, K and V, 3×4×256×64 = 196608 entries; W_O has 256×256 = 65536. Total 262144; biases are excluded.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T1 End Term FN, Q3

Paper: 2026T1_ET_FN · Type: SA · Marks: 2 · Topic(s): w1-encoder-parameter-count

For T5-Base, the model dimension is 768 and FFN dimension is 3072. Ignore bias. Find the number of parameters in the FFN of one encoder layer.

Worked answer

Final answer: 4718592 parameters.

Steps: The original T5 feed-forward block has two matrices [768, 3072] and [3072, 768]; 2×768×3072=4718592 without biases.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T1 End Term FN, Q4

Paper: 2026T1_ET_FN · Type: SA · Marks: 3 · Topic(s): w1-ffn-encoder-stack

A batch of 5 sentences is passed through a Transformer Encoder. The sequences are padded to a maximum length of 20 tokens. The embedding dimension (\(d_{model}\)) is 64. What is the total number of elements (volume) in the output tensor produced by the final layer of the encoder for this batch?

Worked answer

Final answer: 6400 elements.

Steps: The final encoder tensor retains padded shape [5, 20, 64]; 5×20×64 = 6400.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T2 End Term AN, Q2

Paper: 2026T2_ET_AN · Type: MCQ · Marks: 2 · Topic(s): w1-self-attention-qkv

Consider the following Assertion (A) and Reason (R) about the Transformer encoder.

Assertion (A): In a Transformer encoder, each token can attend to every other token in the input sequence through self-attention.
Reason (R): The encoder's self-attention mechanism uses a causal mask to prevent each token from attending to tokens that appear later in the sequence.

Choose the correct option:

A. Both Assertion (A) and Reason (R) are true, and Reason (R) is the correct explanation of Assertion (A).

B. Both Assertion (A) and Reason (R) are true, but Reason (R) is not the correct explanation of Assertion (A).

C. Assertion (A) is true, but Reason (R) is false.

D. Assertion (A) is false, but Reason (R) is true

E. Both Assertion (A) and Reason (R) are false.

Worked answer

Final answer: C.

Steps: In an original Transformer encoder, each unpadded position can attend bidirectionally, while causal masking is used in the decoder. Thus assertion true, reason false; A and B require true reason, and D and E falsely reject the assertion.

Status: derived

verified_by_code: false — Non-numeric response; no numeric checker path applies.

2026 T2 End Term AN, Q18

Paper: 2026T2_ET_AN · Type: MCQ · Marks: 1 · Topic(s): w1-self-attention-qkv

Compute \(W_Q x_m\) and select the correct option

A. \([4, 3, 1, -1]^T\)

B. \([5, 4, -1, 2]^T\)

C. \([5, 3, -1, -1]^T\)

D. \([3, 5, -2, 1]^T\)

Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2026T2_ET_AN/q18-q18-option-a-01-p012.png shows [4, 3, 1, -1]..
Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2026T2_ET_AN/q18-q18-option-c-01-p012.png shows [5, 3, -1, -1]..
Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2026T2_ET_AN/q18-q18-option-d-01-p012.png shows [3, 5, -2, 1]..

Worked answer

Final answer: C: [5, 3, −1, −1]ᵀ

Steps: Using the pictured x₁=(2,1,3,2), the rows of W_Q yield (2+3,1+2,2−3,1−2)=(5,3,−1,−1), option C. The extracted second component 13 is a transcription error.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T2 End Term AN, Q19

Paper: 2026T2_ET_AN · Type: MCQ · Marks: 1 · Topic(s): w1-self-attention-qkv

Compute \(W_K x_n\) and select the correct option

A. \([3, 6, -1, -2]^T\)

B. \([4, 6, -2, -2]^T\)

C. \([5, 4, 2, -1]^T\)

D. \([2, 6, -2, 2]^T\)

Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2026T2_ET_AN/q19-q19-option-a-01-p013.png shows [3, 6, -1, -2]..
Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2026T2_ET_AN/q19-q19-option-b-01-p013.png shows [4, 6, -2, -2]..
Editorial correction: the extracted text differed from the scan (extraction_error); see evidence LLM/sources/exam-papers/assets/da5004_2026T2_ET_AN/q19-q19-option-d-01-p013.png shows [2, 6, -2, 2]..

Worked answer

Final answer: B: [4, 6, −2, −2]ᵀ

Steps: Using the pictured x₃=(1,3,2,4), the rows of W_K yield (1+3,2+4,1−3,2−4)=(4,6,−2,−2), option B. The extracted second component 16 is a transcription error.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T2 End Term FN, Q2

Paper: 2026T2_ET_FN · Type: SA · Marks: 2 · Topic(s): w1-encoder-parameter-count

Consider a Multi-Head Attention sub-layer with the following hyper-parameters:

• Embedding dimension (\(d_{model}\)) = 256
• Number of heads (\(h\)) = 4
• key/value dimension per head (\(d_k = d_v\)) = \(d_{model} / h = 64\).

Calculate the total number of trainable parameters contained only in the projection matrices \(W_i^Q\), \(W_i^K\), \(W_i^V\) and \(W^O\). (Ignore biases). Enter the integer value.

Worked answer

Final answer: 262144 parameters.

Steps: For Q, K and V, 3×4×256×64 = 196608 entries; W_O has 256×256 = 65536. Total 262144; biases excluded.

Status: derived

verified_by_code: false — depends on figure, no code path

2026 T2 End Term FN, Q3

Paper: 2026T2_ET_FN · Type: SA · Marks: 2 · Topic(s): w1-encoder-parameter-count

For T5-Base, the model dimension is 768 and FFN dimension is 3072. Ignore bias. Find the number of parameters in the FFN of one encoder layer.

Worked answer

Final answer: 4718592 parameters.

Steps: Original T5 uses two FFN matrices [768,3072] and [3072,768]: 2×768×3072=4718592, ignoring bias.

Status: derived

verified_by_code: true — solve_numeric.py recomputed adjudicated inputs/conventions; all recorded values and adjudication checks MATCH.

2026 T2 End Term FN, Q4

Paper: 2026T2_ET_FN · Type: SA · Marks: 3 · Topic(s): w1-ffn-encoder-stack

A batch of 5 sentences is passed through a Transformer Encoder. The sequences are padded to a maximum length of 20 tokens. The embedding dimension (\(d_{model}\)) is 64. What is the total number of elements (volume) in the output tensor produced by the final layer of the encoder for this batch?

Worked answer

Final answer: 6400 elements.

Steps: The encoder output is [5,20,64], preserving batch, padded sequence length and hidden width: 5×20×64=6400.

Status: derived

verified_by_code: false — depends on figure, no code path