Course: Machine Learning - Foundations

Test Questions - Solution

Lecture Details: Week 6

1. (1 point) The function f (x, y) = x2 + y 2

A. has no stationary point.
B. has a stationary point at (0, 0).
C. has a stationary point at (1, 1).
D. has a stationary point at (-1, -1).

Answer: B
f (x, y) = x2 + y 2 ,
∂f ∂f
fx = ∂x
= 2x, fy = ∂y
= 2y
Since, fx , fy are 0 at (0, 0). The origin is an stationary point for the function.
   
4 2 1 1
2. (1 point) If A = and B = then A + B is a positive definite matrix. Is
2 2 1 2
this statement true?
A. Yes, it is true
B. No, it is not true

Answer: A
 
5 3
A+B = ,
3 4
Here a = 5, b = 3, c = 4,
Since a > 0, ac − b2 = 5 ∗ 4 − 32 = 11 > 0, A + B is a positive definite matrix.
 
2 −1 1
3. (2 points) The matrix A = −1 2 −1 is
1 −1 2
A. positive definite
B. positive semi-definite
C. negative definite
D. negative semi-definite

Answer: A
Characteristics equation:
Course: Machine Learning - Foundations Page 2 of 7
 
2 − λ −1 1
 −1 2 − λ −1  I = 0
1 −1 2 − λ
The determinant of the obtained matrix is −(λ − 1)2 (λ − 4) = 0
Eigenvalues:
The roots are λ1 = 4, λ2 = 1, λ3 = 1
The eigenvalues are 4, 1, 1. Since, all eigenvalues are positive for the matrix, it is a
positive definite matrix.
4. (1 point) The function f (x, y) = 2x2 + 2xy + 2y 2 − 6x has a stationary point at
A. (2, 1)
B. (1, 2)
C. (-1, 2)
D. (2, -1)

Answer: D
f (x, y) = 2x2 + 2xy + 2y 2 − 6x,
∂f ∂f
fx = ∂x
= 4x + 2y − 6, fy = ∂y
= 2x + 4y
Since, fx , fy are 0 at (2, -1). The origin is an stationary point for the function.
5. (1 point) The correct representation of x2 + y 2 − z 2 − xy + yz + xz in the matrix form
is   
  1 −1 0 x
A. x y z 0 1  0   y
1 1 −1 z
  
  1 −1 0 x
B. x y z 0 −1 0
   y
1 1 −1 z
  
  1 −1 0 x
C. x y z 0 1 0  y 
1 −1 −1 z
  
  −1 −1 0 x
D. x y z  0 1 0  y 
1 1 −1 z

Answer: A
Perform matrix multiplication of two adjacent matrix at a time and check which option
is giving function described in the question.
    
  1 −1 0 x   x
x y z 0 1 0  y  = x + z −x + y + z −z y  = x2 +y 2 −z 2 −xy+yz+xz
1 1 −1 z z
Course: Machine Learning - Foundations Page 3 of 7

6. (2 points) Given f (x, y) = 3x2 + 4xy + 2y 2 , the point (0, 0) is a .

A. maxima
B. minima
C. saddle Point
D. None of these

Answer: B
f (x, y) = 3x2 + 4xy + 2y 2 ,
∂f ∂f
fx = ∂x
= 6x + 4y, fy = ∂y
= 4x + 4y
Since, fx , fy are 0 at (0, 0). The origin is an stationary point for the function.
∂f 2 ∂f 2 ∂f
fxx = ∂x2
= 6, fxy = ∂x∂y
= 4, fyy = ∂y 2
=4
2
Since fxx > 0, D = fxx fyy − fxy = 6 ∗ 4 − (4)2 = 8 > 0, the point (0, 0) is a minima.

7. (1 point)
 (Multiple
 select) Which of the following statements are true about the matrix
4 2
A= ?
2 3
A. A is positive definite.
B. A is positive semi-definite.
C. A is neither positive definite nor positive semi-definite.
D. Can not be determined.

Answer: A
a = 4, b = 2, c = 3, Since a > 0, ac − b2 = 4 ∗ 3 − 22 = 8 > 0, the matrix is positive
definite.
 
1 2
8. (1 point) The matrix A = is positive definite
2 1
A. True
B. False

Answer: B
a = 1, b = 2, c = 1 Since, ac − b2 = 1 ∗ 1 − 22 = −3 is a negative number, the matrix is
not a positive definite matrix.

9. (1 point)
 (Multiple
 select) Which of the following statement is true about the matrix
3 0 0
A = 0 5 0?

0 0 7
Course: Machine Learning - Foundations Page 4 of 7

A. A is positive definite.
B. A is positive semi-definite.
C. A is neither positive definite nor positive semi-definite.
D. Can not be determined

Answer: A
This is a diagonal matrix. For a diagonal matrix, the eigenvalues are elements on its
principal diagonal. Since all the eigenvalues (diagonal elements) are positive, the matrix
is a positive definite matrix.
Questions 10-15 are based on common data
Consider these data points to answer the following questions:
     
0 1 2
x1 = 1 , x2 = 1 , x3 = 1
    
2 1 0

10. (1 point) The mean vector of the data points x1 , x2 , x3 is

 
0
A. 0

0
 
1
B. 1
1
 
0.9
C. 0.6

0.3
 
0.5
D. 0.5

0.5

Answer: B
   
(0 + 1 + 2) 1
n 1 1
Mean vector = X̄ = Σi=1 n xi = 3 (1 + 1 + 1) = 1
 
(2 + 1 + 0) 1

11. (2 points) The covariance matrix C = n1 Σni=1 (xi − x̄)(xi − x̄)T of the data points x1 , x2 , x3
is
 
0 0 0
A. 0 0 0
0 0 0
Course: Machine Learning - Foundations Page 5 of 7
 
033 0 0.33
B.  0 0 0 
0.33 0 0.33
 
0.67 0 −0.67
C.  0 0 0 
−0.67 0 0.67
 
2.88 0 1.44
D.  0 0 0 
1.44 0 2.88

Answer: C
       
−1  0 1  2 0 −2
1  1
   
C = 3( 0  −1 0 1 + 0 0 0 0 + 0  1 0 −1 ) =
  
3
0 0 0 =
 1  0 −1 −2 0 2
0.67 0 −0.67
 0 0 0 
−0.67 0 0.67

12. (2 points) The eigenvalues of the covariance matrix C = n1 Σni=1 (xi − x̄)(xi − x̄)T are
(Note: The eigenvalues should be arranged in the descending order from left to right.)
A. 2.66, 1.18, 0
B. 1.22, 0.74, 0.34
C. 1.34, 0, 0
D. 0.56, 0.33, 0.14

Answer: C
Characteristics equation:
 67 67

100
−λ 0 − 100
 0 −λ 0 I = 0
67 67
− 100 0 100 − λ
The determinant of the obtained matrix is λ2 ( 67
50
− λ) = 0
Eigenvalues:
67
The roots are λ1 = 50
= 1.34, λ2 = 0, λ3 = 0
Eigenvectors:
 67 67
  67 67

100
−λ 0 − 100 − 100 0 − 100
λ1 = 1.34,  0 −λ 0 = 0 − 67
50
0 
67 67 67 67
− 100 0 100
−λ − 100 0 − 100
Course: Machine Learning - Foundations Page 6 of 7
   
−1 −1
1 
The null space of this matrix is 0 , Corresponding eigenvector is, u1 = 2 0  =
  √

  1 1
−0.71
 0 
0.71
 67 67
  67 67

100
−λ 0 − 100 100
0 − 100
λ2 = λ3 = 0,  0 −λ 0 = 0 0 0 
67 67 67 67
− 100 0 100 − λ − 100 0 100
   
0 1
The null space of this matrix are 1, and 0 Corresponding eigenvector are, u2 =
      0 1
0 1 0.71
1, u3 = √1 0 =  0 
2
0 1 0.71

13. (2 points) The eigenvectors of the covariance matrix C = n1 Σni=1 (xi − x̄)(xi − x̄)T are
(Note: The eigenvectors should be arranged in the descending order of eigenvalues from
left to right in the matrix.)
 
1 0 1
A. 0 1 0
1 0 1
 
0.71 0 1
B.  0 0.71 0
0.71 0.71 0
 
−0.71 0 0.71
C.  0 1 0 
0.71 0 0.71
 
0.33 0 0
D. 0.33 1 0
0.34 0 1

Answer: C
Refer to the solution of the previous question.

14. (2 points) The data points x1 , x2 , x3 are projected onto the one dimensional space using
PCA as points z1 , z2 , z3 respectively. (Use eigenvector with the maximum eigenvalue for
this projection.)
     
1 1 1
A. z1 = 1 , z2 = 1 , z3 = 1
    
1 1 1
Course: Machine Learning - Foundations Page 7 of 7
     
0.5 0 −0.5
B. z1 = 0.5, z2 = 0, z3 = −0.5
0.5 0 −0.5
     
0 1 2
C. z1 = 2 , z2 = 1 , z3 = 2
    
2 1 0
     
−1 0 1
D. z1 =  0 , z2 = 0, z3 =  0 
1 0 −1

Answer: D
 
−1
1
λ1 = 1.4, u1 = √
2
 0
1
     
−1 −1 −1
√1 ( 2  0 ) √12  0  =  0 
 
z1 = 2
0 1
1 1 1
     
 −1 1 −1 0
√1 (

z2 = 2
1 1 1  0 ) 2 0 = 0
 √   
1 1 0
     
 −1 1 −1 1
√1 (

z3 = 2
2 1 0  0 ) √2 0 = 0 
   
1 1 −1

15. (1 point) The approximation error J on the given data set is given by Σni=1 ||xi − zi ||2 .
What is the reconstruction error?
A. 3
B. 5
C. 10
D. 20

Answer: A
Approximation Error, J = n1 Σni=1 ||xi − zi ||2 = 31 [(12 + 12 + 12 ) + (12 + 12 + 12 ) + (12 +
12 + 12 )] = 3