Eigenvalue Problems and

Quadratic Forms
ENGINEERING MATHEMATICS
By: Engr. Angelo Ramos
 What is EIGENVALUE

Definition
A scalar associated with a given linear transformation
of a vector space and having the property that there is
some nonzero vector which when multiplied by the
scalar is equal to the vector obtained by letting the
transformation operate on the vector; especially: a
root of the characteristic equation of a matrix.
German word – Eigenwert, eigen means “own”; wert
means “value”.
 EIGENVALUE and EIGENVECTOR

A scalar 𝜆 is called an eigenvalue of an

𝑛 x 𝑛 Matrix 𝐴 if there is a nonzero
vector 𝑥ҧ such that 𝐴𝑥ҧ = 𝜆𝑥.ҧ Such a
vector 𝑥ҧ is called an eigenvector of 𝐴
corresponding to 𝜆.
Solving Eigenvalue
SOLVING EIGENVALUE

The Process
To solve for the eigenvalues, 𝜆𝑖, and the corresponding
eigenvector, 𝑥𝑖ҧ of an 𝑛 x 𝑛 matrix 𝐴, do the following:
1. Multiply an 𝑛 x 𝑛 identity matrix by the scalar 𝜆.
2. Subtract the identity matrix multiple from the matrix 𝐴.
3. Find the determinants of the matrix and the difference.
4. Solve for the values of 𝜆 that satisfy the equation
det 𝐴 − 𝜆i = 0.
5. Solve for the corresponding eigenvector to each 𝜆.
 FINDING THE EIGENVALUES

Example: Find the Eigenvalue of the Matrix A below then

7 3 = (7 − 𝜆)(− 1 − 𝜆) −(3)(3)
𝐴= = −7 − 7𝜆+ 𝜆+ 𝜆2 − 9
3 −1
 Multiply 𝜆 by 𝑖 (identity matrix) = 𝜆2 − 6𝜆 − 16

𝜆𝑖 = 𝜆
1 0
=
𝜆 0  Solve for the values of 𝜆 that satisfy the equation
0 1 0 𝜆
det 𝐴 − 𝜆i = 0ത
 Subtract the identity matrix multiple from the matrix 𝐴
So 𝜆2 − 6𝜆 − 16 = 0
7 3 𝜆 0 7−𝜆 3
𝐴 − 𝜆𝑖 = − =
3 −1 0 𝜆 3 −1 − 𝜆 Then by factoring
 Find the determinants of the matrix and the difference. (𝜆 − 8)(𝜆 + 2) = 0
7−𝜆 3 ie:
d𝑒𝑡
3 −1 − 𝜆 𝜆=8
Eigenvalues
𝜆=−2
 FINDING THE EIGENVECTORS

 Let us use the result in 𝐴 − 𝜆𝑖 Then

7−𝜆 3 −𝑥1 + 3𝑥2 = 0
3 −1 − 𝜆 3𝑥2 = 𝑥1
At 𝜆 = 8 Let 𝑥2 = 1
7−8 3 −1 3 Let us call this 3(1) = 𝑥1
=
3 −1 − 8 3 −9 Matrix B 𝑥1 = 3
ҧ 0ത
Solve: 𝐵𝑥=
−1 3 𝑥1 0
=
3 −9 𝑥2 0 Eigenvector:
𝑥1 𝑥2
3
−1 3 0 −1 3 0 1
ቚ ቚ
3 −9 0 0 0 0
3𝑅1 + 𝑅2 𝑅2
 To check

 𝐴𝑥ҧ = 𝜆𝑥ҧ Then

7 3 3 3
where =8
3 −1 1 1
7 3
𝐴=
3 −1
(7)(3) + (3)(1) 24
=
(3)(3) + (−1)(1) 8
3
𝑥ҧ =
1 24 24
= 
8 8
𝜆=8
 Let us use 𝜆 = -2

 Let us use the result in 𝐴 − 𝜆𝑖 Then

7−𝜆 3 3𝑥1 + 𝑥2 = 0
3 −1 − 𝜆 𝑥2 = −3𝑥1
At 𝜆 = -2 Let 𝑥1 = 1
7 − (−2) 3 9 3 Let us call this 𝑥2 = −3(1)
=
3 −1 − (−2) 3 1 Matrix C 𝑥2 = −3

ҧ 0ത
Solve: C𝑥=
9 3 𝑥1 0
=
3 1 𝑥2 0 Eigenvector:
𝑥1 𝑥2 1
9 3 0 0 0 0 −3
ቚ ቚ
3 1 0 3 1 0
−3𝑅2 + 𝑅1 𝑅1
 To check

 𝐴𝑥ҧ = 𝜆𝑥ҧ Then

7 3 1 1
where = −2
3 −1 −3 −3
7 3
𝐴=
3 −1
(7)(1) + (3)(−3) −2
=
(3)(1) + (−1)(−3) 6
1
𝑥ҧ =
−3 −2 −2
= 
6 6
𝜆 = −2
Find the Eigenvalue of this 3 x 3 matrix

4 6 10
𝐴 = 3 10 13
−2 −6 −8
Sol’n
4 6 10 1 0 0 4−𝜆 6 10
Det 𝐴 − 𝜆𝑖 = 3 10 13 − 𝜆 0 1 0 = 3 10 − 𝜆 13
−2 −6 −8 0 0 1 −2 −6 −8 − 𝜆

10 − 𝜆 13 3 13 3 10 − 𝜆
= 4−𝜆 − 6 + (10)
−6 −8 − 𝜆 −2 −8 − 𝜆 −2 −6
 Continuation

Det 𝑨 − 𝝀𝒊 = 𝟒 − 𝝀 𝟏𝟎 − 𝝀 −𝟖 − 𝝀 − −𝟔 𝟏𝟑 
− 𝟔 𝟑 −𝟖 − 𝝀 − 𝟏𝟑 −𝟐   (−6)(−24 − 3𝜆 + 26)
+(𝟏𝟎) 𝟑 −𝟔 − (−𝟐)(𝟏𝟎 − 𝝀  = (−6)(2 − 3𝜆)

= −12 + 18𝜆
 (4 − 𝜆)(−80 − 10𝜆 + 8𝜆 + 𝜆2 + 78)
= (4 − 𝜆)(−2 − 2𝜆 + 𝜆2 )  (10)(−18 + 20 − 2𝜆)
= (10)(2 − 2𝜆)
= −8 − 8𝜆 + 4𝜆2 + 2𝜆 + 2𝜆2 − 𝜆3
= 20 − 20𝜆
= −𝜆3 + 6𝜆2 − 6𝜆 − 8
 Continuation

Det 𝑨 − 𝝀𝒊 =  +  + 
= −𝝀𝟑 +𝟔𝝀𝟐 − 𝟔𝝀 − 𝟖 − 𝟏𝟐 + 𝟏𝟖𝝀 + 𝟐𝟎 − 𝟐𝟎𝝀
= −𝜆3 + 6𝜆2 − 8𝜆 Equating all factors to zero
Equate to 0 Eigenvalues are:
𝜆3 − 6𝜆2 + 8𝜆 = 0
𝜆1 = 0
𝜆(𝜆2 − 6𝜆 + 8) = 0
𝜆2 = 4
𝜆(𝜆 − 4)(𝜆 − 2) = 0
𝜆3 = 2
 Other solution

4 6 10 4 6 10
𝐴 = 3 10 13 𝑆3 = 𝑑𝑒𝑡 3 10 13 = 0
−2 −6 −8 −2 −6 −8
Sol’n Then
𝜆3 − 6𝜆2 + 8𝜆 − 0 = 0
Write the characteristic equation: 𝜆3 − 𝑆1 𝜆2 + 𝑆2 𝜆 − 𝑆3 = 0
𝜆3 − 6𝜆2 + 8𝜆 = 0
Where
𝜆(𝜆2 - 6𝜆 + 8) = 0
4 6 10
𝑆1 = 4 + 10 − 8 = 6 That is from 3 10 13 𝜆(𝜆 − 4)(𝜆 − 2) = 0
−2 −6 −8
Equating all factors to zero

10 13 4 10 4 6 then
𝑆2 = + + =8 𝜆 = 0, 4, 2 𝐴𝑛𝑠.
−6 −8 −2 −8 3 10
 To solve the Eigenvector

Case 1: 𝜆 = 0
4 6 10 𝑥1
3 10 13 𝑥2 = 0 4𝑥1 + 6𝑥2 + 10𝑥3 = 0
−2 −6 −8 𝑥3 11 11
4 6 10 0 𝑥2 + 𝑥3 = 0
3 2 2
0 11Τ2 11Τ2 ቮ0 𝑅2: 𝑅2 − 4 𝑅1
𝑥2 = −𝑥3
−2 −6 −8 0
Let 𝑥3 = 1
4 6 10 0 Eigenvector
0 11Τ 11Τ ቮ0 Then 𝑥2 = −1
2 2
0 −3 −3 0
1
𝑅3 : 𝑅3 + 𝑅1
−1
2 It will follow that 𝑥1 will be 𝑥 = −1
4 6 10 0 equal to −1. 1
0 11Τ 11Τ ቮ0
2 2 6
0 0 0 0 𝑅3 : 𝑅3 + 𝑅
11 2
 To solve the Eigenvector

Case 2: 𝜆 = 4
4−4 6 10 0 6 10
3 10 − 4 13 = 3 6 13 3 6 13 0
−2 −6 −8 − 4 −2 −6 −12 0 6 10 ቮ0
1
𝑥1 0 0 0 0 𝑅3 : 𝑅3 + 𝑅2
0 6 10 3

3 6 13 𝑥2 = 0 3𝑥1 + 6𝑥2 + 13𝑥3 = 0

−2 −6 −12 𝑥3
6𝑥2 + 10𝑥3 = 0
3 6 13 0
0 6 10 ቮ0 𝑆𝑤𝑎𝑝 𝑅1 to 𝑅2 Let 𝑥3 = 1,
Eigenvector
−2 −6 −12 0 then 𝑥2 = −
5
3 −1
3 6 13 5
𝑥 = −3
0 And 𝑥1 = −1
0 6 10
10 ቮ0 2
1
0 −2 − 0 𝑅3 : 𝑅3 + 𝑅1
3 3
 To solve the Eigenvector

Case 3: 𝜆 = 2
4−2 6 10 2 6 10
3 10 − 2 13 = 3 8 13 2𝑥1 + 6𝑥2 + 10𝑥3 = 0
−2 −6 −8 − 2 −2 −6 −10 −𝑥2 − 2𝑥3 = 0
2 6 10 𝑥1
3 8 13 𝑥2 = 0 Let 𝑥3 = 1
−2 −6 −10 𝑥3 Then 𝑥2 = −2
2 6 10 0 And 𝑥1 = 1
3 Eigenvector
0 −1 −2 ቮ0 𝑅2: 𝑅2 − 2 𝑅1
−2 −6 −10 0 1
2 6 10 0 𝑥 = −2
0 −1 −2 ቮ0 1
0 0 0 0 𝑅3: 𝑅3 + 𝑅1
 EXERCISE

Ans.
Find the Eigenvalue and the Eigenvector of the matrix below. 𝜆 = 5, −1, 2

Eigenvectors
1 2 0 
A  2 2 2
1ൗ
2
𝜆 = 5: 1
0 2 3 1

2
𝜆 = −1: −2
1

−1
𝜆 = 2: − 1ൗ2
1
 SOLUTION

1 2 0 1 2 0
𝐴= 2 2 2 𝑆3 = 𝑑𝑒𝑡 2 2 2 = −10
0 2 3 0 2 3
Sol’n Then
𝜆3 − 6𝜆2 + 3𝜆 − (−10) = 0
Write the characteristic equation: 𝜆3 − 𝑆1 𝜆2 + 𝑆2 𝜆 − 𝑆3 = 0
𝜆3 − 6𝜆2 + 3𝜆 + 10 = 0
Where
1 2 0 𝐵𝑦 𝐶𝑢𝑏𝑖𝑐 𝐹𝑜𝑟𝑚𝑢𝑙𝑎
𝑆1 = 1 + 2 + 3 = 6 That is from 2 2 2
0 2 3
𝜆 = 5, −1, 2 𝐴𝑛𝑠.
2 2 1 0 1 2
𝑆2 = + + =3
2 3 0 3 2 2
 To solve the Eigenvector

Case 1: 𝜆 = 5
1−5 2 0 −4 2 0 −4𝑥1 + 2𝑥2 = 0
2 2−5 2 = 2 −3 2
0 2 3−5 0 2 −2 −2𝑥2 + 2𝑥3 = 0
−4 2 0 𝑥1
𝑥2 = 0 Let 𝑥3 = 1
2 −3 2
0 2 −2 𝑥3 Then 𝑥2 = 1
−4 2 0 0 1
1 And 𝑥1 = Eigenvector
0 −2 2 ቮ0 𝑅2 : 𝑅2 + 𝑅1 2
2
0 2 −2 0 1
2
−4 2 0 0 𝑥= 1
0 −2 2 ቮ0 1
0 0 0 0 𝑅3 : 𝑅3 + 𝑅2
 To solve the Eigenvector

Case 2: 𝜆 = −1
1 − (−1) 2 0 2 2 0
2 2 − (−1) 2 = 2 3 2 2𝑥1 + 2𝑥2 = 0
0 2 3 − (−1) 0 2 4 𝑥2 + 2𝑥3 = 0
2 2 0 𝑥1
2 3 2 𝑥2 = 0 Let 𝑥3 = 1
0 2 4 𝑥3 Then 𝑥2 = −2
2 2 0 0 And 𝑥1 = 2
0 1 2 ቮ0 Eigenvector
𝑅2 : 𝑅2 − 𝑅1
0 2 4 0 2
2 2 0 0 𝑥 = −2
0 1 2 ቮ0 1
0 0 0 0 𝑅3 : 𝑅3 − 2𝑅2
 To solve the Eigenvector

Case 3: 𝜆 = 2
1−2 2 0 −1 2 0
= 2 0 2 −𝑥1 + 2𝑥2 = 0
2 2−2 2
0 2 3−2 0 2 1 4𝑥2 + 2𝑥3 = 0
−1 2 0 𝑥1
2 0 2 𝑥2 = 0 Let 𝑥3 = 1
0 2 1 𝑥3 1
Then 𝑥2 = −
2
−1 2 0 0
0 4 2 ቮ0 𝑅2 : 𝑅2 + 2𝑅1
And 𝑥1 = −1
Eigenvector
0 2 1 0
−1
−1 2 0 0 1
𝑥 = −2
0 4 2 ቮ0
1
0 0 0 0 𝑅3 : 𝑅3 − 𝑅2 1
2
Quadratic Forms
 QUADRATIC FORMS

 A quadratic form on ℝ𝑛 is a function Q defined on ℝ𝑛 whose value at a vector

x in ℝ𝑛 can be computed by an expression of the form

𝒬 𝑥 = 𝑥 𝑇 𝐴𝑥,

where A is an 𝑛 x 𝑛 symmetric matrix.

 The matrix A is called the matrix of the quadratic form.
 Quadratic Forms

 x1 
 Example 1: Let x    . Compute xTAx for the
 x2 
following matrices.

4 0
a. A   
 0 3

 3 2 
b. A   
 2 7 
 QUADRATIC FORMS

 Solution:
4 0   x1   4 x1 
a. x Ax   x
T
1
x2        x1 x2     4 x1
2
 3 x 2
2.
0 3  x2   2
3 x

b. There are two -2 entries in A.

 3 2   x1   3 x1  2 x2 
x Ax   x1 x2  
T
     x1 x2   
 2 7  2x  2 x1
 7 x2

 x1 (3 x1  2 x2 )  x2 ( 2 x1  7 x2 )
 3 x12  2 x1 x2  2 x2 x1  7 x22
 3 x12  4 x1 x2  7 x22
 QUADRATIC FORMS

 The presence of 4x1 x2 in the quadratic form in

Example 1(b) is due to the -2 entries off the diagonal
in the matrix A.

 In contrast, the quadratic form associated with the

diagonal matrix A in Example 1(a) has no x1x2 cross-
product term.
 CHANGE OF VARIABLE IN A QUADRATIC FORM

 Example 2: Make a change of variable that

transforms the quadratic form Q(x)  x1  8 x1 x2  5 x2
2 2

into a quadratic form with no cross-product term.

 Solution: The matrix of the given quadratic form is

 1 4 
A 
 4 5
 Compute for the eigenvalue

1 −4 1 0 1−𝜆 −4
 −𝜆 = =0
−4 −5 0 1 −4 5−𝜆
 1 − 𝜆 −5 − 𝜆 − −4 −4 = 0
 −5 − 𝜆 + 5𝜆 + 𝜆2 − 16 = 0
 𝜆2 + 4𝜆 − 21 = 0
 𝜆−3 𝜆+7 =0
 𝜆 = 3, −7
Compute for the eigenvectors

 Case 1: 𝜆=3
1−3 −4 −2 −4
 = =0 Eigenvector
−4 −5 − 3 −4 −8
2
𝑥=
 −2𝑥1 − 4𝑥2 = 0 −1

 −2𝑥1 = 4𝑥2
 𝑙𝑒𝑡 𝑥2 = −1
 -2𝑥1 = 4(−1)
 𝑥1 = 2
 Compute for the eigenvectors

 Case 2: 𝜆=-7
1 − (−7) −4 8 −4 Eigenvector
 = =0
−4 −5 − (−7) −4 2 1
𝑥=
2
 8𝑥1 − 4𝑥2 = 0
 8𝑥1 = 4𝑥2
 𝑙𝑒𝑡 𝑥1 = 1
 8(1) = 4𝑥2
 𝑥2 = 2
CHANGE OF VARIABLE IN A QUADRATIC FORM

2 1
 modal matrix
−1 2

2 1 2 1
22 +(−1)2 12 +22 5 5
 −1 2 = −1 2
22 +(−1)2 12 +22 5 5
normalized modal matrix
CHANGE OF VARIABLE IN A QUADRATIC FORM
CHANGE OF VARIABLE IN A QUADRATIC FORM

2 1
𝑥1 5 5 𝑦1
 𝑥 = −1 2 𝑦2
2
5 5
2 1
 𝑥1 = 𝑦1 + 𝑦2
5 5
−1 2
 𝑥2 = 𝑦1 + 𝑦2
5 5
 CHANGE OF VARIABLE IN A QUADRATIC FORM

𝑄 𝑥 = 𝑥1 2 − 8𝑥1 𝑥2 − 5𝑥2 2

Substitute the value of 𝑥1 and 𝑥2 to the above equation

2 2
2 1 2 1 −1 2 −1 2
𝑄 𝑥 = 𝑦1 + 𝑦2 −8 𝑦1 + 𝑦2 𝑦1 + 𝑦2 − 5 𝑦1 + 𝑦2
5 5 5 5 5 5 5 5

Expanding and simplifying the above equation will give us

𝑦1
𝑄(𝑥) = 3(𝑦1 )2 − 7(𝑦2 )2 𝐶𝑎𝑛𝑜𝑛𝑖𝑐𝑎𝑙 𝐹𝑜𝑟𝑚 𝑦1 𝑦2 = 3 0
0 −7 𝑦2
.: 𝑥1 2 − 8𝑥1 𝑥2 − 5𝑥2 2 = 3𝑦1 2 − 7𝑦2 2
Classification of
Quadratic Forms
CLASSIFICATIONS OF QUADRATIC FORMS

1. Positive Definite:
(a). A quadratic form Y  x Ax  0 is positive definite if the nested principal minors
T

of A is given as
a11 a12 ... a1n
a11 a12 a13
a11 a12 a21 a22 .... a2 n
a11  0,  0, a21 a22 a23  0,........, 0
a21 a22 ... ... ... ...
a31 a32 a33
an1 an 2 ... ann

Evidently a matrix A is positive definite only if det(A)>0

(b). A quadratic form 𝑦 = 𝑥𝑇𝐴𝑥 be positive definite if all the eigenvalues of A are
positive.
CLASSIFICATIONS OF QUADRATIC FORMS

2. Positive Semi-definite:
(a) A quadratic form Y  xT Ax is positive semi-definite if the nested principal
minors of A is given as
a11 a12 ... a1n
a11 a12 a21 a22 a2 n
a11  0,  0,........, ....
0
a21 a22 ... ... ... ...
an1 an 2 ... ann

(b). A quadratic form 𝑦 = 𝑥𝑇𝐴𝑥 is positive semi-definite if at least one eigenvalue

of A is zero while the remaining roots are positive.
 CLASSIFICATIONS OF QUADRATIC FORMS

3. Negative Definite:
(a). A quadratic form Y  xT Ax is negative definite if the nested principal minors
of A are given as a a a
11 12 13
a11 a12
a11  0,  0, a21 a22 a23  0,........
a21 a22
a31 a32 a33

Evidently a matrix A is negative definite only if (-1)n× det(A)>0; where det(A) is

either negative or positive depending on the order n of A.
(b). A quadratic form 𝑦 = 𝑥𝑇𝐴𝑥 be negative definite if all the eigenvalue Roots of
A are negative.
CLASSIFICATIONS OF QUADRATIC FORMS

4. Negative Semi-definite:
(a)A quadratic form Y  xT Ax is negative semi-definite if the nested principal
minors of A is given as
a11 a12 a13
a11 a12
a11  0,  0, a21 a22 a23  0,........
a21 a22
a31 a32 a33
Evidently a matrix A is negative semi-definite only if (1) n A  0 ,that is,
det(A)≥0 ( det(A)≤0 ) when n is odd( even).
(b). A quadratic form y  xT Ax is negative semi-definite if at least one
eigenvalue of A is zero while the remaining roots are negative.
CLASSIFICATIONS OF QUADRATIC FORMS

5. Definite:

If all of the condition were not satisfied or not meet, then it is

DEFINITE.
CLASSIFICATIONS OF QUADRATIC FORMS

Example 1: Determine matrix classification in Quadratic Form

2 −1 −1
𝐴 = −1 2 −1
−1 −1 3
a11 a12 2 1
a11  2  0,   3  0,
a21 a22 1 2

a11 a12 a13 2 1 1

2 1 1 1 1 2
a21 a22 a23   1 2 1  2  ( 1)  ( 1)
1 3 1 3 1 1
a31 a32 a33 1 1 3
 2(6  1)  ( 3  1)  (1  2)  3  0
ie: the matrix is positive definite.
CLASSIFICATIONS OF QUADRATIC FORMS

Example 2: Determine matrix classification in Quadratic Form

1 2 3
𝐴= 2 2 4
3 4 9
a11 a12 1 2
a11  1  0,   2  0,
a21 a22 2 2

a11 a12 a13 1 2 3

2 4 2 4 2 2
a21 a22 a23  2 2 4 1 2 3
4 9 3 9 3 4
a31 a32 a33 3 4 9
 1(18  16)  2(18  12)  3(8  6)  4  0
ie: the matrix is definite.
CLASSIFICATIONS OF QUADRATIC FORMS

Exercise: Determine matrix classification in Quadratic Form

25 0 0
𝐴= 0 34 −12
0 −12 41
CLASSIFICATIONS OF QUADRATIC FORMS

Solution
a11 a12 25 0
a11  25  0,   850  0,
a21 a22 0 34

a11 a12 a13 25 0 0

24  12
a21 a22 a23  0 34  12  25
 12 41
a31 a32 a33 0  12 41

 25(984  144)  31,250  0

ie: the matrix is positive definite.
Generalized Eigenvalue
Problems
GENERALIZED EIGENVALUE PROBLEMS

Let 𝐴 and 𝐵 be square matrices of order 𝑛. The

g𝑒𝑛𝑒𝑟𝑎𝑙𝑖𝑧𝑒𝑑 𝑒𝑖𝑔𝑒𝑛𝑣𝑎𝑙𝑢𝑒 problem is to find eigenvalues,𝜆 , and
corresponding eigenvectors, 𝑥 ≠ 0, such that

𝐴𝑥 = 𝜆𝐵𝑥,

For given 𝐴 and 𝐵, the set of all matrices of the form 𝐴 − 𝜆𝐵 is called
a 𝑝𝑒𝑛𝑐𝑖𝑙, and 𝜆 and 𝑥 are said to be an eigenvalue and eigenvectors
of the pencil 𝐴 − 𝜆𝑩.
 GENERALIZED EIGENVALUE PROBLEMS

If 𝐴𝑥 = 𝜆𝐵𝑥 then 𝐴𝑥 − 𝜆𝐵𝑥 = 0 so 𝐴 − 𝜆𝐵 = 0

Example 1
Reduce the following quadratic forms simultaneously to canonical form
𝐽 = 𝑥 𝑇 𝐴𝑥 = 𝑥1 − 2𝑥2 + 𝑥3 2
𝐾 = 𝑥 𝑇 𝐵𝑥 = 𝑥1 2 + 2𝑥2 2 + 𝑥3 2
where
1 −2 1 1 0 0
𝐴 = −2 4 −2 , 𝐵= 0 2 0
1 −2 1 0 0 1
GENERALIZED EIGENVALUE PROBLEMS

𝐽 = 𝑥 𝑇 𝐴𝑥 = 𝑥1 − 2𝑥2 + 𝑥3 2

𝐾 = 𝑥 𝑇 𝐵𝑥 = 𝑥1 2 + 2𝑥2 2 + 𝑥3 2

1 −2 1 1 0 0
𝐴 − 𝜆𝐵 = −2 4 −2 − 𝜆 0 2 0
1 −2 1 0 0 1
GENERALIZED EIGENVALUE PROBLEMS

then
1−𝜆 −2 1
𝐴 − 𝜆𝐵 = −2 4 − 2𝜆 −2 = 0
1 −2 1−𝜆
= 1 − 𝜆 4 − 2λ 1 − 𝜆 − −2 ⋅ −2 − −2 −2 1 − 𝜆 − 1 ⋅ −2
+1 −2 ⋅ −2 − 1(4 − 2𝜆) = 0
= 1 − 𝜆 4 − 4𝜆 − 2𝜆 + 2𝜆2 − 4 + 2 −2 + 2λ + 2 + 4 − 4 + 2𝜆 = 0
= 1 − 𝜆 2𝜆2 − 6𝜆 + 4𝜆 + 2λ = 0
= 2𝜆2 − 6𝜆 − 2𝜆3 + 6𝜆2 + 6λ = 0
= 2𝜆3 − 8𝜆2 = 2(𝜆3 − 4𝜆2 ) = 0
= 𝜆3 − 4𝜆2 = 𝜆2 λ − 4 = 0
So 𝜆1 = 𝜆2 = 0, 𝜆3 = 4
 GENERALIZED EIGENVALUE PROBLEMS

Finding the eigenvectors

Case 1: 𝜆1 = 0 1 −2 1 𝑥1
1−𝜆 −2 1 0 0 0 𝑥2 = 𝑥1 − 2𝑥2 + 𝑥3 = 0
𝐴 − 𝜆𝐵 = −2 4 − 2𝜆 −2 0 0 0 𝑥3
1 −2 1−𝜆
1−0 −2 1
= −2 4 − 2(0) −2 For 𝜆1 = 0 For 𝜆2 = 0
1 −2 1−0
1 −2 1 Let Let
= −2 4 −2 𝑥1 = 1 and 𝑥2 = 0 𝑥1 = 1 and 𝑥2 = 1
1 −2 1 then 𝑥3 = −1 then 𝑥3 = 1
Applying Gaussian elimination 1 1
So 𝜆1 = 0: 0 So 𝜆2 = 0: 1
1 −2 1 0 −1 1
−2 4 −2 ቮ0 𝑅2 : 𝑅2 + 2𝑅1
1 −2 1 0 𝑅3 : 𝑅3 − 𝑅1
 GENERALIZED EIGENVALUE PROBLEMS

Case 2: 𝜆3 = 4
1−4 −2 1
𝐴 − 𝜆𝐵 = −2 4 − 2(4) −2
1 −2 1−4 −3 −2 1 𝑥1 −3𝑥1 − 2𝑥2 + 𝑥3 = 0
−3 −2 1 8 8
0 − ൗ3 − ൗ3 𝑥2 = 8
− ൗ 𝑥 − 8ൗ 𝑥 = 0
= −2 −4 −2 0 0 0 𝑥3 3 2 3 3
1 −2 −3
Applying Gaussian elimination So 𝑥3 = −𝑥2
Let 𝑥3 = 1, then 𝑥2 = −1
−3 −2 1 0
−2 −4 −2 ቮ0 𝑅2 : 𝑅2 − 2Τ3 𝑅1 and also 𝑥1 will be = 1
1 −2 −3 0 𝑅3 : 𝑅3 + 1Τ3 𝑅1
1
−3 −2 1 So 𝜆3 = 4: −1
0
0 − 8ൗ3 8
− ൗ3 ቮ0 1
1 − 8ൗ3 − 8ൗ3 0 𝑅3 : 𝑅3 − 𝑅2
GENERALIZED EIGENVALUE PROBLEMS

Associated B-orthonormal eigenvectors

are The B-orthogonal matrix is thus

2 2
1 1
𝑥1 = 1, 0, −1 𝑇 2
2 2
2
1 1
1 𝑄= 0 −
𝑥 = 1, 1, 1 𝑇
2 2 2
2
2 1 1
1 −
𝑥 = 1, −1, 1 𝑇
3 2 1 2
2
 GENERALIZED EIGENVALUE PROBLEMS

The transformation from 𝑥 to 𝑦 is then

2 2
0 −
2 2 1 0 0 𝑥1
1 1 1
𝑦 = 𝑄𝑇 𝐵𝑥 = 0 2 0 𝑥2
2 2 2 0 0 1 𝑥3
1 1 1
−
2 2 2
𝑇
2 1 1
= 𝑥1 − 𝑥3 , 𝑥1 + 𝑥2 + 𝑥3 , (𝑥 − 2𝑥2 + 𝑥3 )
2 2 2 1

and the canonical form of 𝐽 and 𝐾 are

𝑱 = 𝟒𝒚𝟑 𝟐 = 𝒙𝟏 − 𝟐𝒙𝟐 + 𝒙𝟑 𝟐

𝑲 = 𝒚𝟏 𝟐 + 𝒚𝟐 𝟐 + 𝒚𝟑 𝟐 = 𝒙𝟏 𝟐 + 𝟐𝒙𝟐 𝟐 + 𝒙𝟑 𝟐
Minimum Principles for
Matrix Equations
 MINIMUM PRINCIPLES FOR MATRIX EQUATIONS

Consider the equation

𝑨𝒙 = 𝒄
where the matrix 𝐴 is symmetric and positive definite. If the vector 𝒙 is a displacement and
the vector 𝒄 is the corresponding applied force, then the virtual work done by force 𝒄, acting
through a virtual displacement is
𝑾 = 𝟏ൗ𝟐 𝒙𝑻 𝑨𝒙
Since the work done by the applied forces in causing displacement of a stable mechanical
system should be positive if the displacement is different from zero,
𝑾 = 𝟏ൗ𝟐 𝒙𝑻 𝑨𝒙 > 𝟎
 MINIMUM PRINCIPLES FOR MATRIX EQUATIONS

Minimum Functional Theorem

Let 𝑨 be a symmetric positive definite matrix. Then 𝒙∗ is the solution of 𝑨𝒙 = 𝒄 if and only if
𝒙∗ minimizes
𝑭 𝒙 = 𝒙𝑻 𝑨𝒙 − 𝟐𝒙𝑻 𝒄
To prove the Minimum Functional Theorem, first let 𝒙∗ be the solution of 𝑨𝒙 = 𝒄.
Substituting 𝒄 = 𝑨𝒙∗ into the above equation and performing an algebraic manipulation.
𝑭 𝒙 = 𝒙𝑻 𝑨𝒙 − 𝟐𝒙𝑻 𝑨𝒙∗ = 𝒙 − 𝒙∗ 𝑻 𝑨 𝒙 − 𝒙∗ − 𝒙∗𝑻 𝑨𝒙∗
 MINIMUM PRINCIPLES FOR MATRIX EQUATIONS

Example
Let a symmetric positive definite matrix 𝐴 and a vector 𝑐 be given as
𝟐 𝟏 𝟏 𝟏
𝑨= 𝟏 𝟑 𝟐 , 𝒄= 𝟐
𝟏 𝟐 𝟑 𝟏
Then the function of equation is
𝑭 𝒙 = 𝒙 − 𝒙∗ 𝑻 𝑨 𝒙 − 𝒙∗ − 𝒙∗𝑻 𝑨𝒙∗
𝟐 𝟐
= 𝒙𝟏 − 𝟏ൗ𝟒 + 𝒙𝟐 − 𝟑ൗ𝟒 + 𝒙𝟏 − 𝟏ൗ𝟒 + 𝒙𝟑 + 𝟏ൗ𝟒
𝟐
+ 𝟐 𝒙𝟏 − 𝟑ൗ𝟒 + 𝒙𝟑 + 𝟏ൗ𝟒 − 𝟑ൗ𝟐
The vector 𝑥 that minimizes 𝐹(𝑥), hence the solution of 𝐴𝑥 = 𝑐, is obtained as

𝒙𝟏 = 𝟏ൗ𝟒 , 𝒙𝟐 = 𝟑ൗ𝟒 , 𝒙𝟑 = 𝟏ൗ𝟒

 MINIMUM PRINCIPLES FOR MATRIX EQUATIONS

Check by Gauss Jordan elimination

𝟐 𝟏 𝟏 𝒙𝟏 𝟏 1ൗ 1ൗ 1ൗ 1ൗ 1ൗ 1
1 1 0 5 5
𝟏 𝟑 𝟐 𝒙𝟐 = 𝟐 2 2 2 𝑅1 : 𝑅1 − 𝑅2
2
𝟏 𝟐 𝟑 𝒙𝟑 𝟏 0 5ൗ2 3ൗ ተ 3ൗ 𝑅2 : 𝑅2 − 𝑅1 0 1 3ൗ5 3ൗ5
2 2
2𝒙𝟏 +𝒙𝟐 +𝒙𝟑 = 𝟏 1 2 3 1 0 3ൗ 5ൗ 1ൗ
2 2 2
𝒙𝟏 + 𝟑𝒙𝟐 + 𝟐𝒙𝟑 = 𝟐 1 1ൗ2 1ൗ 1ൗ
2 2 1 0 1ൗ5 1ൗ5
𝒙𝟏 + 𝟐𝒙𝟐 + 𝟑𝒙𝟑 = 𝟏 0 5ൗ2 3ൗ 3ൗ
2 2 0 1 3ൗ5 3ൗ5
2 1 1 1 0 3ൗ2 5ൗ 1ൗ 𝑅3 : 𝑅3 − 𝑅1 3
2 2 0 0 8ൗ5 − 2ൗ5 𝑅3 : 𝑅3 − 𝑅2
2
1 3 2อ 2
1 2 3 1 1 1ൗ2 1ൗ 1ൗ
2 2 1ൗ
1 0 ൗ51 5
0 1 3ൗ 3ൗ 2
1 1ൗ2 1ൗ2 1ൗ2
1 𝑅2 : 𝑅2
𝑅1 : 𝑅1 5 5 3
1 3ൗ5ተ ൗ5
2 5
0
1 3 2 ቮ 2 0 3ൗ2 5ൗ 1ൗ
2 2 5
1 2 3 1 0 0 1 − 1ൗ4 𝑅3 : 𝑅3
8
MINIMUM PRINCIPLES FOR MATRIX EQUATIONS

1ൗ 1
1 0 0 4 𝑅1: 𝑅1 − 5 𝑅3 Then
0 1 3ൗ ቮ 3ൗ 𝟏
5 5 𝒙𝟏 =
0 0 1 − 1ൗ 𝟒
4
𝟑
1ൗ 𝒙𝟐 =
1 0 0 4 𝟒
3
0 1 0อ 3ൗ4 𝑅2: 𝑅2 − 5 𝑅3 𝟏
0 0 1 1 𝒙𝟑 = −
− ൗ4 𝟒
 MINIMUM PRINCIPLES FOR MATRIX EQUATIONS

RITZ METHOD
The dimension of the vector 𝒙 in 𝑨𝒙 = 𝒄 be 𝑛 and let 𝒙𝟏 , 𝒙𝟐 , . . . , 𝒙𝒎 be linearly independent
vectors in 𝑹𝒏 , where 𝒎 > 𝒏. Consider an approximation to the solution 𝒙 of the form.
𝒙 ≈ 𝒂𝟏 𝒙𝟏 + 𝒂𝟐 𝒙𝟐 + . . . +𝒂𝒎 𝒙𝒎 = 𝒂𝒊 𝒙𝒊
Substituting the above equation to 𝑭 𝒙 = 𝒙𝑻 𝑨𝒙 − 𝟐𝒙𝑻 𝒄
𝒊 𝒊 𝑻 𝒋 𝒊 𝑻
𝒇 𝒂 ≡ 𝑭 𝒂𝒊 𝒙 = 𝒂𝒊 𝒙 𝑨 𝒂𝒋 𝒙 − 𝟐 𝒂𝒊 𝒙 𝒄
𝑻
= 𝒂𝒊 𝒂𝒋 𝒙𝒊𝑻 𝑨𝒙𝒋 − 𝟐𝒂𝒊 𝒊
𝒙 𝒄
 MINIMUM PRINCIPLES FOR MATRIX EQUATIONS

In order for 𝑓(𝑎) to be minimum with respect to the coefficients 𝑎𝑖 , it is necessary that the
derivative of this function with respect to each component 𝑎𝑖 be equal to 0, which results in
the following equations:
𝝏𝒇(𝒂)
= 𝟎 = 𝒂𝒋 𝒙𝒌𝑻 𝑨𝒙𝒋 + 𝒂𝒊 𝒙𝒊𝑻 𝑨𝒙𝒌 − 𝟐𝒙𝒌𝑻 𝒄, 𝒌 = 𝒍, . . . . , 𝒎
𝝏𝒂𝒌

or, noting that 𝒙𝒌𝑻 𝑨𝒙𝒋 = 𝒙𝒋𝑻 𝑨𝒙𝒌 , in 𝐴-scalar product notation as
𝒙𝒌 , 𝒙𝒋 𝑨𝒂𝒋 = 𝒙𝒌𝑻 𝑨𝒙𝒋 𝒂𝒋 = 𝒙𝒌𝑻 𝒄, 𝒌 = 𝒍, . . . . , 𝒎
 MINIMUM PRINCIPLES FOR MATRIX EQUATIONS

MINIMIZATION OF RAYLEIGH QUOTIENT

For a positive definite matrix 𝑨, the Rayleigh quotient is defined for vectors 𝒙 ≠ 𝟎 as
𝒙𝑻 𝑨𝒙
𝑹(𝒙) ≡ 𝑻
𝒙 𝒙

Let the Rayleigh quotient take on its minimum value 𝜆 at some 𝑥 = 𝑥 ∗ ≠ 0; i.e.,
𝒙∗𝑻 𝑨𝒙∗ 𝒙𝑻 𝑨𝒙
𝟎 < 𝝀 ≡ ∗𝑻 ∗ ≤ 𝑻
𝒙 𝒙 𝒙 𝒙
 MINIMUM PRINCIPLES FOR MATRIX EQUATIONS

For all 𝑥 ≠ 0 in 𝑅𝑛 . Let 𝑦 be any vector in 𝑅𝑛 and form the function

𝒙 ∗ + 𝝉𝒚 𝑻 𝑨 𝒙∗ + 𝝉𝒚
𝑹 𝒙∗ + 𝝉𝒚 =
𝒙∗ + 𝝉𝒚 𝑻 𝒙∗ + 𝝉𝒚
where 𝜏 is a real variable. This function takes on its minimum at 𝜏 = 0, so its necessary that

𝒅𝑹 𝟐 𝒙∗𝑻 𝒙∗ 𝒙∗𝑻 𝑨𝒚 − 𝟐 𝒙∗𝑻 𝑨𝒙∗ 𝒙∗𝑻 𝒚

=𝟎=
𝒅𝝉 𝝉=𝟎 𝒙∗𝑻 𝒙∗ 𝟐
then 𝒚𝑻 𝑨𝒙∗ − 𝝀𝒙∗ = 𝟎
and 𝑨𝒙∗ = 𝝀𝒙∗