Lecture 3: Matrix and Matrix Operations

• Representation, row vector, column vector, element of a matrix.

• Examples of matrix representations – Tables and spreadsheets
• Scalar-Matrix operation: Scaling a matrix
• Vector-Matrix operation: Multiplication of matrix and a vector.
• Computation as dot product between vectors
• Interpreting as linear combination of column vectors
• Properties of matrix-vector multiplication
• Matrix-Matrix operations
• Multiplication
• Example: Feedback control systems
• Example: Transitive closure
• Addition of matrices
• Transpose of a matrix
• Special Matrices
• Square Matrix
• Identity Matrix
• Diagonal Matrix
• Symmetric Matrix
• Anti-symmetric Matrix
• Upper/Lower Triangular Matrices
• Orthogonal Matrix

1
Matrix Representation
• Matrices can be assumed as a sequence of vectors of the same dimension.
• Representation is similar to a 2D array in programming.
• Example:
2 −1 0.5
𝒂1 = −1 , 𝒂2 = 1 , 𝒂3 = 5 , 𝒃1 = 2 1 , 𝒃2 = 0 1
0 −2 −1

2 −1 0.5
• 𝐴 = 𝒂1 𝒂2 𝒂3 = −1 1 5 (using Column vectors)
0 −2 −1
𝒃1 2 1
• 𝐵= = (using Row vectors)
𝒃2 0 1

2
Size of a Matrix
• Size of a matrix is represented as (# of rows)×(# of columns)
• Example:
2 −1 0.5
• 𝐴 = −1 1 5 is a 3 × 3 matrix
0 −2 −1

2 2
• 𝐵 = −1 1 is a 3 × 2 matrix
0 0
• 𝐴𝑖,𝑗 is the component of 𝐴 in the 𝑖-th row and 𝑗-th column.
• In above example 𝐴3,2 = −2
3
Matrix representation of vectors
• Vectors can also be considered as matrices.
• Row vectors are 1 × 𝑛 matrices, where 𝑛 is the number of
components.
• Column vectors are 𝑛 × 1 matrices, where 𝑛 is the number of
components.
• Example:
2
• Column vector −1 is a 3 × 1 matrix.
0
• Row vector 2 1 is 1 × 2 matrix.

4
When are two matrices equal?
• Two matrices are equal if their sizes and corresponding elements are
equal.

• Example:
4 1
2 −1 0.5 −1
2 2
• −1 1 5 = −1 1 5 , both are 3 × 3 matrices.
0 −2 −1 1 − 1 −2 −1

5
Scalar - Matrix Operations
• Scalars can be multiplied with matrices.
• Similar to scalar-vector multiplication, the scalar is multiplied with
every element of the matrix
• Example:
2 3 4 6
•2 2 1 = 4 2
0 −1 0 −2

2 0 −2 0
• −1 =
0 −2 0 2

6
Vector and Matrix Operations
A vector can be multiplied with a matrix in two possible ways. i.e.
Matrix-Vector multiplication or Vector-Matrix multiplication.

Matrix-Vector Multiplication: 𝐴𝑏 = 𝑐
• Matrix 𝑨𝑚×𝑛 can be multiplied with column vector 𝒃𝑛×1 to get a column vector
𝒄𝑚×1 .

Vector-Matrix Multiplication: 𝑎𝐵 = 𝑐
• Row vector 𝒂1×𝑛 can be multiplied with matrix 𝐵𝑛×𝑚 to get a row vector 𝒄1×𝑚 .

7
Matrix-Vector Multiplication
• Matrix 𝐴𝑚×𝑛 can be multiplied with column vector 𝒃𝑛×1 to get a
column vector 𝒄𝑚×1 .
• 𝑖-th component of 𝒄 is dot product of 𝑖-th row of 𝐴 with 𝒃.

𝑏1
𝒂𝟏 𝑎11 𝑎12 𝑎13 𝒂𝟏 ∙ 𝒃
𝐴𝒃 = 𝒂𝟐 𝑎21 𝑎12 𝑎13 𝑏2 =
𝒂𝟐 ∙ 𝒃
𝑏3

8
Matrix-Vector Multiplication
• Number of columns of the matrix and the dimension (number of
components) of the column vector must be equal.

2 3 2
1
2 1 = 2
0
0 −1 0

• What if the column vector is orthogonal to the rows of the matrix?

• Obviously, the result is a zero vector, since the dot product of orthogonal vectors is
zero.

2 1 1 0
=
−4 −2 −2 0

9
Vector-Matrix Multiplication
• Row vector 𝒂1×𝑛 can be multiplied with matrix 𝐵𝑛×𝑚 to get a row
vector 𝒄1×𝑚 .
• 𝑖-th component of 𝒄 is dot product of 𝒂 and the 𝑖-th column of 𝐵.
𝒃1 𝒃2

𝑏11 𝑏12
𝒂𝐵 = 𝑎1 𝑎2 𝑎3 𝑏21 𝑏22 = 𝒂 ∙ 𝒃1 𝒂 ∙ 𝒃2
𝑏31 𝑏32

10
Vector-Matrix Multiplication
• Number of components of the row vector and number of rows of the
matrix must be equal.

2 2 0
1 0 = 2 2 0
3 1 −1

• What if the row vector is orthogonal to columns of matrix?

• The result is a zero vector.

2 −4
1 −2 = 0 0
1 −2

11
Matrix - Matrix Operations
• Matrix-Matrix Multiplication
𝑨𝑩

• Matrix-Matrix Addition
𝑨+𝑩

• Matrix-Matrix Subtraction
𝑨−𝑩

12
Matrix-Matrix Multiplication
• Two matrices 𝑨𝑛×𝑚 and 𝑩𝑘×𝑙 can be multiplied as 𝑨𝑩, only if 𝑚 = 𝑘
• The result is a matrix of the size of 𝑛 × 𝑙 (𝑪𝑛×𝑙 )

• Example:
1 0
2 −1 2
• −1 1 is possible
1 0 3
0 −1

2 −1 2 1 0
• is NOT possible
1 0 3 0 1

13
Matrix-Matrix Multiplication
• Multiplication of matrices 𝑨𝑩 is nothing but the dot products of row
vectors of 𝑨 with the column vectors of 𝑩
• The component (𝑖, 𝑗) of the resulting matrix is the result of the dot
product of 𝑖-th row vector of 𝑨 with the 𝑗-th column vector of 𝑩.
• If 𝑨𝑛×𝑚 and 𝑩𝑚×𝑙 , so 𝑨 has 𝑛 row vectors and 𝑩 has 𝑙 column
vectors. Therefore, the result matrix is 𝑛 × 𝑙.

14
Matrix-Matrix Multiplication (as dot product)
𝒃1 𝒃2

𝑏11 𝑏12
𝒂1 𝑎11 𝑎12 𝑎13 𝒂1 ∙ 𝒃1 𝒂1 ∙ 𝒃2
𝑨𝑩 = 𝒂2 𝑎21 𝑎12 𝑎13 𝑏21 𝑏22 =
𝒂2 ∙ 𝒃1 𝒂2 ∙ 𝒃2
𝑏31 𝑏32

15
Matrix-Matrix Multiplication (as linear combination)

Each Column of AB is a linear combination of the

columns of A using weights from the corresponding
column of B.

16
Properties of Matrix Multiplication
• AB ≠ BA -- not commutative
• A(BC) = (AB)C -- associative law
• A(B+C) = AB + AC -- left distributive law
• (B+C)A = BA + CA -- right distributive law
• r(AB) = (rA)B = A(rB) for any scalar r
• IA = AI = A
• (AB)T = BTAT

17
Inner and Outer Products
• Inner Product: Inner product of two vectors results in a scalar.
−2
𝑎𝑇𝑏 = 1 0 −1 1 = −2 + 0 − 2 = −4
2

• Outer Product: Outer product of two vectors results in a matrix.

1 −2 1 2
𝑎𝑏𝑇 = 0 −2 1 2 = 0 0 0
−1 2 −1 −2

18
Matrix Addition
• Two matrices can be added only if they have the same dimension
• In matrix addition corresponding elements of the matrices are added.
• Example:

2 3 −1 0 1 3
• 2 1 + 2 1 = 4 2
0 −1 1 2 1 1

1 0.5
2 0
• 1 0 + Cannot be added
0 −2
1 −1
19
Properties of Matrix Addition
• A+B = B+A -- commutative law
• A + (B + C) = (A + B) + C -- associative law
• A+0=0+A=A
•A–A=0
• (A + B)T = AT + BT

20
Matrix Subtraction
• Two matrices can be subtracted only if they have the same dimension
• In matrix subtraction corresponding components of matrices are
subtracted.
• Example:

2 3 −1 0 3 3
• 2 1 − 2 1 = 0 0
0 −1 1 2 −1 −3
1 0.5
2 0
• 1 0 − Cannot be subtracted
0 −2
1 −1
21
Matrix Transpose
• Transpose of a matrix is a matrix where its rows are columns of the
original matrix. (And its columns are the rows of the original matrix.)

Row 1

Col 2
Col 1
Row 2

⋮
⋮
A AT

• Transpose converts row vectors to column vectors and vice-versa

22
Matrix Transpose
• Transpose of matrix 𝐴 is noted as 𝐴𝑇 (NOT A to the power of 𝑇).
• If size of 𝐴 is 𝑚 × 𝑛 ,then size of 𝐴𝑇 is 𝑛 × 𝑚.
• Component (𝑖, 𝑗) in original matrix is the component (𝑗, 𝑖) in the
transpose matrix.
• Example:
2 2 2 −1 0
• 𝐴 = −1 𝑇
1 ,𝐴 = 2 1 0
0 0

23
Properties of Matrix Transpose
• Transpose of transpose of a matrix is the matrix itself
• (𝐴𝑇 )𝑇 = 𝐴
• Superposition property
• (𝐴 + 𝐵)𝑇 = 𝐴𝑇 + 𝐵𝑇
• Scaling property
• (𝑐𝐴)𝑇 = 𝑐𝐴𝑇
• Transpose of a product of matrices equals the product of their
transpose in the reverse order
• (𝐴𝐵)𝑇 = 𝐵𝑇 𝐴𝑇
24
Special Matrix
• Square Matrix
• Symmetric Matrix
• Anti-Symmetric Matrix
• Diagonal Matrix
• Identity Matrix
• Upper/Lower Triangular Matrix
• Orthogonal Matrix

25
Square matrix
• Square matrix is a matrix whose number of rows is equal to its
number of columns.
• Square matrix 𝐴 is a 𝑛 × 𝑛 matrix.
• Example:
2 −1 0.5
• −1 1 5 is a square 3 × 3 matrix.
0 −2 −1

2 1
• is a square 2 × 2 matrix.
0 1

26
Symmetric matrix
• A symmetric matrix is a square matrix which is equal to its transpose.
• 𝐴 = 𝐴𝑇 implies 𝐴 is a symmetric matrix.
• Components (𝑖, 𝑗) and (𝑗, 𝑖) of the symmetric matrix are equal.
(𝐴𝑖,𝑗 = 𝐴𝑗,𝑖 )
• Example:
2 0
•
0 1
2 −1 0.5
• −1 1 5
0.5 5 −1

27
Antisymmetric matrix
• An antisymmetric matrix is a square matrix which is equal to its transpose
multiplied by −1.
• 𝐴 = −𝐴𝑇 if 𝐴 is an antisymmetric matrix.
• Components (𝑖, 𝑗) and (𝑗, 𝑖) of an antisymmetric matrix are equal in value
and different in sign. (𝐴𝑖,𝑗 = −𝐴𝑗,𝑖 )
• Example:
0 1
•
−1 0
0 −1 0.5
• 1 0 −5
−0.5 5 0

28
Diagonal matrix
• A diagonal matrix is a square matrix whose non-diagonal components
are all zero.
• Diagonal matrices are symmetric, as well.
diagonal
• Example: components
2 0
•
0 1
0
2 0 0
• 0 1 0 0
0 0 −1

29
Identity matrix
• Identity matrix is a diagonal matrix whose diagonal values are all 1.
• Identity matrix is denoted by 𝐼.
• Example:
1 0
• is a 2 × 2 identity matrix
0 1

1 0 0
• 0 1 0 is a 3 × 3 identity matrix.
0 0 1

30
Upper/Lower triangular matrix
• Upper triangular matrix is a square matrix whose elements below the
diagonal are all 0.
• Lower triangular matrix is a square matrix whose elements above the
diagonal are all 0.
• Example:
2 −1 0.5
• 0 1 5 is an upper triangular matrix
0 0 −1
2 0 0
• −1 1 0 is a lower triangular matrix
0.5 5 −1

31
Orthogonal/Orthonormal Matrix
• Matrix 𝐴 is an orthonormal matrix if it is a square matrix and
• 𝐴𝐴𝑇 = 𝐴𝑇 𝐴 = 𝐼, where 𝐼 is an identity matrix.

• Every row is orthogonal to every other row, and every column is orthogonal
to every other column. Every row and every column is a unit vector.

𝑎11 𝑎12 𝑎13 𝑎11 𝑎21 𝑎31 𝒂1 ∙ 𝒂1 𝒂1 ∙ 𝒂2 𝒂1 ∙ 𝒂3 1 0 0

𝐴𝐴𝑇 = 𝑎21 𝑎22 𝑎23 𝑎12 𝑎22 𝑎32 = 𝒂2 ∙ 𝒂1 𝒂2 ∙ 𝒂2 𝒂2 ∙ 𝒂3 = 0 1 0
𝑎31 𝑎32 𝑎33 𝑎13 𝑎23 𝑎33 𝒂3 ∙ 𝒂1 𝒂3 ∙ 𝒂2 𝒂3 ∙ 𝒂3 0 0 1

32
Orthogonal/Orthonormal Matrix
• Example:

2 1 2 1
−
5 5 5 5 1 0
=
1 2 1 2 0 1
−
5 5 5 5

(Orthogonal matrices are a very important class of matrices. They have

fascinating properties which we will see later. )

33