SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

Course Content

II Semester

Course Code: 24EMAB102 Course Title: Linear Algebra

L-T-P: 3-0-1 Credits: 4 Contact Hrs: 5hrs/week
ISA Marks: 50 ESA Marks: 50 Total Marks: 100
Teaching Hrs: 40 Tutorial/Practical Hrs: 28 Exam Duration: 3 hrs.
Lab Schedule:

Week Concepts Topics Activity Outcome Exercise

Exercise on Python ----
programming covering Installation of
Installation
the Python essential Jupyter notebook,
of Python,
concepts execution of basic
Introduction Jupyter
Watch: Python for Python programs,
1. to Python notebook,
Beginners - Full Course understanding
and NumPy and
Python essentials,
necessary
https:// practice using
libraries
www.youtube.com/ NumPy.
watch?v=rfscVS0vtbw
Implementation of ---
control structures and Develop basic
Practice
functions in Python programming skills,
Control exercises on
Watch: Python understand and
Structures loops,
2. Functions implement control
and conditionals,
structures, practice
Functions and defining
https:// defining and using
functions
www.youtube.com/ functions in Python.
watch?v=NSbOtYzIQI0
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
Implement matrix Car Problem
Understand matrix
addition, scalar
Exercises operations,
multiplication
using NumPy implement matrix
Watch: Matrix
for matrix addition and scalar
Matrix Operations in NumPy
3. addition and multiplication using
Operations
scalar NumPy, develop
https://youtu.be/
multiplicatio skills in
g9ItebvM9K4?
n manipulating
si=V2gGjFBmtYhy-fXl
matrices.
Solve linear
equations
Solve systems of
Solving Solve system of linear using
linear equations,
4. Linear equations, find inverse NumPy,
find and verify
Equations matrices calculate
inverse matrices.
inverse
matrices
Compute dot product,
cross product of
vectors Watch: Vector Compute dot and
Exercises on
Operations in Python cross products,
dot product
Vector understand vector
5. and cross
Operations https:// operations, and
product
www.youtube.com/ apply these
calculations
watch? concepts.
v=fqxWBj0cmDY

6. Basis and Identify linearly Determine Identify linearly

Dimension independent vectors, linear independent
find basis of vector independenc vectors, compute
space e, find basis basis for vector
spaces, and
understand the
concept of
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
dimension.
Compute orthogonal
projections,
understand and
Orthogonal Exercises on
Compute orthogonal apply orthogonality
7. Vectors and orthogonal
projections of vectors in vector spaces,
Projections projections
practical
applications of
projections.
Apply Gram-
Apply Gram-Schmidt Gram- Schmidt process,
Gram-
process to set of Schmidt verify orthogonality,
8. Schmidt
vectors to verify orthogonaliz understand the
Process
orthogonality ation construction of
orthonormal bases.
Compute and
interpret
Eigenvalues
Compute eigenvalues, Calculate eigenvalues and
and
9. eigenvectors of eigenvalues, eigenvectors, apply
Eigenvector
matrices and page rank eigenvectors these concepts to
s
practical problems
like page rank.
Understand and
implement matrix
Diagonaliza Diagonalize diagonalization,
tion and Implement matrix matrices, apply
10.
Application diagonalization practical diagonalization to
s applications solve real-world
problem of image
compression.
11. Positive Test matrix for positive Exercises on Test matrices for
Definite definiteness in positive positive
Matrices optimization problems definiteness definiteness,
testing understand the
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
importance in
optimization
problems, practical
applications of
finding positive
definiteness of a
input graph of a
social network
analysis case
Implement SVD, apply
it to given application Implement Singular
Watch: Singular Value Perform SVD, Value
Singular
Decomposition apply to data Decomposition,
Value
12. analysis and apply SVD to solve
Decomposit
https:// other real-world problem
ion (SVD)
www.youtube.com/ applications of noise reduction in
watch?v=P5mlg91as1c an input image.

Text Books:

1. Linear Algebra and its Applications, Gilbert Strang , 4 Edition, Cengage India Private Limited 2005.

2. Linear Algebra and Its Applications, David C. Lay, 5th Edition - Pearson -2023.

Reference Books:

1. Boyd, Stephen, and Lieven Vandenberghe. “Introduction to applied linear algebra: vectors, matrices,
and least squares”, Cambridge university press, 2018.

2. Linear algebra for computer science “Essential Mathematics for Computer Scientists” M.
THULASIDAS, Singapore Management University, 2021.
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

Examples for exercises

1.Matrix Operations

Exercise: Matrix Operations with Image Processing

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

Objective: Understand and implement matrix operations including matrix addition and scalar
multiplication using NumPy. Develop skills in manipulating matrices by applying these operations to image
processing.

Given image above shows a colour image represented as matrix values. Now, suppose you are working
with grayscale images, which is represented as matrix where each element corresponds to a pixel's
intensity. You will apply matrix operations to modify and enhance images.
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

Image Representation:

Consider a grayscale image represented by a 5x5 matrix where each value represents the intensity of a
pixel (0 is black, 255 is white).

Example Image Matrix (Original):

I = [30, 45, 60, 45, 30]

[45, 60, 75, 60, 45]

[60, 75, 90, 75, 60]

[45, 60, 75, 60, 45]

[30, 45, 60, 45, 30]

Transformation Matrix:

We will use a transformation matrix T to change the brightness of the image. Consider the following matrix
to add to the original image:

T =[10, 10, 10, 10, 10]

[10, 10, 10, 10, 10]

[10, 10, 10, 10, 10]

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
[10, 10, 10, 10, 10]

[10, 10, 10, 10, 10]

Steps

Part A: Implementing Matrix Addition

Load the matrices:

Use NumPy to create the matrices for the original image and the transformation matrix.

import numpy as np

# Original image matrix

I = np.array([

[30, 45, 60, 45, 30],

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
[45, 60, 75, 60, 45],

[60, 75, 90, 75, 60],

[45, 60, 75, 60, 45],

[30, 45, 60, 45, 30]

])

# Transformation matrix

T = np.array([

[10, 10, 10, 10, 10],

[10, 10, 10, 10, 10],

[10, 10, 10, 10, 10],

[10, 10, 10, 10, 10],

[10, 10, 10, 10, 10]

])

Perform matrix addition: Compute the transformed image by adding the transformation matrix T to the
original image matrix I

# Transformed image matrix

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
I_transformed = I + T

Part B: Implementing Scalar Multiplication

Apply scalar multiplication: Use NumPy to multiply the original image matrix I by the scalar value α.

# Scalar value

alpha = 1.5

# Scalar multiplication

I_scaled = I * alpha

# Display the scaled image matrix.

Ensure pixel values are within valid range: As grayscale images are used, pixel values should be
between 0 and 255. After scalar multiplication, clip the values to this range.

# Clip values to range [0, 255]

I_scaled = np.clip(I_scaled, 0, 255)

Code:

import numpy as np

# Original image matrix

I = np.array([

[30, 45, 60, 45, 30],

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
[45, 60, 75, 60, 45],

[60, 75, 90, 75, 60],

[45, 60, 75, 60, 45],

[30, 45, 60, 45, 30]

])

# Transformation matrix

T = np.array([

[10, 10, 10, 10, 10],

[10, 10, 10, 10, 10],

[10, 10, 10, 10, 10],

[10, 10, 10, 10, 10],

[10, 10, 10, 10, 10]

])

# Scalar value

alpha = 1.5
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
# Matrix addition

I_adjusted = I + T

# Scalar multiplication

I_scaled = I * alpha

# Clip values to range [0, 255]

I_scaled = np.clip(I_scaled, 0, 255)

# Display results

print("Original Image Matrix:\n", I)

print("Transformed Image Matrix (after adding T):\n", I_transformed)

print("Scaled Image Matrix (after multiplying by alpha):\n", I_scaled)

Discussion

 Compare the original image matrix with the transformed and scaled images using matrix operations.
How do the operations affect the image?

 Discuss the effect of matrix addition and scalar multiplication on the pixel values.

 Try different values for the transformation matrix T and scalar α. observe how changes in these
values affect the image.
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

2.System of linear equations

Exercise: Supply and Demand Model using system of linear equations

Objective: Solve a system of linear equations to model supply and demand, and find and verify the inverse
of a matrix using NumPy.

Example: Suppose you are a data analyst tasked with determining the stable of linear equations that you
need to solve. Additionally, you will find and verify the inverse matrix to understand its role in solving the
system.

Consider a market where two goods, Goods X and Goods Y, are being analyzed. The supply and demand
equations for these goods are given by:

1. 2x+3y=18 (Demand equation for Goods X)

2. 4x+5y=30 (Demand equation for Goods Y)

where x and y represent the quantities of Goods X and Goods Y, respectively.

Matrix Formulation:
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
These equations can be represented in matrix form as:

Ax=b

Where:

A= [ 24 35] , x=[ xy ] and b = [ 1830]

Steps

Part A: Solving the System of Linear Equations

1. Set up the matrices: Use NumPy to define the matrices A and b

import numpy as np

# Coefficient matrix

A = np.array([

[2, 3],

[4, 5]

])

# Constant matrix

b = np.array([18, 30])
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
2. Solve the system: Use NumPy to solve for x (the quantities of Goods X and Goods Y).

# Solve the system of equations

x = np.linalg.solve(A, b)

print("Solution for x and y:", x)

Part B: Finding and Verifying the Inverse Matrix

1. Find the inverse of matrix A: Compute the inverse of A using NumPy.

# Compute the inverse of matrix A

A_inv = np.linalg.inv(A)

print("Inverse of matrix A:\n", A_inv)

2. Verify the inverse: Check that AA−1=I and A−1A=I.

# Verify the inverse

identity_matrix = np.dot(A, A_inv)

print("A * A_inv:\n", identity_matrix)

# Verify identity matrix

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
I = np.identity(A.shape[0])

print("Identity matrix:\n", I)

3. Use the inverse to solve the system:

Verify the solution by computing x=A−1b.

# Compute the solution using the inverse

x_inv_method = np.dot(A_inv, b)

print("Solution using inverse method:", x_inv_method)

Full Code:

import numpy as np

# Coefficient matrix
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
A = np.array([

[2, 3],

[4, 5] ])

# Constant matrix

b = np.array([18, 30])

# Solve the system of equations

x = np.linalg.solve(A, b)

print("Solution for x and y:", x)

# Compute the inverse of matrix A

A_inv = np.linalg.inv(A)

print("Inverse of matrix A:\n", A_inv)

# Verify the inverse

identity_matrix = np.dot(A, A_inv)

print("A * A_inv:\n", identity_matrix)

# Verify identity matrix

I = np.identity(A.shape[0])
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
print("Identity matrix:\n", I)

# Compute the solution using the inverse

x_inv_method = np.dot(A_inv, b)

print("Solution using inverse method:", x_inv_method)

Discussion

 What do the values of x and y represent in the context of the supply and demand
model?

 How does the inverse matrix method help in solving systems of linear equations?

3.Vector operations

Exercise: Vector Operations and One-Hot Encoding

Objective: Compute and understand dot and cross products involving one-hot vectors, and apply these concepts to analyze
relationships between categories.

One-hot vectors are a popular method for encoding categorical data in various machine learning and data processing tasks. One-hot
vectors are commonly used to represent categorical data in a format that machine learning models can process. Each category in a
dataset is represented as a binary vector where only one element is set to 1 (indicating the presence of the category) and all other
elements are set to 0.

In a dataset with three categories: ['pedestrian', 'car', 'motorcycle'], these could be represented as:
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

You are working with a dataset where categories

Column are represented
vector spaces by one-hot vectors. You need to:

1. Compute the dot product to measure similarity between categories.

2. Analyze relationships between categories using cross products, which can be useful in detecting orthogonality (independence)
of categories.

Given Data:

Consider a dataset with three categories: 'pedestrian', 'car', 'motorcycle’. Represent each category using one-hot vectors
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

[]
1
Category A (pedestrian): A= 0
0

[]
0
Category B (car): B= 1
0

[]
0
Category C (motorcycle): C= 0
1

Steps

Part A: Compute the Dot Product

1. Measure Similarity: Compute the dot product between different pairs of one-hot vectors to determine the similarity between
categories. Since one-hot vectors are orthogonal, the dot product should be 0 between different categories and 1 when
comparing a vector with itself.

Part B: Compute the Cross Product

1. Analyze Orthogonality: Compute the cross product between pairs of one-hot vectors to check if they produce vectors that
indicate orthogonality in the context of high-dimensional space.
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
Code

import numpy as np

# Define the one-hot vectors

A = np.array([1, 0, 0])

B = np.array([0, 1, 0])

C = np.array([0, 0, 1])

# Compute dot products

dot_AB = np.dot(A, B)

dot_AC = np.dot(A, C)

dot_BC = np.dot(B, C)

dot_AA = np.dot(A, A)

dot_BB = np.dot(B, B)

dot_CC = np.dot(C, C)

print("Dot product A · B:", dot_AB)

print("Dot product A · C:", dot_AC)

print("Dot product B · C:", dot_BC)

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
print("Dot product A · A:", dot_AA)

print("Dot product B · B:", dot_BB)

print("Dot product C · C:", dot_CC)

# Compute cross products

cross_AB = np.cross(A, B)

cross_BC = np.cross(B, C)

cross_CA = np.cross(C, A)

print("Cross product A × B:", cross_AB)

print("Cross product B × C:", cross_BC)

print("Cross product C × A:", cross_CA)

Discussion

 Verify the dot products and their values. Discuss the similarity between categories and the concept of orthogonality.

 Analyze the cross products to understand their significance in high-dimensional space. Discuss how they might be used to
understand relationships between categories.

4. Identify linearly independent vectors; compute a basis for a feature space

Exercise: Feature Selection and Basis Computation based on linear independence

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
Objective: Identify linearly independent vectors, compute a basis for a feature space, and understand the concept of dimension in the
context of feature selection for machine learning.

Example: In machine learning, feature selection is important to identify a set of features that capture the most information while
avoiding redundancy. This involves determining a basis for the feature space, which consists of linearly independent vectors.

You are working with a dataset where each feature is represented as a vector. Your goal is to:

1. Determine which features (vectors) are linearly independent.

2. Compute a basis for the feature space spanned by these features.
3. Understand the dimension of the feature space.

Given Data:

Consider a dataset with the following feature vectors:

[]
1
Feature Vector 1: f1= 2
3

[]
2
Feature Vector 2: f2= 4
6

[]
0
Feature Vector 3: f3= 1
2

[]
1
Feature Vector 4: f4= 0
1
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
Steps

Part A: Identify Linearly Independent Vectors

Create a Matrix: Combine the feature vectors into a matrix where each row represents a feature vector.

Compute the Rank: The rank of the matrix will give the number of linearly independent vectors. You can use NumPy's
np.linalg.matrix_rank function to find this.

Part B: Compute the Basis for the Feature Space

Find the Basis: Use NumPy's np.linalg.svd (Singular Value Decomposition) to compute a basis for the vector space. Alternatively,
use np.linalg.qr to perform QR decomposition, which will also give you a basis.

Part C: Understand the Dimension of the Feature Space

Determine the Dimension: The dimension of the feature space is equal to the rank of the matrix, which is the number of linearly
independent vectors.

Code

import numpy as np

# Define the feature vectors

f1 = np.array([1, 2, 3])

f2 = np.array([2, 4, 6])

f3 = np.array([0, 1, 2])

f4 = np.array([1, 0, 1])
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

# Create a matrix with feature vectors as rows

matrix = np.array([f1, f2, f3, f4])

# Compute the rank of the matrix (number of linearly independent vectors)

rank = np.linalg.matrix_rank(matrix)

print("Rank of the matrix (number of linearly independent vectors):", rank)

# Compute the basis using QR decomposition

Q, R = np.linalg.qr(matrix.T)

basis = Q[:, :rank].T

print("Basis for the feature space:\n", basis)

# Dimension of the feature space

dimension = rank

print("Dimension of the feature space:", dimension)

Discussion
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
 Discuss which vectors are linearly independent and why some vectors are dependent.

 Analyze the computed basis and how it represents the feature space.

5.Orthogonal Vectors and Projections

Exercise: Orthogonal Projections and Dimensionality Reduction

Objective: Compute orthogonal projections, understand and apply orthogonality in vector spaces, and explore practical applications
of projections in data fitting and dimensionality reduction.

In machine learning, techniques like Principal Component Analysis (PCA) use orthogonal projections are to reduce the dimensionality
of data while preserving as much variance as possible. This exercise involves projecting data points onto a subspace spanned by a set
of basis vectors, which is fundamental for dimensionality reduction and feature extraction.
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

You have a dataset with 2-dimensional feature vectors and want to project these vectors onto a 1-dimensional subspace. This subspace
is spanned by a given vector, representing a principal component or direction of interest. The goal is to:

1. Compute the orthogonal projection of each data point onto this subspace.
2. Understand how orthogonality helps in simplifying the data while retaining key features.

Given Data: Data Points

x1= [ 12] x2= [ 34 ] x3= [ 56]

Subspace Vector (Basis Vector): u= [ 11]
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

Steps

Part A: Compute Orthogonal Projections

1. Normalize the Basis Vector: Compute the unit vector in the direction of u. This is necessary for projecting onto the direction
2. Compute Projections: For each data point x, compute the orthogonal projection onto the direction of u

Proju(x) = x.u/u.u u

Part B: Understand and Apply Orthogonality

1. Compute the Orthogonal Component: For each data point x, compute the orthogonal component that is orthogonal to u. This
is the difference between the original vector and its projection.

Part C: Visualize the Data

1. Visualize the Data: Plot the original data points, their projections, and the orthogonal components to visualize how the data is
simplified by projecting onto the subspace.

Code

import numpy as np

import matplotlib.pyplot as plt

# Define the basis vector

u = np.array([1, 1])

# Normalize the basis vector

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
u_norm = u / np.linalg.norm(u)

print("Normalized basis vector u:", u_norm)

# Define the data points

x1 = np.array([1, 2])

x2 = np.array([3, 4])

x3 = np.array([5, 6])

# Compute projections

def project_onto_basis(x, u):

return (np.dot(x, u) / np.dot(u, u)) * u

proj_x1 = project_onto_basis(x1, u_norm)

proj_x2 = project_onto_basis(x2, u_norm)

proj_x3 = project_onto_basis(x3, u_norm)

print("Projection of x1 onto u:", proj_x1)

print("Projection of x2 onto u:", proj_x2)

print("Projection of x3 onto u:", proj_x3)

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
# Compute orthogonal components

def orthogonal_component(x, proj):

return x - proj

orthog_x1 = orthogonal_component(x1, proj_x1)

orthog_x2 = orthogonal_component(x2, proj_x2)

orthog_x3 = orthogonal_component(x3, proj_x3)

print("Orthogonal component of x1:", orthog_x1)

print("Orthogonal component of x2:", orthog_x2)

print("Orthogonal component of x3:", orthog_x3)

# Define the data points and projections

data_points = np.array([x1, x2, x3])

projections = np.array([proj_x1, proj_x2, proj_x3])

orthogonal_components = np.array([orthog_x1, orthog_x2, orthog_x3])

# Plot data points

plt.figure(figsize=(10, 6))

plt.quiver(*data_points.T, angles='xy', scale_units='xy', scale=1, color='r', label='Data Points')

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
plt.quiver(*projections.T, angles='xy', scale_units='xy', scale=1, color='b', label='Projections')

plt.quiver(*orthogonal_components.T, angles='xy', scale_units='xy', scale=1, color='g', label='Orthogonal Components')

# Plot the basis vector

plt.quiver(0, 0, u_norm[0], u_norm[1], angles='xy', scale_units='xy', scale=1, color='k', label='Basis Vector u')

# Set labels and title

plt.xlim(-1, 7)

plt.ylim(-1, 7)

plt.axhline(0, color='gray', linestyle='--')

plt.axvline(0, color='gray', linestyle='--')

plt.xlabel('X-axis')

plt.ylabel('Y-axis')

plt.title('Data Points, Projections, and Orthogonal Components')

plt.legend()

plt.grid(True)

plt.show()

Discussion
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
 Analyze how the projections simplify the representation of the data.

 Discuss the orthogonal components and how they relate to the original data.

6.Gram-Schmidt orthogonalization

Exercise: Analyzing Network Traffic Using the Gram-Schmidt Process

Objective: Apply the Gram-Schmidt process to network traffic data to construct orthonormal bases for feature vectors, and analyze
the implications for network traffic pattern recognition and anomaly detection.

Example: In network traffic analysis, different traffic patterns (e.g., normal and anomalous behaviors) can be represented as feature
vectors. Orthogonalizing these vectors can help in identifying and analyzing patterns more effectively, improving network security
and performance.

You have network traffic data represented as feature vectors where each vector captures various attributes (e.g., packet size, protocol
type, etc.) of network traffic. The goal is to:

1. Apply the Gram-Schmidt process to orthogonalize the feature vectors.

2. Analyze how orthogonalizing affects the identification of different traffic patterns.
3. Use the orthonormal basis for detecting anomalies.

Given Data:

Consider a simplified dataset of network traffic where each feature vector represents a different network traffic sample:

[]
3
Feature Vector 1: v1= 7
2
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

[]
6
Feature Vector 2: v2= 2
4

[]
1
Feature Vector 3: v3= 5
8

Steps

Part A: Apply the Gram-Schmidt Process

Implement the Gram-Schmidt Process: Write a function to apply the Gram-Schmidt process to the given feature vectors to
obtain an orthonormal basis.

Part B: Verify Orthogonality

Check Orthogonality: Compute the dot products between pairs of orthonormal vectors to verify that they are orthogonal.

Part C: Use the Orthonormal Basis for Anomaly Detection

Project New Data Points: Project a new network traffic vector onto the orthonormal basis to determine if it is consistent with
the known patterns.

Code

import numpy as np
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

def gram_schmidt(vectors):
orthonormal_basis = []
for v in vectors:
for u in orthonormal_basis:
v -= np.dot(v, u) * u
v = v / np.linalg.norm(v)
orthonormal_basis.append(v)
return np.array(orthonormal_basis)

# Define the feature vectors

v1 = np.array([3, 7, 2])
v2 = np.array([6, 2, 4])
v3 = np.array([1, 5, 8])

# Apply the Gram-Schmidt process

vectors = np.array([v1, v2, v3])
orthonormal_basis = gram_schmidt(vectors)
print("Orthonormal Basis:\n", orthonormal_basis)

def check_orthogonality(vectors):
num_vectors = vectors.shape[0]
for i in range(num_vectors):
for j in range(i + 1, num_vectors):
dot_product = np.dot(vectors[i], vectors[j])
print(f"Dot product of vector {i+1} and vector {j+1}:", dot_product)

check_orthogonality(orthonormal_basis)

def project_onto_basis(x, basis):

projections = [np.dot(x, b) * b for b in basis]
return np.sum(projections, axis=0)
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

def detect_anomaly(new_vector, basis, threshold=0.1):

projection = project_onto_basis(new_vector, basis)
error = np.linalg.norm(new_vector - projection)
return error > threshold

# Define a new network traffic vector

new_vector = np.array([5, 3, 6])

# Detect anomaly
is_anomaly = detect_anomaly(new_vector, orthonormal_basis)
print("Is the new vector an anomaly?", is_anomaly)

Discussion

 Discuss how the Gram-Schmidt process transformed the feature vectors into an orthonormal basis.
 Analyze how the orthonormal basis helps in identifying anomalies in network traffic data.

7. Eigenvalues and Eigenvectors

Exercise: Computing Eigenvalues and Eigenvectors for PageRank

Objective: Compute and interpret eigenvalues and eigenvectors in the context of the PageRank algorithm. Understand how these
concepts are used to rank web pages based on their link structure.

Example: PageRank is a system used by Google to rank web pages in search engine results based on their importance. It uses the link
structure of the web to assign a rank to each page. The PageRank algorithm involves computing the dominant eigenvector of a
transition matrix, where the eigenvalues and eigenvectors help in understanding the importance of each page.
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
You are given a simplified web with four pages and their link structure. You will compute the PageRank vector by finding the
dominant eigenvector of the corresponding transition matrix.

Given Data:

Consider a web with four pages A, B, C, and D with the following link structure:

 Page A links to Pages B and C

 Page B links to Pages A and D
 Page C links to Page D
 Page D links to Pages A and B

Steps

Part A: Construct the Transition Matrix

Build the Transition Matrix: Construct the transition matrix M where Mij represents the probability of transitioning from page i to
page j. Each element of the matrix should be 0 or the reciprocal of the number of links from a given page.

# Transition matrix

M = np.array([ [0, 0.5, 0.5, 0], # Transitions from A

[0.5, 0, 0, 0.5], # Transitions from B

[0, 0, 0, 1], # Transitions from C

[0.5, 0.5, 0, 0] # Transitions from D ])

Part B: Compute Eigenvalues and Eigenvectors

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
Calculate Eigenvalues and Eigenvectors: Compute the eigenvalues and eigenvectors of the transition matrix. Identify the dominant
eigenvector, which represents the PageRank vector.

Part C: Interpret the Results

Analyze PageRank Vector: Interpret the PageRank vector to understand the relative importance of each page. Higher values in the
PageRank vector indicate more important pages.

Code
import numpy as np

# Define the transition matrix

M = np.array([
[0, 0.5, 0.5, 0], # Transitions from A
[0.5, 0, 0, 0.5], # Transitions from B
[0, 0, 0, 1], # Transitions from C
[0.5, 0.5, 0, 0] # Transitions from D
])

print("Transition Matrix M:\n", M)

# Compute eigenvalues and eigenvectors

eigenvalues, eigenvectors = np.linalg.eig(M)
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
# Find the index of the dominant eigenvalue
dominant_index = np.argmax(np.real(eigenvalues))

# Dominant eigenvector
dominant_eigenvector = np.real(eigenvectors[:, dominant_index])
dominant_eigenvector = dominant_eigenvector / np.sum(dominant_eigenvector) # Normalize

print("Eigenvalues:\n", eigenvalues)
print("Dominant Eigenvector (PageRank Vector):\n", dominant_eigenvector)

# Interpret PageRank vector

pages = ['A', 'B', 'C', 'D']
for i, page in enumerate(pages):
print(f"Page {page} has a PageRank of {dominant_eigenvector[i]:.4f}")

Discussion

 Discuss how the eigenvalues and eigenvectors are used to determine the PageRank vector.
 Analyze the PageRank values to identify the most important pages in the web structure.

 Modify the link structure (e.g., add or remove links) and observe how the PageRank vector changes.

8.Implement matrix diagonalization

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
Exercise: Feature Reduction in Software Metrics Using Matrix Diagonalization

Objective: Understand and implement matrix diagonalization to reduce the dimensionality of software metrics data. Apply PCA
to analyze and visualize software metrics to uncover key patterns and relationships.

Software engineers often collect various metrics (e.g., code complexity, test coverage, defect counts, code churn) from software
projects. Analyzing these metrics can help identify patterns, manage project performance, and make data-driven decisions.
Diagonalization and PCA can be used to reduce the number of metrics while retaining the most important information, making it
easier to analyze and visualize the data.

You have a dataset containing various metrics from different software projects. You will apply PCA to reduce the dimensionality of
this dataset, helping to identify the principal components that explain the most variance in the data.

Given Data:

Assume you have a dataset with the following software metrics collected from several projects:

 Metric 1: Code Complexity

 Metric 2: Test Coverage
 Metric 3: Defect Count
 Metric 4: Code Churn

Code
Project Code Complexity Test Coverage Defect Count
Churn

P1 50 0.80 5 100

P2 70 0.60 15 200

P3 30 0.90 2 50

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
P4 60 0.70 10 150

P5 90 0.50 25 300

Exercise
1. Data Preparation
2. Create the DataFrame
3. Extract Features and Standardize
4. Compute the Covariance Matrix
5. Matrix Diagonalization - Perform Eigen Decomposition
6. Sort Eigenvalues and Eigenvectors
7. Project Data onto Principal Components
8. Select Top Principal Components
9. Plot the Principal Components

Code
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# Step 1: Create the DataFrame

data = {
'Project': ['P1', 'P2', 'P3', 'P4', 'P5'],
'Code Complexity': [50, 70, 30, 60, 90],
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
'Test Coverage': [0.80, 0.60, 0.90, 0.70, 0.50],
'Defect Count': [5, 15, 2, 10, 25],
'Code Churn': [100, 200, 50, 150, 300]
}

df = pd.DataFrame(data)

# Step 2: Extract Features and Standardize

features = df[['Code Complexity', 'Test Coverage', 'Defect Count', 'Code Churn']]
features_mean = features.mean()
features_std = features.std()
features_standardized = (features - features_mean) / features_std

# Step 3: Compute the Covariance Matrix

covariance_matrix = np.cov(features_standardized.T)

# Step 4: Perform Eigen Decomposition

eigenvalues, eigenvectors = np.linalg.eig(covariance_matrix)

# Step 5: Sort Eigenvalues and Eigenvectors

sorted_indices = np.argsort(eigenvalues)[::-1]
eigenvalues_sorted = eigenvalues[sorted_indices]
eigenvectors_sorted = eigenvectors[:, sorted_indices]

# Step 6: Select Top Principal Components

top_components = eigenvectors_sorted[:, :2]

# Step 7: Transform the Data

transformed_data = features_standardized.dot(top_components)

# Step 8: Create a DataFrame with Principal Components

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
pca_df = pd.DataFrame(data=transformed_data, columns=['Principal Component 1', 'Principal Component 2'])
pca_df['Project'] = df['Project']

# Step 9: Plot the Principal Components

plt.figure(figsize=(10, 6))
plt.scatter(pca_df['Principal Component 1'], pca_df['Principal Component 2'], s=100, c='blue', edgecolor='k')

for i, project in enumerate(pca_df['Project']):

plt.text(pca_df['Principal Component 1'][i], pca_df['Principal Component 2'][i], project,
fontsize=12, ha='right')

plt.xlabel('Principal Component 1')

plt.ylabel('Principal Component 2')
plt.title('PCA of Software Metrics')
plt.grid(True)
plt.show()

# Step 10: Print Explained Variance

explained_variance = eigenvalues_sorted / np.sum(eigenvalues_sorted)
cumulative_variance = np.cumsum(explained_variance)

print(f'Explained variance ratio: {explained_variance[:2]}')

print(f'Cumulative explained variance: {cumulative_variance[:2]}')

Discussion
 Discuss the proportion of variance explained by each principal component.
 Analyze the scatter plot. Identify any patterns or clusters.
 Interpret the results based on the principal components.
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

9. Testing Matrices for Positive Definiteness

Experiment: Testing Positive Definiteness of matrix in community detection of a social network analysis

Objective:

To test if matrices related to community detection in a social network (such as similarity matrices) are positive definite, and
understand the implications of this property for identifying and analyzing communities. Community detection algorithms utilize
similarity matrices to identify clusters or communities within a network. Positive definiteness of these matrices can affect the stability
and reliability of the community detection process.

Given dataset: (adjacency matrix of a small network)

Node1, Node2, Node3, Node4

0, 1, 0, 0

1, 0, 1, 1

0, 1, 0, 1

0, 1, 1, 0

Steps:

1. Data Preparation

2. Compute Similarity Matrix

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
3. Test for positive definiteness

4. Community Detection (Louvain Method: Detects communities in the network)

Code

Running the Code: the required libraries to be installed: pip install numpy pandas networkx matplotlib

i)Case 1 : Not positive definite

import numpy as np
import pandas as pd
import networkx as nx
import matplotlib.pyplot as plt
from networkx.algorithms.community import louvain_communities
from matplotlib.colors import ListedColormap

# Create a sample adjacency matrix

adjacency_matrix = np.array([
[0, 1, 0, 0],
[1, 0, 1, 1],
[0, 1, 0, 1],
[0, 1, 1, 0]
])
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING

# Create a NetworkX graph from the adjacency matrix

G = nx.from_numpy_array(adjacency_matrix)

# Compute the similarity matrix (here it is the same as adjacency matrix)

similarity_matrix = adjacency_matrix

# Check if the matrix is symmetric

if np.allclose(similarity_matrix, similarity_matrix.T):
print("Matrix is symmetric.")
else:
print("Matrix is not symmetric.")

# Compute eigenvalues
eigenvalues, _ = np.linalg.eig(similarity_matrix)
print("Eigenvalues:", eigenvalues)
if np.all(eigenvalues > 0):
print("Matrix is positive definite.")
else:
print("Matrix is not positive definite.")

# Perform community detection using Louvain method

communities = louvain_communities(G)
community_mapping = {}
for i, community in enumerate(communities):
for node in community:
community_mapping[node] = i

# Draw the network

pos = nx.spring_layout(G) # Position nodes using the spring layout

# Create a color map for communities

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
cmap = ListedColormap(['#ff9999', '#66b3ff', '#99ff99', '#ffcc99'])
node_colors = [cmap(community_mapping[node]) for node in G.nodes()]

plt.figure(figsize=(8, 6))
nx.draw(G, pos, with_labels=True, node_color=node_colors, edge_color='gray', node_size=500, font_size=16, font_color='black',
cmap=cmap)
plt.title("Network with Detected Communities")
plt.show()

i)Case 2 : positive definite

import numpy as np
import networkx as nx
import matplotlib.pyplot as plt
from networkx.algorithms.community import louvain_communities
from matplotlib.colors import ListedColormap

# Function to check if a matrix is positive definite

def is_positive_definite(matrix):
eigenvalues = np.linalg.eigvals(matrix)
return np.all(eigenvalues > 0)

# Create a positive definite matrix (symmetric and all eigenvalues positive)

# Example: Using a simple positive definite matrix
positive_definite_matrix = np.array([
[2, -1, 0],
[-1, 2, -1],
[0, -1, 2]
])

# Check if the matrix is symmetric

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
if np.allclose(positive_definite_matrix, positive_definite_matrix.T):
print("Matrix is symmetric.")
else:
print("Matrix is not symmetric.")

# Check if the matrix is positive definite

if is_positive_definite(positive_definite_matrix):
print("Matrix is positive definite.")
else:
print("Matrix is not positive definite.")

# Create a NetworkX graph from the positive definite matrix

G = nx.from_numpy_array(positive_definite_matrix)

# Perform community detection using Louvain method

communities = louvain_communities(G)
community_mapping = {}
for i, community in enumerate(communities):
for node in community:
community_mapping[node] = i

# Draw the network

pos = nx.spring_layout(G) # Position nodes using the spring layout

# Create a color map for communities

cmap = ListedColormap(['#ff9999', '#66b3ff', '#99ff99'])
node_colors = [cmap(community_mapping[node]) for node in G.nodes()]

plt.figure(figsize=(8, 6))
nx.draw(G, pos, with_labels=True, node_color=node_colors, edge_color='gray', node_size=500, font_size=16, font_color='black',
cmap=cmap)
plt.title("Network with Detected Communities")
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
plt.show()

Discussion
 Positive definiteness of the similarity matrix implies that the relationships between nodes are well-defined and non-redundant,
leading to stable and reliable community detection results.
 If the matrix is not positive definite, it implies issues such as redundancy in the attributes in the network representation.

10. Singular Value Decomposition (SVD)

Experiment: Applying SVD to a Labeled User-Movie Rating Matrix

Objective:

To apply Singular Value Decomposition (SVD) to a 4x4 user-movie rating matrix. This example demonstrates how to apply Singular
Value Decomposition (SVD) to a user-movie rating matrix. Applying SVD helps in dimensionality reduction and feature extraction.
By visualizing the original matrix, singular values, compressed user values, and reconstructed matrix, we gain insights into how SVD
reduces dimensionality and captures essential features of the data, which is useful for recommendation systems and other data analysis
tasks.

Experiment Steps:
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
1. Define a 4x4 User-Movie Rating Matrix
2. Apply SVD:
o Perform SVD to decompose the matrix into U, Σ, and V
3. Reconstruct the Matrix:
o Use the decomposed matrices to reconstruct the original matrix.
4. Visualize the Results:
o Display the original matrix, singular values, compressed user values, and the reconstructed matrix.

Code

Install Required Libraries: pip install numpy matplotlib

import numpy as np
import matplotlib.pyplot as plt

# Step 1: Define a 4x4 User-Movie Rating Matrix of 0 to 5

# Users: Alice, Bob, Charlie, Dana
# Movies: Inception, The Matrix, Interstellar, Avengers
users = ['Alice', 'Bob', 'Charlie', 'Dana']
movies = ['Inception', 'The Matrix', 'Interstellar', 'Avengers']

A = np.array([
[5, 4, 0, 2],
[3, 0, 0, 5],
[0, 2, 4, 0],
[1, 0, 5, 4]
])

# Step 2: Apply SVD

U, S, VT = np.linalg.svd(A)

# Convert S into a diagonal matrix

 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
Sigma = np.zeros_like(A, dtype=float)
Sigma[:len(S), :len(S)] = np.diag(S)

# Step 3: Reduce dimensions

k = 2 # Number of dimensions to keep
U_reduced = U[:, :k]
S_reduced = np.diag(S[:k])
VT_reduced = VT[:k, :]

# Reconstruct the matrix using the reduced dimensions

A_reconstructed = np.dot(U_reduced, np.dot(S_reduced, VT_reduced))

# Step 4: Visualize the results

plt.figure(figsize=(20, 8))

# Original Matrix
plt.subplot(1, 4, 1)
plt.title('Original Matrix')
plt.imshow(A, cmap='viridis', interpolation='none')
plt.colorbar()
plt.xticks(ticks=np.arange(len(movies)), labels=movies, rotation=45)
plt.yticks(ticks=np.arange(len(users)), labels=users)
plt.xlabel('Movies')
plt.ylabel('Users')

# Singular Values
plt.subplot(1, 4, 2)
plt.title('Singular Values')
plt.bar(range(len(S)), S, color='orange')
plt.xlabel('Index')
plt.ylabel('Singular Value')
 SCHOOL OF COMPUTER SCIENCE AND ENGINEERING
# Compressed User Values
plt.subplot(1, 4, 3)
plt.title('Compressed User Values')
plt.imshow(U_reduced, cmap='viridis', interpolation='none')
plt.colorbar()
plt.xticks(ticks=np.arange(k), labels=[f'Feature {i+1}' for i in range(k)])
plt.yticks(ticks=np.arange(len(users)), labels=users)
plt.xlabel('Compressed Features')
plt.ylabel('Users')

# Reconstructed Matrix
plt.subplot(1, 4, 4)
plt.title('Reconstructed Matrix')
plt.imshow(A_reconstructed, cmap='viridis', interpolation='none')
plt.colorbar()
plt.xticks(ticks=np.arange(len(movies)), labels=movies, rotation=45)
plt.yticks(ticks=np.arange(len(users)), labels=users)
plt.xlabel('Movies')
plt.ylabel('Users')

plt.tight_layout()
plt.show()
Discussion

Reduce Dimensions:

 We keep only the top k=2 singular values .This reduces the representation of users and movies to two dimensions.
 Reducing the dimensionality of the user-movie matrix helps in compressing the data.
 The reduced dimensions highlight the most important features that explain the majority of the variance in the data.