MACHINE

LEARNING
WITH PYTHON
BY SYAZWANA SAHAR
CONTENT/GOALS/
OBJECTIVES
1 INTRODUCTION TO
MACHINE LEARNING
 What is machine learning?
 “Machine Learning is any process by which a system improves
performance from experience.” – Hebert Simon
 Definition by Tom Mitchell (1988)

Machine learning is the study of algorithms that

 Improve their performance P
 At some task T
 With experience E
A well-defined learning task is given by <P,T,E>
 1 INTRODUCTION TO
MACHINE LEARNING

Slide credit: Pedro Domingos

1 INTRODUCTION TO
MACHINE LEARNING

"Can anyone share an example of a machine

learning application you’ve encountered or read
about?"
 1 INTRODUCTION TO
MACHINE LEARNING
Medical
Stock market diagnosis
trading

Image
Text & Real world Real world recognition
speech machine machine
recognition learning learning
application application
s s
Product
(1D data) (2D data) recommendati
Fraud on
detection

Traffic
Email spam prediction
filtering
 1.1 TYPES OF MACHINE
LEARNING
1) Supervised learning
(classification,
regression)
 Given: training data
+ desired outputs
(labels)
2) Unsupervised
learning
 Given: training data
(without desired
outputs)
3) Reinforcement
 Rewards from
sequence of actions

Photo source: (Peng et al., 2021)

 1.1 TYPES OF MACHINE
LEARNING
1) Supervised learning 2) Unsupervised learning
(classification, regression)  Given: training data (without desired
 Given: training data + desired outputs)
outputs (labels)

Photo source: https://databasetown.com/supervised-learning-algorithms/

Photo source: https://databasetown.com/unsupervised-learning-types-applications/
 1.1 TYPES OF MACHINE
LEARNING
3) Reinforcement
 is all about making decisions sequentially. In simple words, we can say that the output
depends on the state of the current input and the next input depends on the output of
the previous input

•Agent: The learner or decision-

maker.
•Action: All possible moves the
agent can make.
• Environment: Everything the
agent interacts with.
•Reward: Feedback from the
environment based on the action
taken.

Photo source: (Niklas Lang, 2022)

1.1 TYPES OF MACHINE
LEARNING

What is the type of machine learning for below example?

1. Credit assignment problem, game playing, robot in a maze
2. Social network analysis, market segmentation, astronomical data analysis
3. Email spam detection, house price prediction, medical diagnosis, image
classification, speech recognition
2 REGRESSION
 Statistical approach
used to analyze the
relationship
between a
dependent variable
(target variable)
and one or more
independent
variables (predictor
variables).
 Example: prediction
of house prices
based on different
parameters like
house age, distance
from the main road,
location, area, etc.
Photo source: https://www.coursera.org/lecture/ml-regression/what-is-the-course-about-
0JhTY
 2.1 REGRESSION MODEL Linear Regression (LR)

• Simple LR
• Multiple LR
• Ridge regression
• Lasso regression
• Elastic net

Non-linear Regression
(a) Linear regression (b) Polynomial regression (c) Support vector regression (NLR)
• Polynomial regression
• Support vector regression
• K-nearest neighbour (KNN)
regression
• Decision tree
• Random forest regression

(d) KNN regression

2.1 SIMPLE LINEAR
REGRESSION
• Y is the dependent variable
• X is the independent variable
• is the intercept
• is the slope
 MAE – mean absolute
error

2.2 EVALUATION METRICS MSE – mean square error

RMSE – root mean square
error
– R- squared
MAPE – mean absolute
percentage error

Photo Source: (Demir and Sahin, 2024)

 3 MACHINE LEARNING
PIPELINE

Photo Source: https://towardsdatascience.com/machine-learning-pipelines-feature-engineering-numbers-29f53aaec82a?

ref=broutonlab.com
4 PYTHON IMPLEMENTATION
OF LINEAR REGRESSION
5 BASIC STEPS TO IMPELEMENT MODEL:
• Import the packages and classes that
1 you need

• Provide data to work with, and

2 eventually do appropriate
transformations
• Create a regression model and fit it with
3 existing data

• Check the results of model fitting to

4 know whether the model is satisfactory

• Apply the model for predictions

5
ASSIGNMENT
 Assignment Title: Predicting Housing Prices with a Linear Regression
Model
 1) Dataset: Use the Boston Housing Dataset available via scikit-learn or
the California Housing Dataset from the StatLib repository. Boston Housing
Dataset: Available directly in scikit-learn. California Housing Dataset:
Available through fetch_california_housing in sklearn.datasets.
 2) Tasks: Load the dataset using sklearn.datasets.load_boston() or
sklearn.datasets.fetch_california_housing().Explore the dataset by
visualizing the relationship between one feature (e.g., average number of
rooms per dwelling) and the target variable (house prices).Implement a
simple linear regression model to predict house prices based on the chosen
feature. Visualize the regression line and evaluate the model using metrics
such as Mean Squared Error (MSE) and R² score. Write a brief analysis of
the model's performance.
 3) Deliverables: Python code for data loading, exploration, model
implementation, and evaluation. A short report (1-2 pages) summarizing
findings and conclusions.
2.2 MULTIPLE LINEAR
REGRESSION
 Tunjuk contoh based on case Multiple layer regression
study

y=β0​+β1​X1+β2​X2+………βn​Xn

• Y is the dependent variable

• X1, X2, …, Xn are the
independent variables
• β0 is the intercept
• β1, β2, …, βn are the slopes
COST FUNCTION FOR
LINEAR REGRESSION
Mean Squared Error (MSE) Cost
function – calculates the average
of the squared errors between the
predicted value and the true value
Y
GRADIENT DESCENT FOR
LINEAR REGRESSION
Gradient descent – a derivative
that defines the effects on outputs
of the function with a variation in
inputs

Differentiate the cost function with respect to :

Note: is the learning rate

GRADIENT DESCENT FOR
LINEAR REGRESSION
Gradient descent – a derivative
that defines the effects on outputs
of the function with a variation in
inputs

Differentiate the cost function with respect to :

Note: is the learning rate

EVALUATION METRICS
 Mean Square Error (MSE) – to  Mean Absolute Error (MAE) –
quantify the accuracy of lower MAE value indicates
model’s prediction better model performance.

• n is the number of data points. • n is the number of observations

• is the actual or observed value • represents the actual values.
for the ith data point.
• represents the predicted values
• is the predicted value for the
ith data point.
EVALUATION METRICS
 Root Mean Square Error
(RMSE) – it describes how well
the observed data points
match the expected values

 RMSE is not as good as R-

squared Error.
EVALUATION METRICS
 R-squared – is a statistic that  Residual sum of squares (RSS)
indicates how much variation the – measurement of difference
developed model can capture. between the output that was
observed and what was
anticipated.

 Total sum of squares (TSS) - The sum of

the data points’ errors from the
answer variable’s mean
1.1 TYPES OF MACHINE
LEARNING
Types of machine learning

Supervised Unsupervised Reinforcement

Regression
- Risk Regression
Dimensionality
reduction
Gaming
Finance sector

assessment
- Risk assessment -Text mining
Manufacturing
- Score prediction -Face recognition
Robot navigation
- Image recognition

- Score
prediction
Classification
- Fraud detection
Clustering
- Biology
-Email spam detection - City planning
-Image classification -Targeted marketing
 2.1 REGRESSION MODEL
Linear regression
model
 Simple LR
 Multiple LR
 Ridge regression
 Lasso regression
 Elastic net

Linear regression Non-linear regression