Introduction to Machine Learning — Lecture Notes
Generated by ChatGPT — August 09, 2025
Table of Contents
1 1. What is Machine Learning? (Overview & Types)
2 2. Supervised Learning: Regression & Classification
3 3. Optimization: Gradient Descent & Regularization
4 4. Common Algorithms (Decision Trees, SVM, k-NN)
5 5. Neural Networks — Basics
6 6. Model Evaluation Metrics
7 7. Overfitting, Underfitting & Cross-Validation
8 8. Feature Engineering & Preprocessing
9 9. Practical ML Workflow & Tips
10 10. Example Walkthrough (Classification)
11 11. Practice Problems and Further Reading
1. What is Machine Learning?
Machine Learning (ML) is a field of computer science that gives systems the ability to learn from data and make
predictions or decisions without being explicitly programmed for each task. Broadly, ML approaches fall into three
categories: supervised, unsupervised, and reinforcement learning.
Supervised learning: models learn a mapping from inputs X to outputs y using labeled examples.
Unsupervised learning: uncover structure or patterns in unlabeled data (e.g., clustering, dimensionality
reduction).
Reinforcement learning: agents learn to take actions that maximize cumulative reward through interaction with
an environment.
2. Supervised Learning: Regression & Classification
Supervised problems come in two main flavors: - Regression: predict continuous outputs (e.g., house price) -
Classification: predict discrete labels (e.g., spam vs. ham)
Linear regression (simple): model y ≈ w·x + b. The goal is to find parameters that minimise a loss (e.g., mean
squared error).
Logistic regression for binary classification uses the sigmoid function σ(z)=1/(1+e^{-z}) and optimizes
cross-entropy loss.
3. Optimization: Gradient Descent & Regularization
Gradient descent iteratively updates parameters in the direction of negative gradient of the loss: θ := θ - α ∇_θ
L(θ). Choose learning rate α carefully.
Regularization helps prevent overfitting. Common forms: - L2 (Ridge): adds λ||θ||^2 to loss - L1 (Lasso): adds
λ||θ||_1 and can induce sparsity
4. Common Algorithms
Decision Trees: partition feature space with axis-aligned splits; easy to interpret but can overfit. Random Forests
aggregate many trees to reduce variance.
Support Vector Machines (SVM): find a margin-maximizing hyperplane; kernel trick enables non-linear decision
boundaries.
k-Nearest Neighbors (k-NN): non-parametric, prediction based on closest training examples; sensitive to
feature scaling.
5. Neural Networks — Basics
A neural network composes layers of linear transformations and non-linear activation functions. The simplest unit
is a perceptron. Common activations: ReLU, sigmoid, tanh.
Training uses backpropagation to compute gradients and an optimizer (SGD, Adam) to update weights.
6. Model Evaluation Metrics
Classification metrics: - Accuracy = (TP + TN) / (TP + TN + FP + FN) - Precision = TP / (TP + FP) - Recall = TP /
(TP + FN) - F1 = 2 * (precision * recall) / (precision + recall) - ROC AUC: area under ROC curve;
threshold-independent
Regression metrics: MSE (mean squared error), MAE (mean absolute error), R^2 (coefficient of determination).
7. Overfitting, Underfitting & Cross-Validation
Overfitting: model performs well on training data but poorly on unseen data. Underfitting: model is too simple to
capture underlying patterns.
Cross-validation (e.g., k-fold) provides a robust estimate of generalization performance and helps in model
selection.
8. Feature Engineering & Preprocessing
Common preprocessing steps: - Handle missing values (imputation) - Scale features
(standardization/normalization) - Encode categorical variables (one-hot, target encoding) - Feature selection and
creation (polynomials, interactions)
Good features often matter more than the algorithm itself.
9. Practical ML Workflow & Tips
• Define the objective and evaluation metric clearly.
• Explore and clean the data: visualise distributions and identify issues.
• Start with simple baseline models (e.g., linear/logistic regression).
• Iterate on features, then try more complex models if needed.
• Use cross-validation, keep a held-out test set, and avoid data leakage.
• Track experiments and hyperparameters.
10. Example Walkthrough (Binary Classification)
Given a dataset with features (age, income, clicks) and label (purchase? yes/no): 1. Inspect distributions and
missingness. 2. Encode categorical features and scale numeric features. 3. Split into train/validation/test. 4. Train
a logistic regression as a baseline, evaluate with AUC and F1. 5. If performance is low, try tree-based models or
feature interactions.
11. Practice Problems & Further Reading
Practice problems:
• Implement linear regression with gradient descent and compare with a library solver.
• Train a decision tree on a small dataset and visualise the splits.
• Compare feature scaling effects on k-NN performance.
• Use k-fold cross-validation to tune hyperparameters of a random forest.
Suggested further reading (introductory):
• A general introductory ML textbook or online course (search for recent university lecture notes).
• Official documentation and tutorials for scikit-learn, TensorFlow, or PyTorch.
• Research surveys or review articles for specific subtopics.
Notes: These lecture notes provide a concise overview useful for beginners and as a revision resource. They are
original, copyright-free, and intended for educational sharing.
