• Unsupervised Learning

Agenda
• Types of Unsupervised Learning
• Clustering types
• k-Means Algorithm & Use cases
• Optimization objective
• Random Initialization
• Use cases
• Principal components analysis
• Independent components analysis,
• Reinforcement Learning
Un Supervised Machine Learning
• Unsupervised learning is a type
of machine learning algorithm
used to draw inferences from
datasets consisting of input data
without labeled responses.
• We want to explore the data to
find some intrinsic structures
in them.

• It is best used if you want to find

patterns but don’t know exactly
what you’re looking for.
Supervised vs. Unsupervised Machine Learning
Parameters Supervised machine learning Unsupervised machine learning
technique technique

Input Data Algorithms are trained using labeled Algorithms are used against data
data. which is not labelled

Computational Complexity Supervised learning is a simpler Unsupervised learning is

method. computationally complex

Accuracy Highly accurate and trustworthy Less accurate and trustworthy

method. method.
Disadvantages of Unsupervised Learning
•You cannot get precise information regarding data sorting, and the output
as data used in unsupervised learning is labeled and not known.
•Less accuracy of the results is because the input data is not known and not
labeled by people in advance. This means that the machine requires to do
this itself.
•The spectral classes do not always correspond to informational classes.
•The user needs to spend time interpreting and label the classes which
follow that classification.
•Spectral properties of classes can also change over time so you can't have
the same class information while moving from one image to another.
Types of Unsupervised Learning
Unsupervised learning problems further grouped
into clustering and association problems.
Clustering
• Clustering is an important concept when it
comes to unsupervised learning. It mainly deals
with finding a structure or pattern in a
collection of uncategorized data. Clustering
algorithms will process your data and find
natural clusters(groups) if they exist in the data.
You can also modify how many clusters your
algorithms should identify. It allows you to
adjust the granularity of these groups.
Types of Unsupervised Learning: Association
• Association rules allow you to establish
associations amongst data objects inside
large databases.
• This unsupervised technique is about
discovering interesting relationships
between variables in large databases.
• Examples:
• A subgroup of cancer patients grouped by
their gene expression measurements
• Groups of shopper based on their
browsing and purchasing histories
• Movie group by the rating given by movies
viewers
• People that buy a new home most likely to
buy new furniture
Association Rules Network with z=BAGEL
Clustering Types

• Hierarchical clustering
• K-means clustering
• Principal Component Analysis
• Singular Value Decomposition
• Independent Component Analysis
Uses of clustering algorithms

• Engineering sciences: pattern recognition, artificial intelligence,

cybernetics etc. Typical examples to which clustering has been applied
include handwritten characters, samples of speech, fingerprints, and
pictures.

• Life sciences (biology, botany, zoology, entomology, cytology,

microbiology): the objects of analysis are life forms such as plants,
animals, and insects.

• Information, policy and decision sciences: the various applications of

clustering analysis to documents include votes on political issues, survey
of markets, survey of products, survey of sales programs, and R & D.
What are clustering algorithms?
• Clustering is a technique for finding similarity groups in data, called
clusters. I.e.,
• it groups data instances that are similar to (near) each other in one
cluster and data instances that are very different (far away) from
each other into different clusters.
• Clustering is often called an unsupervised learning task as no class
values denoting an a priori grouping of the data instances are given,
which is the case in supervised learning.
• Due to historical reasons, clustering is often considered synonymous with
unsupervised learning.
• In fact, association rule mining is also unsupervised
 An illustration

• The data set has four

natural groups of data points,
i.e., 4 natural clusters.
What is clustering for?
• Example 1: groups people of similar sizes together to make “small”, “medium” and “large” T-
Shirts.
• Tailor-made for each person: too expensive
• One-size-fits-all: does not fit all.
• Example 2: In marketing, segment customers according to their similarities
• To do targeted marketing.
• Example 3: Given a collection of text documents, we want to organize them according to their
content similarities,
• To produce a topic hierarchy
• In fact, clustering is one of the most utilized data mining techniques.
• It has a long history, and used in almost every field, e.g., medicine, psychology, botany, sociology,
biology, archeology, marketing, insurance, libraries, etc.
• In recent years, due to the rapid increase of online documents, text clustering
becomes important.
Aspects of clustering

• A clustering algorithm
• Partitional clustering
• Hierarchical clustering
• A distance (similarity, or dissimilarity) function
• Clustering quality
• Inter-clusters distance Þ maximized
• Intra-clusters distance Þ minimized
• The quality of a clustering result depends on the algorithm, the distance
function, and the application.
K-means clustering
Algorithm
1.Clusters the data into k groups where k is predefined.
2.Select k points at random as cluster centers.
3.Assign objects to their closest cluster center according to the Euclidean distance function.
4.Calculate the centroid or mean of all objects in each cluster.
5.Repeat steps 2, 3 and 4 until the same points are assigned to each cluster in consecutive
rounds.
Where Should I Set up My New Coffee Shops?
• Let's say we want to open three
new coffee shops in a town
called, George - this is all
hypothetically thinking. However,
we need to scope out the prime
locations for our coffee shops to
thrive.
• Outlined below, we have a data
set of all the people that make
use of coffee shops regularly in
the surrounding area:
Where Should I Set up My New Coffee Shops?
• Let's say we want to open three
new coffee shops in a town
called, George - this is all
hypothetically thinking. However,
we need to scope out the prime
locations for our coffee shops to
thrive.
• Outlined below, we have a data
set of all the people that make
use of coffee shops regularly in
the surrounding area:
Where Should I Set up My New Coffee Shops?
• In step 2, we need to group each
observation to a certain center
point, and we do this by computing
the distance between each
observation and each center point.

• Once this is done, we will classify

the observation by putting it into a
group whose center is closest to it:
Step 3

▪ We can now move on to and re-

compute the group centre by taking the
mean of all the vectors in the group.
▪ Basically, we move our stars (center
points) right to the middle of our
current groupings.

▪ Looking at our results above, it shows

that some of the observations might
now be closer to another center point
than their previous positions.
Repeat steps 2 - 3 for a set number of iterations or when convergence has been
reached
K-means Properties: Optimization

▪ Minimizes aggregate intra(cluster distance)

• total squared distance from point to center of its cluster
• same as variance if Euclidian distance is used
▪ Converges to a local minimum
• different starting points very different results
• run several times with random starting points
• pick clustering that yields smallest aggregate distance
▪ Nearby points may not end up in the same cluster
• the following clustering is a stable local minimum
PRINCIPAL COMPONENT ANALYSIS

▪ Principal Component Analysis, or PCA, is a

dimensionality-reduction method that is often used
to reduce the dimensionality of large data sets, by
transforming a large set of variables into a smaller one
that still contains most of the information in the large
set.
▪ Reducing the number of variables of a data set
naturally comes at the expense of accuracy, but the
trick in dimensionality reduction is to trade a little
accuracy for simplicity. Because smaller data sets are
easier to explore and visualize and make analyzing
data much easier and faster for machine learning
algorithms without extraneous variables to process.
▪ Reduce the number of variables of a data set, while
preserving as much information as possible.
Steps in PCA

STEP 1: STANDARDIZATION:

Mathematically, this can be done by subtracting the mean and

dividing by the standard deviation for each value of each variable.
STEP 2: COVARIANCE MATRIX COMPUTATION
Variables are highly correlated in such a way that they contain
redundant information. So, in order to identify these correlations,
we compute the covariance matrix.
STEP 3: COMPUTE THE EIGENVECTORS AND EIGENVALUES OF THE
COVARIANCE MATRIX TO IDENTIFY THE PRINCIPAL COMPONENTS
Eigenvectors and eigenvalues are the linear algebra concepts that
we need to compute from the covariance matrix in order to
determine the principal components of the data. Before getting to
the explanation of these concepts, let’s first understand what do
we mean by principal components.
STEP 4: FEATURE VECTOR
Choose whether to keep all these components or discard those of lesser significance (of low
eigenvalues), and form with the remaining ones a matrix of vectors that we call feature vector.

LAST STEP : RECAST THE DATA ALONG THE PRINCIPAL COMPONENTS AXES
The aim is to use the feature vector formed using the eigenvectors of the covariance matrix, to reorient the data
from the original axes to the ones represented by the principal components (hence the name Principal
Components Analysis). This can be done by multiplying the transpose of the original data set by the transpose of
the feature vector
 Independent Component Analysis (ICA)

ICA is a machine learning technique to separate independent sources from a

mixed signal. Unlike principal component analysis which focuses on maximizing
the variance of the data points, the independent component analysis focuses on
independence, i.e. independent components.
Difference between PCA and ICA –

PRINCIPAL COMPONENT ANALYSIS INDEPENDENT COMPONENT ANALYSIS

It reduces the dimensions to avoid the It decomposes the mixed signal into its
problem of overfitting. independent sources’ signals.

It deals with the Principal Components. It deals with the Independent Components.

It doesn’t focus on the issue of variance

It focuses on maximizing the variance.
among the data points.

It focuses on the mutual orthogonality It doesn’t focus on the mutual orthogonality

property of the principal components. of the components.

It doesn’t focus on the mutual It focuses on the mutual independence of

independence of the components. the components.
Reinforcement learning
▪ Reinforcement learning is an area of Machine Learning.

▪ It is about taking suitable action to maximize reward in a particular

situation. It is employed by various software and machines to find
the best possible behavior or path it should take in a specific
situation.

▪ Reinforcement learning differs from the supervised learning in a

way that in supervised learning the training data has the answer
key with it so the model is trained with the correct answer itself
whereas in reinforcement learning, there is no answer but the
reinforcement agent decides what to do to perform the given
task.

▪ In the absence of a training dataset, it is bound to learn from its

experience.
▪ Reinforcement learning is the training of machine learning models
to make a sequence of decisions.
Applications of Reinforcement Learning
• Here are applications of Reinforcement Learning:
• Robotics:
• RL is used in Robot navigation, Robo-soccer, walking,
juggling, etc.
• Control:
• RL can be used for adaptive control such as Factory
processes, admission control in telecommunication, and
Helicopter pilot is an example of reinforcement learning.
• Game Playing:
• RL can be used in Game playing such as tic-tac-toe, chess,
etc.
• Chemistry:
• RL can be used for optimizing the chemical reactions.
• Business:
• RL is now used for business strategy planning.
• Manufacturing:
• In various automobile manufacturing companies, the
robots use deep reinforcement learning to pick goods and
put them in some containers.
• Finance Sector:
• The RL is currently used in the finance sector for evaluating
trading strategies.
Characteristics of Reinforcement Learning

Here are important characteristics of reinforcement

learning
•There is no supervisor, only a real number or reward
signal
•Sequential decision making
•Time plays a crucial role in Reinforcement problems
•Feedback is always delayed, not instantaneous
•Agent's actions determine the subsequent data it
receives
Types of Reinforcement Learning

Two kinds of reinforcement learning methods are:

Positive:
It is defined as an event, that occurs because of specific behavior. It
increases the strength and the frequency of the behavior and impacts
positively on the action taken by the agent.
This type of Reinforcement helps you to maximize performance and sustain
change for a more extended period. However, too much Reinforcement may
lead to over-optimization of state, which can affect the results.
Negative:
Negative Reinforcement is defined as strengthening of behavior that occurs
because of a negative condition which should have stopped or avoided. It
helps you to define the minimum stand of performance. However, the
drawback of this method is that it provides enough to meet up the minimum
behavior.
Important terms in Reinforcement Learning

•Agent: It is an assumed entity which performs actions in an environment to gain some reward.
•Environment (e): A scenario that an agent has to face.
•Reward (R): An immediate return given to an agent when he or she performs specific action or task.
•State (s): State refers to the current situation returned by the environment.
•Policy (π): It is a strategy which applies by the agent to decide the next action based on the current state.
•Value (V): It is expected long-term return with discount, as compared to the short-term reward.
•Value Function: It specifies the value of a state that is the total amount of reward. It is an agent which should be
expected beginning from that state.
•Model of the environment: This mimics the behavior of the environment. It helps you to make inferences to be
made and also determine how the environment will behave.
•Model based methods: It is a method for solving reinforcement learning problems which use model-based
methods.
•Q value or action value (Q): Q value is quite similar to value. The only difference between the two is that it takes
an additional parameter as a current action.
• Exploration vs Exploitation trade-off:In order to build an optimal policy, the agent faces the dilemma of exploring
new states while maximizing its reward at the same time.
Difference between Reinforcement learning and Supervised learning:

REINFORCEMENT LEARNING SUPERVISED LEARNING

Reinforcement learning is all about making In Supervised learning the decision is made
decisions sequentially. In simple words we on the initial input or the input given at the
can say that the output depends on the state start
of the current input and the next input
depends on the output of the previous input
In Reinforcement learning decision is Supervised learning the decisions are
dependent, So we give labels to sequences independent of each other so labels are
of dependent decisions given to each decision.
Example: Chess game Example: Object recognition
Markov Decision Processes (MDPs)

▪ MDPs are mathematical frameworks to describe an environment in

reinforcement learning and almost all RL problems can be formalized using
MDPs.

▪ The Markov property states that,“ The future is independent of the past given the
present.”

▪ An MDP consists of a set of finite environment states S, a set of possible

actions A(s) in each state, a real valued reward function R(s) and a transition
model P(s’, s | a).

▪ However, real world environments are more likely to lack any prior
knowledge of environment dynamics.
▪ Model-free RL methods come handy in such cases.
Markov Decision Processes (MDPs)
Markov Decision Processes (MDPs)
Action Based Future Situations
Markov Decision Processes (MDPs)
Action Based Future Situations
Markov Decision Processes (MDPs)
Action Based Future Situations
Q-Learning
Q-learning is a model-free reinforcement learning algorithm.

Q-learning is a values-based learning algorithm.

Value based algorithms updates the value function based on an equation.

Q-learning is an off-policy learner. Means it learns the value of the optimal policy
independently of the agent’s actions.
On the other hand, an on-policy learner learns the value of the policy being carried
out by the agent, including the exploration steps and it will find a policy that is
optimal, taking into account the exploration inherent in the policy.
Q-learning Definition

•Q*(s,a) is the expected value (cumulative discounted reward) of doing a

in state s and then following the optimal policy.

•Q-learning uses Temporal Differences(TD) to estimate the value of

Q*(s,a). Temporal difference is an agent learning from an environment
through episodes with no prior knowledge of the environment.
•The agent maintains a table of Q[S, A], where S is the set
of states and A is the set of actions.
•Q[s, a] represents its current estimate of Q*(s,a).
Q-learning Algorithm Process
Example

Let's understand this method by

the following example:
•There are five rooms in a building
which are connected by doors.
•Each room is numbered 0 to 4
•The outside of the building can
be one big outside area (5)
•Doors number 1 and 4 lead into
the building from room 5
Q-Learning
Next, you need to associate a reward value to each door:
•Doors which lead directly to the goal have a reward of 100
•Doors which is not directly connected to the target room gives
zero reward
•As doors are two-way, and two arrows are assigned for each
room
•Every arrow in the above image contains an instant reward
value

For example, an agent traverse from room number 2 to 5

•Initial state = state 2
•State 2-> state 3
•State 3 -> state (2,1,4)
•State 4-> state (0,5,3)
•State 1-> state (5,3)
•State 0-> state 4
Conclusion:

From the above, we can say that Reinforcement Learning is one of the
most interesting and useful parts of Machine learning.
In RL, the agent explores the environment by exploring it without any
human intervention. It is the main learning algorithm that is used in
Artificial Intelligence.

But there are some cases where it should not be used, such as if you
have enough data to solve the problem, then other ML algorithms can
be used more efficiently.

The main issue with the RL algorithm is that some of the parameters
may affect the speed of the learning, such as delayed feedback.
Thank You