Scribd Logo
Upload

Welcome to Scribd!

  • 35 views

    Clustering Technique: Mohammad Ali Joneidi

    Uploaded by

    madal
    Clustering techniques like k-means, hierarchy, and self-organizing maps can be applied to problems that have already been solved with supervised classification. The document discusses using these clustering methods on a face detection problem previously solved with supervised learning. It provides details on how each clustering technique works, including defining the Euclidean distance and updating weights. For k-means, random initial centers from each class are chosen. The self-organizing map uses a neighborhood function and learning rate to adjust weights of nearby nodes based on a best matching unit. It achieved best performance on the face detection data after 2000 training epochs.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Clustering Technique: Mohammad Ali Joneidi

    Uploaded by

    madal
    35 views3 pages
    Clustering techniques like k-means, hierarchy, and self-organizing maps can be applied to problems that have already been solved with supervised classification. The document discusses using these clustering methods on a face detection problem previously solved with supervised learning. It provides details on how each clustering technique works, including defining the Euclidean distance and updating weights. For k-means, random initial centers from each class are chosen. The self-organizing map uses a neighborhood function and learning rate to adjust weights of nearby nodes based on a best matching unit. It achieved best performance on the face detection data after 2000 training epochs.

    Original Description:

    clustring via kmeans and som and hirechi

    Original Title

    Report

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Clustering techniques like k-means, hierarchy, and self-organizing maps can be applied to problems that have already been solved with supervised classification. The document discusses using these clustering methods on a face detection problem previously solved with supervised learning. It provides details on how each clustering technique works, including defining the Euclidean distance and updating weights. For k-means, random initial centers from each class are chosen. The self-organizing map uses a neighborhood function and learning rate to adjust weights of nearby nodes based on a best matching unit. It achieved best performance on the face detection data after 2000 training epochs.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    35 views3 pages

    Clustering Technique: Mohammad Ali Joneidi

    Uploaded by

    madal
    Clustering techniques like k-means, hierarchy, and self-organizing maps can be applied to problems that have already been solved with supervised classification. The document discusses using these clustering methods on a face detection problem previously solved with supervised learning. It provides details on how each clustering technique works, including defining the Euclidean distance and updating weights. For k-means, random initial centers from each class are chosen. The self-organizing map uses a neighborhood function and learning rate to adjust weights of nearby nodes based on a best matching unit. It achieved best performance on the face detection data after 2000 training epochs.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 3

    Clustering Technique

    Mohammad Ali Joneidi


    Department of Electronic and Computer Engineering
    Shahid Rajaee Univercity ,Tehran Iran
    joneidimohamad@gmail.com

    AbstractClustering is traditionally viewed as an unsupervised


    method for data analysis. However, in some cases information
    about the problem domain is available in addition to the data
    instances themselves. In this report, we demonstrate how the
    popular clustering method such as k-means,hierarchy and som
    can cluster a solved problem. In this experiments we use
    clustering method to cluster face detection problem that solved
    befor with supervised classifications.
    Index
    TermsK-means,Hierarchy,Self
    Oriented
    Mapping,Clustering

    I. I NTRODUCTION
    Clustering algorithms are generally used in an unsupervised
    fashion. They are presented with a set of data instances that
    must be grouped according to some notion of similarity. The
    algorithm has access only to the set of features describing
    each object. it is not given any information (e.g., labels) as
    to where each of the instances should be placed within the
    partition. However, in real application domains, it is often
    the case that the experimenter possesses some background
    knowledge (about the domain or the data set) that could be
    useful in clustering the data. Traditional clustering algorithms
    have no way to take advantage of this information even when
    it does exist.In continues We use this clustering techniques
    1) Hierarchy
    2) K-means
    3) Self Oriented Map

    Fig. 2.

    III. H IERARCHY
    Given a set of N items (in this problem 400) to be clustered,
    and an NN distance (or similarity) matrix, the basic process
    of hierarchical clustering is this:
    1) Start by assigning each item to its own cluster, so that
    if you have N items, you now have N clusters, each
    containing just one item. Let the distances (similarities)
    between the clusters equal the distances (similarities)
    between the items they contain.
    2) Find the closest (most similar) pair of clusters and merge
    them into a single cluster, so that now you have one less
    cluster.
    3) Compute distances (similarities) between the new cluster
    and each of the old clusters.
    4) Repeat steps 2 and 3 until all items are clustered into a
    single cluster of size N.

    II. E UCLIDEAN DISTANCE


    for calculating similarity we use Euclidean distance because
    our data feature vector is numerical and best way to compare
    numeric data is comparing their distance

    Fig. 1.

    for making distance bigger to completely separate each other


    in this problem we dont Calculate root and just Calculate
    Square distance
    Fig. 3. diagram of hierarchy clustering

    In step 3 we Calculate Euclidean distance between train data


    (384 1) and means of all data labeled to each Cluster
    IV. K- MEANS
    K-means is one of the simplest unsupervised learning
    algorithms that solve the well known clustering problem.
    The procedure follows a simple and easy way to classify a
    given data set through a certain number of clusters (in this
    problem 40 cluster) fixed a priori. The main idea is to define
    40 centroids, one for each cluster. These centroids shoud be
    placed in a cunning way because of different location causes
    different result. So, the better choice is to place them as much
    as possible far away from each other.for better performance
    we choose each centroids from one class.The next step is to
    Calculate each point distance to a given data set and associate
    it to the nearest centroid. When no point is pending, the
    clustering is completed and grouping is done.

    V. SOM
    Kohonen Self-Organizing Maps are a type of neural
    network.Self- Organizing Maps are aptly named. SelfOrganizing is because no supervision is required. SOMs
    learn on their own through unsupervised competitive learning.
    Maps is because they attempt to map their weights to conform
    to the given input data. The nodes in a SOM network attempt
    to become like the inputs presented to them. In this sense,
    this is how they learn. They can also be called Feature Maps,
    as in Self-Organizing Feature Maps. Retaining principle
    features of the input data is a fundamental principle of
    SOMs, and one of the things that makes them so valuable.
    The structure of a SOM is fairly simple, and is best understood
    with the use of an illustration such as Figure 4
    Figure 4 is a 4x4 SOM network . It is easy to overlook this
    structure , but there are a few key things to notice. First, each
    map node is connected to each input node. For this small
    4x4 node network, that is 4x4x3=48 connections.Secondly,
    notice that map nodes are not connected to each other. The
    nodes are organized in this manner, as a 2-D grid makes
    it easy to visualize the results. This representation is also
    useful when the SOM algorithm is used. In this configuration,
    each map node has a unique (i,j) coordinate. This makes it
    easy to reference a node in the network, and to calculate the
    distances between nodes. Because of the connections only
    to the input nodes, the map nodes are oblivious as to what
    values their neighbors have. A map node will only update its
    weights (explained next) based on what the input vector tells it.

    Fig. 4. Structure of SOM

    2) A vector is chosen at random from the set of training


    data and presented to the network.(a random number
    between 1 and 400)
    3) . Every node in the network is examined to calculate
    which ones weights are most like the input vector. The
    winning node is commonly known as the Best Matching
    Unit (BMU).
    4) . The radius of the neighborhood of the BMU is calculated. This value starts large. Typically it is set to
    be the radius of the network, diminishing each timestep. (we consider this value by distance/variance that
    variance depended to number of iteration).
    5) . Any nodes found within the radius of the BMU,
    calculated , are adjusted to make them more like the
    input vector . The closer a node is to the BMU, the
    more its weights are altered.
    6) . Repeat 2) for N iterations.
    in the step 3 we update map node weight base on this formula:
    W (t + 1) = W (t) + (t) (D(t) W (t))

    A. The SOM Algorithm


    The Self-Organizing Map algorithm can be broken up into
    6 steps.
    1) Each nodes weights are initialized.(in face detection
    problem we select random number between 0 and 1

    where W (t + 1) is news map node weights,W (t) is


    old map node weights,(t) is a restraint due to distance from
    BMU, usually called the neighborhood function, is learning
    rate that such as Radius depended to number of iteration and
    D(t) is the input data selected in step 2

    Fig. 6. Accuracy of Som for face detection problem for various epoch number

    over fitting
    VII. R EFERENCES

    Fig. 5. Size of Radius depended to Iteration

    in this example we consider a variable called variance tha


    t calculated as fallow in each iteration:
    
    
    variance = exp 2iterationnumber
    epochnumber
    we use variance to determine radius of each iteration
    as fallow:
    
    
    distance
    exp 2variance
    that distance is the square distance between neighbor
    node and winner node row and col and at the end we
    calculate in each iteration as fallow :
    
    
    = exp iterationnumber
    epochnumber
    we have 400 data instance we know that there are 40
    cluster so we define map with size 5 8 that each node is
    a 384 1 vector. figure 6 demonstrate performance of Som
    technique for various number of epoch

    VI. C ONCLUSION
    In hierarchy clustering we have low performance because
    our data are not hierarchy base in k-means clustering with
    random initial center have low accuracy for enhancing performance we consider center of each cluster one of that cluster
    sample randomize.in som clustering after 2000 epoch for 400
    train we have best performance and for higher epoch we have

    Kohonen Self-Organizing Maps by Shyam M.


    Guthikonda 2005
    Self Organizing Maps Fundamentals by John A. Bullinaria 2004
    Constrained K-means Clustering with Background
    Knowledge by Kiri Wagsta,Claire Cardie 2000

    You might also like