Scribd Logo
Upload

Welcome to Scribd!

  • 15 views

    Decision Tree (Autosaved)

    Uploaded by

    Bhardwaj Diwakar
    Decision trees are a type of supervised learning algorithm that can be used for both classification and regression problems. They use a tree-like model of decisions and their consequences to determine the target value of a new dataset. Decision trees break down a dataset into smaller and smaller subsets while associating decisions with the results. The final subset at the end of the branch is called a leaf node which gives the classification or prediction output. Attributes are selected using measures like information gain or gini index to build the tree. Random forest is an ensemble technique that builds many decision trees and outputs the mode of the classifications.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd

    Decision Tree (Autosaved)

    Uploaded by

    Bhardwaj Diwakar
    15 views14 pages
    Decision trees are a type of supervised learning algorithm that can be used for both classification and regression problems. They use a tree-like model of decisions and their consequences to determine the target value of a new dataset. Decision trees break down a dataset into smaller and smaller subsets while associating decisions with the results. The final subset at the end of the branch is called a leaf node which gives the classification or prediction output. Attributes are selected using measures like information gain or gini index to build the tree. Random forest is an ensemble technique that builds many decision trees and outputs the mode of the classifications.

    Original Title

    Decision Tree [Autosaved]

    Copyright

    © © All Rights Reserved

    Available Formats

    PPTX, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Decision trees are a type of supervised learning algorithm that can be used for both classification and regression problems. They use a tree-like model of decisions and their consequences to determine the target value of a new dataset. Decision trees break down a dataset into smaller and smaller subsets while associating decisions with the results. The final subset at the end of the branch is called a leaf node which gives the classification or prediction output. Attributes are selected using measures like information gain or gini index to build the tree. Random forest is an ensemble technique that builds many decision trees and outputs the mode of the classifications.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd
    Download as pptx, pdf, or txt
    15 views14 pages

    Decision Tree (Autosaved)

    Uploaded by

    Bhardwaj Diwakar
    Decision trees are a type of supervised learning algorithm that can be used for both classification and regression problems. They use a tree-like model of decisions and their consequences to determine the target value of a new dataset. Decision trees break down a dataset into smaller and smaller subsets while associating decisions with the results. The final subset at the end of the branch is called a leaf node which gives the classification or prediction output. Attributes are selected using measures like information gain or gini index to build the tree. Random forest is an ensemble technique that builds many decision trees and outputs the mode of the classifications.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd
    Download as pptx, pdf, or txt
    You are on page 1of 14

    LOGISTIC REGRESSION

    Decision Tree is a Supervised learning technique that can be used for both


    classification and Regression problems, but mostly it is preferred for solving
    Classification problems. It is a tree-structured classifier, where internal nodes
    represent the features of a dataset, branches represent the decision
    rules and each leaf node represents the outcome.
    In a Decision tree, there are two nodes, which are the Decision Node and Leaf
    Node. Decision nodes are used to make any decision and have multiple branches,
    whereas Leaf nodes are the output of those decisions and do not contain any further
    branches.
    The decisions or the test are performed on the basis of features of the given dataset.
    It is a graphical representation for getting all the possible solutions to a
    problem/decision based on given conditions.
    It is called a decision tree because, similar to a tree, it starts with the root node,
    which expands on further branches and constructs a tree-like structure.
    In order to build a tree, we use the CART algorithm, which stands
    for Classification and Regression Tree algorithm.
    A decision tree simply asks a question, and based on the answer (Yes/No), it further
    split the tree into subtrees.
    Below diagram explains the general structure of a decision tree:
    DECISION TREE
    TERMINOLOGIES
    Root Node: Root node is from where the decision tree starts. It represents the entire
    dataset, which further gets divided into two or more homogeneous sets.
    Leaf Node: Leaf nodes are the final output node, and the tree cannot be segregated
    further after getting a leaf node.
    Splitting: Splitting is the process of dividing the decision node/root node into sub-
    nodes according to the given conditions.
    Branch/Sub Tree: A tree formed by splitting the tree.
    Pruning: Pruning is the process of removing the unwanted branches from the tree.
    Parent/Child node: The root node of the tree is called the parent node, and other
    nodes are called the child nodes.
    Q
    Step-1: Begin the tree with the root node, says S, which contains the complete
    dataset.
    Step-2: Find the best attribute in the dataset using Attribute Selection Measure
    (ASM).
    Step-3: Divide the S into subsets that contains possible values for the best attributes.
    Step-4: Generate the decision tree node, which contains the best attribute.
    Step-5: Recursively make new decision trees using the subsets of the dataset created
    in step -3. Continue this process until a stage is reached where you cannot further
    classify the nodes and called the final node as a leaf node.
    ATTRIBUTE SELECTION
    MEASURES
    While implementing a Decision tree, the main issue arises that how to select the best
    attribute for the root node and for sub-nodes. So, to solve such problems there is a
    technique which is called as Attribute selection measure or ASM. By this
    measurement, we can easily select the best attribute for the nodes of the tree. There
    are two popular techniques for ASM, which are:
    Information Gain
    Gini Index
    INFORMATION GAIN:

    Information gain is the measurement of changes in entropy after the segmentation of a dataset based on an attribute.
    It calculates how much information a feature provides us about a class.
    According to the value of information gain, we split the node and build the decision tree.
    A decision tree algorithm always tries to maximize the value of information gain, and a node/attribute having the highest
    information gain is split first. It can be calculated using the below formula:
    Information Gain= Entropy(S)- [(Weighted Avg) *Entropy(each feature) 
    Entropy: Entropy is a metric to measure the impurity in a given attribute. It specifies randomness in data. Entropy can be
    calculated as:
    Entropy(s)= -P(yes)log2 P(yes)- P(no) log2 P(no)
    Where,
    S= Total number of samples
    P(yes)= probability of yes
    P(no)= probability of no
    GINI INDEX:

    Gini index is a measure of impurity or purity used while creating a decision tree in
    the CART(Classification and Regression Tree) algorithm.
    An attribute with the low Gini index should be preferred as compared to the high
    Gini index.
    It only creates binary splits, and the CART algorithm uses the Gini index to create
    binary splits.
    Gini index can be calculated using the below formula:
    Gini Index= 1- ∑jPj2
    RANDOM FOREST
    "Random Forest is a classifier that contains a number of decision trees on various
    subsets of the given dataset and takes the average to improve the predictive accuracy
    of that dataset." Instead of relying on one decision tree, the random forest takes the
    prediction from each tree and based on the majority votes of predictions, and it
    predicts the final output.
    The greater number of trees in the forest leads to higher accuracy and prevents the
    problem of overfitting.
    It can be used for both Classification and Regression problems in ML. It is based on
    the concept of ensemble learning, which is a process of combining multiple
    classifiers to solve a complex problem and to improve the performance of the model.
    STEPS FOR RANDOM FOREST
    ALGORITHM
    Random Forest works in two-phase first is to create the random forest by combining N
    decision tree, and second is to make predictions for each tree created in the first phase.
    The Working process can be explained in the below steps and diagram:
    Step-1: Select random K data points from the training set.
    Step-2: Build the decision trees associated with the selected data points (Subsets).
    Step-3: Choose the number N for decision trees that you want to build.
    Step-4: Repeat Step 1 & 2.
    Step-5: For new data points, find the predictions of each decision tree, and assign the
    new data points to the category that wins the majority votes.

    You might also like