Random Forest is a versatile machine learning algorithm capable of performing both regression and classification tasks. It is an ensemble method that operates by constructing a multitude of decision trees during training and outputting the average prediction of the individual trees (for regression) or the mode of the classes (for classification).

Random Forest is an ensemble learning method used for classification and regression tasks. It is built from multiple decision trees and combines their outputs to improve the model's accuracy and control over-fitting.

For classification, the mode of the classes predicted by individual trees is taken (majority vote).

For regression, the average of the outputs of the individual trees is taken.

Each tree is trained on a random sample of the original data, drawn with replacement (bootstrap sample). This means some data points may appear multiple times in a sample while others may not appear at all.

Each node in the tree is split using the best split among a random subset of the features. This process adds an additional layer of randomness, contributing to the robustness of the model.

For classification tasks, the final prediction is based on the majority vote from all the trees. For regression tasks, the final prediction is the average of all the tree predictions.

Below is a simple example of using Random Forest for a classification task with the Iris dataset.

import numpy as np

import pandas as pd

from sklearn.datasets import load_iris

Random Forest provides a way to measure the importance of each feature in making predictions.

import matplotlib.pyplot as plt

Using Grid Search for hyperparameter tuning.

from sklearn.model_selection import GridSearchCV

Below is a simple example of using Random Forest for a regression task with the Boston housing dataset.

import numpy as np

import pandas as pd

from sklearn.datasets import load_boston