Indian Institute of Information Technology, Bhopal

Department of Information Technology

IV Year VII Semester

DATA SCIENCE (IT - 324)

SUBMITTED TO: SUBMITTED BY:

Dr.Rekha Kaushik Yash Gangwar
(21U03015)

Data science Assignment 1- 2024

1. Q: What is linear regression, and how does it model the relationship between variables?
A: Linear regression is a statistical method used to model the relationship between a dependent
variable (target) and one or more independent variables (predictors). It finds the best-fitting line
by minimizing the sum of squared differences between observed and predicted values. The
equation is:
Y = β0 + β1X + ε
where Y is the dependent variable, X is the independent variable, β0 is the intercept, β1 is the
slope (coefficient), and ε is the error term. Linear regression assumes a linear relationship, can
extend to multiple variables (multiple linear regression), and is often a starting point for more
complex analyses. It aims to minimize the sum of squared residuals using the Ordinary Least
Squares (OLS) method.

2. Q: Define the term 'residual' in the context of linear regression.

A: A residual is the difference between an observed value and the corresponding predicted value
by a regression model. Mathematically:
Residual = yi - ŷi,
where yi is the actual value and ŷi is the predicted value. Residuals represent the vertical
distance between the observed data point and the regression line. Analyzing residuals helps
assess model fit and validate assumptions. Patterns in residuals can indicate issues like non-
linearity or heteroscedasticity. A well-fitted model should have residuals randomly distributed
around zero.
3. Q: What does the coefficient of determination (R²) represent in a linear regression
model?
A: R², or the coefficient of determination, measures the proportion of variance in the dependent
variable that is predictable from the independent variable(s). It ranges from 0 to 1, where R² = 1
indicates perfect prediction, and R² = 0 means the model does not explain any variance. It can be
interpreted as the percentage of variance in Y explained by X but is sensitive to outliers and the
number of predictors. A high R² should be considered alongside other diagnostics as it doesn’t
guarantee a good model.

4. Q: How do you interpret the slope and intercept in a simple linear regression equation?
A:
• Slope (β1): Indicates the change in the dependent variable (Y) for each unit change in
the independent variable (X). For example, if β1 = 2, Y increases by 2 units for every 1
unit increase in X. A negative slope indicates an inverse relationship.
• Intercept (β0): Represents the value of Y when X is 0. While the slope represents the
rate of change, the intercept may not always have a meaningful interpretation, especially
if X=0 is outside the observed data range.
In multiple regression, each slope represents the effect of that variable while holding others
constant.

5. Q: Describe the least squares method. How is it used to estimate the parameters of a
linear regression model? How would you interpret the p-value associated with a regression
coefficient?
A: The least squares method minimizes the sum of the squared residuals (differences between
observed and predicted values). It estimates β0 and β1 by minimizing:
Σ(yi - ŷi)².
The p-value tests the null hypothesis that a coefficient is zero. A small p-value (typically < 0.05)
suggests that the predictor is statistically significant. The least squares method is sensitive to
outliers and finds the line that minimizes squared vertical distances from data points. A small p-
value indicates the variable is a significant predictor, though the strength of the relationship
requires further interpretation.

6. Q: What is the difference between the adjusted R² and the regular R², and why might
you use adjusted R²?
A:
- R² only measures the goodness of fit but increases as more predictors are added, regardless
of whether they improve the model.
- Adjusted R² adjusts for the number of predictors and penalizes the inclusion of unnecessary
variables. It is used when comparing models with different numbers of predictors.

7. Q: Given the age values (13, 15, 16, 16, 19, 20, 20, 21, 22, 22, 25, 25, 25, 25, 30, 33, 33, 35,
35, 35, 35, 36, 40, 45, 46, 52, 70), calculate various statistical measures.
A:
- Mean: 28.85
- Median: 25
- Mode: 25 and 35 (bimodal)
- Midrange: (13 + 70) / 2 = 41.5
- First Quartile (Q1): 20
- Third Quartile (Q3): 35
- Five-number summary: Min = 13, Q1 = 20, Median = 25, Q3 = 35, Max = 70

8. Q: Calculate the following distances between the points (22, 1, 42, 10) and (20, 0, 36, 8):
A:
- Euclidean distance: √((22-20)² + (1-0)² + (42-36)² + (10-8)²) = 6.633
 - Manhattan distance: |22-20| + |1-0| + |42-36| + |10-8| = 10
- Minkowski distance (q=3): ((|22-20|³ + |1-0|³ + |42-36|³ + |10-8|³))^(1/3) = 6.301
- Supremum distance: max(|22-20|, |1-0|, |42-36|, |10-8|) = 6

9. Q: What is the difference between Type I and Type II errors in statistical hypothesis
testing?
A:
• Type I Error: Rejecting the null hypothesis when it is true (false positive).
• Type II Error: Failing to reject the null hypothesis when it is false (false negative).
• Type I Error (α) is the significance level of the test.
• Type II Error (β) is related to the power of the test (1 - β).
• There's often a trade-off between these errors; reducing one type can increase the
other.
• The choice of significance level affects the likelihood of these errors.
10. Q: How do you interpret p-values in the context of hypothesis testing, and what are the
common misconceptions?
A: p-value represents the probability of observing the data, or something more extreme, assuming
the null hypothesis is true. It helps to determine whether the observed effect is statistically significant.
• A small p-value (typically < 0.05) suggests rejecting the null hypothesis.
• A large p-value suggests that there isn't enough evidence to reject the null hypothesis.
Misconceptions:
• A p-value does not prove the alternative hypothesis; it only indicates evidence against
the null hypothesis.
• A low p-value does not imply that the effect is practically significant—it only shows
statistical significance.

11. Q: What is the purpose of confidence intervals, and how do you interpret them in data
science experiments?
A: Confidence intervals provide a range of values within which the true population parameter
is likely to fall, with a specified level of confidence (e.g., 95%). It gives more context to point
estimates by quantifying uncertainty.

12. Q: What is multicollinearity, and why is it a problem in regression analysis?

A: Multicollinearity occurs when independent variables are highly correlated, which makes it
difficult to determine the individual effect of each variable on the dependent variable. This can
lead to unreliable and unstable estimates of the regression coefficients.
13. Q: When would you use a t-test vs. an ANOVA test in data science?
A:
- A t-test is used to compare the means of two groups.
- ANOVA (Analysis of Variance) is used when comparing the means of three or more groups.

14. Q: How do you assess the goodness of fit for a regression model using statistical
metrics?
A:
- R² measures the proportion of variance explained by the model.
- Adjusted R² adjusts for the number of predictors.
- RMSE (Root Mean Square Error) and MSE (Mean Square Error) quantify prediction errors.

15. Q: What is the difference between parametric and non-parametric statistical tests, and
when would you use each?
A:
• Parametric tests assume the data follows a specific distribution (e.g., t-test).
• Non-parametric tests make no assumptions about the data's distribution (e.g., Mann-
Whitney U test). Non-parametric tests are used when the data doesn't meet the
assumptions required for parametric tests.
• Parametric tests are generally more powerful when assumptions are met.
• Non-parametric tests are more robust to outliers and work well with ordinal data.
• Examples of non-parametric tests include Wilcoxon rank-sum test, Kruskal-Wallis test,
and Spearman's rank correlation.
• The choice between parametric and non-parametric tests often depends on sample size,
data distribution, and measurement scale.

16. Q: What are the assumptions of linear regression, and how do you check if they hold in
a dataset?
A:
• Linearity: The relationship between the predictors and the target is linear.
• Independence: Observations are independent of each other.
• Homoscedasticity: The variance of residuals is constant across all levels of the
independent variable.
• Normality of residuals: The residuals should be normally distributed. You can check
these assumptions by plotting residuals and conducting statistical tests like the Durbin-
Watson test for independence.
 • Scatter plots of predictor variables vs. the target variable to check linearity.
• Q-Q plots of residuals to check normality.
• Residual vs. fitted value plots to check homoscedasticity.
• Variance Inflation Factor (VIF) to check for multicollinearity.
17. Q: What are some of the techniques used for sampling? What is the main advantage of
sampling?
A:
• Techniques: Simple random sampling, stratified sampling, cluster sampling, and
systematic sampling are common techniques.
• Main advantage: Sampling allows making inferences about a population without
needing to collect data from the entire population, which is time-saving and cost-
effective.
• Stratified sampling ensures representation of subgroups in the population.
• Cluster sampling is useful when it's difficult to sample individuals randomly.
• Convenience sampling, though not random, is sometimes used due to practical
constraints.
• The choice of sampling technique depends on the research question, population
characteristics, and available resources.

18. Q: What are RMSE and MSE in a linear regression model?

A:
- MSE (Mean Square Error): The average of the squared differences between observed and
predicted values.
MSE = (1/n) Σ(yi - ŷi)²
- RMSE (Root Mean Square Error): The square root of MSE and provides the error in the
same units as the dependent variable.
RMSE = √MSE

19. Q: Given a dataset with actual values Y = [5, 8, 12, 10, 15] and predicted values Y_pred
= [4, 7, 10, 11, 13], calculate the bias and explain how to calculate the variance.
A:
• Bias: The average difference between the actual and predicted values. Bias = (1/n)
Σ(Y_pred - Y) = -0.6
• Variance: The variability of predictions. Variance = (1/(n-1)) Σ(Y_pred - Ȳ_pred)²
• Bias measures the average tendency of the model to over or underpredict.
• Negative bias indicates underprediction on average.
• Variance measures how spread out the predictions are.
 • Low bias and low variance are desirable for a good model (bias-variance tradeoff).

20. Q: Given X = [-2, -1, 0, 1, 2, 3, 4, 5, 6, 7] and Y = [-0.5267, 1.3517, 3.8308, 5.5853,
7.5497, 9.9172, 11.2858, 13.7572, 15.7537, 17.3804], how would you find the parameters of
the linear regression model and MSE?
A: To find the parameters of a linear regression model (slope β1\beta_1β1 and intercept β0\
beta_0β0) and the mean squared error (MSE) for the given data, you can follow these steps:
1. Slope (β1\beta_1β1) and Intercept (β0\beta_0β0) Using the Least Squares Method:
The goal of linear regression is to find a line that minimizes the sum of squared residuals (the
differences between observed and predicted values).
The formulas for the slope and intercept are:
• Slope β1\beta_1β1:
β1=∑(Xi−Xˉ)(Yi−Yˉ)∑(Xi−Xˉ)2\beta_1 = \frac{\sum (X_i - \bar{X})(Y_i - \bar{Y})}{\sum
(X_i - \bar{X})^2}β1=∑(Xi−Xˉ)2∑(Xi−Xˉ)(Yi−Yˉ)
Where Xˉ\bar{X}Xˉ and Yˉ\bar{Y}Yˉ are the means of the X and Y values, respectively.
• Intercept β0\beta_0β0:
β0=Yˉ−β1Xˉ\beta_0 = \bar{Y} - \beta_1 \bar{X}β0=Yˉ−β1Xˉ
Pros:
• Simplicity: The least squares method is a straightforward approach that minimizes the
sum of squared residuals, providing a simple formula for slope and intercept.
• Interpretability: The result is easy to interpret in terms of linear relationships between
the variables.
Cons:
• Sensitivity to Outliers: Least squares can be highly affected by outliers, which can
distort the slope and intercept.
• Assumptions: It assumes that the relationship between the variables is linear and that the
residuals are normally distributed with constant variance (homoscedasticity), which
might not always hold.
2. Mean Squared Error (MSE):
Once you have the slope and intercept, you can predict the values of Y for each corresponding X
using the formula:
Yi^=β0+β1Xi\hat{Y_i} = \beta_0 + \beta_1 X_iYi^=β0+β1Xi
Where Yi^\hat{Y_i}Yi^ is the predicted value of YYY.
To calculate the MSE:
MSE=1n∑i=1n(Yi−Yi^)2MSE = \frac{1}{n} \sum_{i=1}^{n} (Y_i - \hat{Y_i})^2MSE=n1
i=1∑n(Yi−Yi^)2
Where nnn is the number of data points, YiY_iYi is the actual Y value, and Yi^\hat{Y_i}Yi^ is
the predicted Y value.
Pros:
• Effective Error Measurement: MSE gives a clear measure of how well the model fits
the data by penalizing large residuals (differences between predicted and actual values).
• Widely Used: It is a standard metric that is easy to compute and interpret.
Cons:
• Sensitive to Outliers: Like least squares, MSE can be influenced heavily by outliers, as
the errors are squared.
• Lacks Interpretability: MSE gives squared units, making it harder to directly interpret
in the same scale as the data. To avoid this, Root Mean Squared Error (RMSE) is
sometimes preferred.
In practice, after calculating the slope and intercept, you could use them to predict Y-values for
each X and then compute the MSE to assess the model's accuracy.

21. Q: How would you fit a third-degree polynomial regression line for the dataset: x = [0,
2, 4, 6, 8], Y = [0, 7, 63, 215, 510]? How would you predict Y for x = 3?
A:  Fit a polynomial regression of the form Y = β0 + β1x + β2x² + β3x³ using least squares
method. Then use the resulting equation to predict Y for x = 3. Additional steps:
• Use a library like NumPy or scikit-learn to perform polynomial regression.
• Create a design matrix with columns [1, x, x², x³].
• Use ordinary least squares to find the coefficients β0, β1, β2, β3.
• Once you have the equation, substitute x = 3 to predict Y.
• Evaluate the model fit using R² or RMSE.
• Consider cross-validation to assess how well the model generalizes.
22. Q: What are the basic assumptions of the Linear Regression Algorithm?
A: The basic assumptions are linearity, independence, homoscedasticity, normality of
residuals, and no multicollinearity.
• Linearity: The relationship between X and Y is linear.
• Independence: Observations are independent of each other.
• Homoscedasticity: Constant variance of residuals across all levels of X.
• Normality of residuals: The residuals should be normally distributed.
• No multicollinearity: Independent variables are not highly correlated with each other.
• No endogeneity: The independent variables are not correlated with the error term.

23. Q: Explain the Gradient Descent algorithm with respect to linear regression.
 A:  Gradient descent is an optimization algorithm used to minimize the cost function by
iteratively updating the model parameters in the direction of the negative gradient of the cost
function. Additional details:
• It starts with initial values for the parameters and iteratively improves them.
• The learning rate determines the step size in each iteration.
• Types include batch gradient descent, stochastic gradient descent, and mini-batch
gradient descent.
• The algorithm converges when the change in the cost function becomes very small.
• It can be used for both linear and non-linear optimization problems.
24. Q: Explain exploratory data analysis. What are some basic operations to perform EDA
in Python?
A: Exploratory Data Analysis (EDA) is the process of summarizing and visualizing the key
characteristics of a dataset, often using statistical graphics. Basic operations in Python include:
• df.describe() for summary statistics,
• df.info() for data types,
• sns.pairplot(df) for pairwise relationships.
• Histograms and box plots for distribution analysis
• Correlation matrices and heatmaps for variable relationships
• Time series plots for temporal data
• Missing value analysis and handling
• Outlier detection using IQR or z-score methods
• Feature engineering based on initial insights

25. Q: Explain Decision tree induction.

A : Decision tree induction is a supervised learning method used for classification and
regression. It splits the data into subsets based on the feature that provides the highest
information gain or Gini index, creating a tree-like structure.
• It uses a top-down, recursive approach known as recursive partitioning.
• Popular algorithms include ID3, C4.5, and CART.
• Prone to overfitting if not pruned or regularized.
• Can handle both numerical and categorical data.
• Provides interpretable models with feature importance.
• Forms the basis for ensemble methods like Random Forests and Gradient Boosting.
 26. Q: Explain Standardization and normalization.
 A:
Standardization transforms data to have a mean of 0 and a standard deviation of 1.
Normalization scales data to a range (typically [0, 1]). These techniques are used to ensure that
features have similar scales and improve the performance of machine learning algorithms.
Standardization formula: z = (x - μ) / σ
Normalization formula: x_norm = (x - x_min) / (x_max - x_min)
Standardization is preferred when the data follows a normal distribution.
Normalization is useful when you need bounded values.
Both help in faster convergence for gradient descent-based algorithms.
Some algorithms (e.g., neural networks) often perform better with scaled data.
It's important to apply the same scaling to both training and test data.