COMP5318

notebookp1-9-10
September 14, 2023
1 Part 1: Classification
Please include any imports (allowed by Ed) you require throughout your notebook in the first cell.
[2]: # Import all libraries

# to make this notebook's output stable across runs
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import os
from scipy import signal
%matplotlib inline
np.random.seed(0)
1.1 Data loading and preprocessing

Load the cardio_diseases_modified.csv dataset and set the X and y variables to the data and class
respectively.
You will need to load this file into numpy arrays for the attribute data and the labels. So that we
can test your code more effectively, please complete this task inside the given function scaffold, and
have your function return these arrays.
Begin by looking at the file and its format. Notice any missing values and how they are encoded in
the file. In the returned X array, missing values should be encoded as np.nan. All numerical values
should be positive numbers, and any invalid numbers should be encoded as np.nan.
This dataset includes categorical values which should be changed to numerical values for some
methods. Binary categorical values [No,Yes] or [F,M] should be replaced to [0,1]. Other categorical
values should be changed from [Normal, Above normal, Well above normal] to [0,0.5,1]
While there are multiple ways to load the file correctly, a suggested function to use is pd.read_csv.
Look through the documentation to check which arguments you will need to pass to the function to
load the file correctly. If you choose to use this approach, you will need to extract the appropriate
numpy arrays from the pandas dataframe, and exclude any headers.
The X array returned by your function should have shape (number of examples, number of at-
tributes), and the y array returned by your function should have shape (number of examples,). We
will also test your function with some different datasets with the same data types, delimiters, and
encoding of missing values. However, these files may have a different filename, number of examples
1
and/or attributes, so you should not hard code these values in your solution. There will not be any
missing target values, and the targets will always be in the final column.
[3]: ### TEST FUNCTION: test_data_loading
# DO NOT REMOVE THE LINE ABOVE
import os
def load_heart_disease_data(filename):
"""Load the dataset located at the filname string as described above."""
data = pd.read_csv(filename)
#data = data.replace(to_replace='?', value=np.nan)
#data = data.replace(to_replace='', value=np.nan)
#data = data.replace(to_replace='Yes', value=1)
#data = data.replace(to_replace='No', value=0)
#data = data.replace(to_replace='F', value=0)
#data = data.replace(to_replace='M', value=1)
#data = data.replace(to_replace='Normal', value=0)
#data = data.replace(to_replace='Above normal', value=0.5)
#data = data.replace(to_replace='Well above normal', value=1)
df = pd.DataFrame(data)
non_numeric_columns = df.select_dtypes(exclude=[np.number])
df = df.select_dtypes(include=[np.number])
df[df < 0] = np.nan
df = pd.concat([df, non_numeric_columns], axis=1)
df.replace('Normal', 0, inplace=True)
df.replace('Above normal', 0.5, inplace=True)
df.replace('Well above normal', 1, inplace=True)
df.replace('F', 0, inplace=True)
df.replace('No', 0, inplace=True)
df.replace('M', 1, inplace=True)
df.replace('Yes', 1, inplace=True)
df.replace('', np.nan, inplace=True)
df.replace('?', np.nan, inplace=True)
df = df.applymap(lambda x: np.nan if isinstance(x, str) else x)
X = df.iloc[:, :-1].values
y = df.iloc[:, -1].values
#X=data.iloc[:,:-1].values
#y = data.iloc[:,-1].values
z=df.iloc[548:562,:]
print(X)
#print(y)
print(z)
#print(data)
#print('The shape of X: ',X.shape)
#print('The shape of y: ',y.shape)
2
# TODO
return X, y
#return None
X,y=load_heart_disease_data('cardio_diseases.csv')
[[ 51. 162. 82. … 0. 0. 1.]

[ 50. 172. 65. … 0. 0. 1.]
[ 58. 169. 79. … 0. 0. 0.]
…
[ 63. 161. 69. … 0. 0. 1.]
[ 48. 160. 58. … 0. 0. 0.]
[ 56. 163. 64. … 0. 1. 1.]]
AGE HEIGHT WEIGHT AP_HIGH AP_LOW GENDER CHOLESTEROL GLUCOSE \
548 58 165.0 87.0 120.0 80.0 0 0.0 0.0
549 56 NaN 78.0 120.0 80.0 1 0.0 0.0
550 44 170.0 64.0 120.0 80.0 1 0.0 0.0
551 39 165.0 65.0 80.0 120.0 0 0.0 0.0
552 49 165.0 60.0 110.0 70.0 0 0.0 0.0
553 46 178.0 78.0 110.0 70.0 1 0.5 0.5
554 44 147.0 53.0 90.0 70.0 0 0.0 0.0
555 44 163.0 66.0 NaN 80.0 0 0.0 0.0
556 42 166.0 61.0 120.0 80.0 1 0.5 0.5
557 55 169.0 73.0 110.0 70.0 1 0.0 0.0
558 48 188.0 89.0 110.0 70.0 1 0.0 0.0
559 54 154.0 98.0 130.0 80.0 0 0.0 0.0
560 50 176.0 72.0 120.0 80.0 1 0.0 0.0
561 46 164.0 62.0 110.0 70.0 0 0.0 0.0
SMOKE ALCOHOL PHYSICAL_ACTIVITY CARDIO_DISEASE

548 0.0 0.0 0.0 0
549 0.0 0.0 1.0 0
550 1.0 0.0 1.0 0
551 0.0 0.0 1.0 0
552 0.0 0.0 1.0 0
553 0.0 0.0 0.0 0
554 0.0 0.0 1.0 0
555 0.0 0.0 1.0 0
556 0.0 0.0 1.0 0
557 0.0 0.0 1.0 0
558 1.0 0.0 1.0 0
559 0.0 0.0 1.0 0
560 0.0 0.0 1.0 0
561 0.0 0.0 1.0 0
[4]: ### SKIP

# This cell won't be marked. Use it to try out your code.
3
Next, you should investigate the attributes of the dataset and find any invalid values for any
attributes of the data.
Replace any missing or invalid values with the mean of that attribute for numeric data and the most
frequent value for categorical data. Take a look at the following documentation to see a suitable
function to perform this using sklearn: Documentation.
Many of the classification algorithms we will use will benefit from normalisation, so you should
perform min-max normalisation on appropriate attributes.
[5]: ### TEST FUNCTION: test_preprocessing
from sklearn.impute import SimpleImputer
import scipy.sparse as sp
from sklearn.preprocessing import MinMaxScaler
def process_data(X):
"""Fill missing (np.nan) values in the input array as described above."""
"""Scale data using MinMaxScaler as described above."""
df=pd.DataFrame(X)
catecolumns=[]
numcolumns=[]
for column in df.columns:
unique=df[column].unique()
if 0 in unique and 1 in unique:
catecolumns.append(column)
else:
numcolumns.append(column)
zh = SimpleImputer(missing_values=np.nan,strategy='most_frequent')
df[catecolumns]=zh.fit_transform(df[catecolumns])
#print(catecolumns)
#print(df[catecolumns])
#z=np.bincount(data[column])
#iz=int(z)
#iz=z.astype(np.int)
#data[column] = data[column].replace(to_replace='', value=iz)
#data[column] = data[column].replace(to_replace=np.nan, value=z)
#print(data[column])
#print(df[321:327])
X=df.values
imp = SimpleImputer(missing_values=np.nan, strategy='mean')
z1=imp.fit_transform(X)
#print(z1[431:440])
tool = MinMaxScaler(feature_range=(0, 1)) #��1.��0
X_scaled = tool.fit_transform(z1) #��values�array
# TODO
#print(X_scaled[431:440])
return X_scaled
4
X_scaled=process_data(X)
[6]: ### SKIP

1.2 Implementing classification algorithms and comparing with 10-fold strati-

fied cross-validation.
For the following tasks, you are required to implement functions which create algorithms and
evaluate them with 10 fold cross validation.
In order to make this reproducible, it is important that the folds are kept consistent across runs.
You can utilise sklearn.model_selection.StratifiedKFolds (documentation) to provide con-
sistency in the generation of the 10 folds. This object can be passed into the cv argument of
cross_val_score. Make sure to shuffle the data in this process, and set random_state to be 0.
Use the variable name cvKFold.
[7]: ### TEST FUNCTION: test_cvkfold
from sklearn.model_selection import StratifiedKFold
cvKFold=StratifiedKFold(n_splits=10, random_state=0, shuffle=True)
cvKFold.get_n_splits(X_scaled, y)
print(cvKFold)
StratifiedKFold(n_splits=10, random_state=0, shuffle=True)
[8]: ### SKIP

K-Nearest Neighbors
We have seen how to implement a KNN classifier in the lab. Your task is to implement a KNN for
classification using sklearn.neighbors.KNeighborsClassification (documentation)
Use the function definition below, so that any appropriate hyperparameters can optionally be passed
in and accessed as a dictionary. Note you also can pass arguments as a dictionary to functions (such
as sklearn constructors) using the ** syntax.
Fill in the function to perform K-nearest neighbors. Test the function with K=11 and Manhattan
distance.
[9]: ### TEST FUNCTION: test_k_nearest_neighbors
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score
from sklearn.model_selection import cross_val_score
#K-Nearest Neighbors
def knnClassifier(X, y, **hyperparams):
knn= KNeighborsClassifier( **hyperparams) #1:manhattan,2:eu
5
knn.fit(X, y)
folds=cross_val_score(knn,X,y,cv=cvKFold)
#for fold,folds in enumerate(folds,start=1):
#print(f"Fold{fold}:Accuracy={folds:.2f}")
#print("Cross-validation scores:{}".format(folds))
score=folds.mean()
return knn,score
# TODO
# return model, score
#return None
knn,score=knnClassifier(X_scaled,y,n_neighbors=11,p=1)
print(f"Cross validation score: {score:.2f}")
Cross validation score: 0.71
[10]: ### SKIP

Naive Bayes
Fill in the function to use the Gaussian Naive Bayes function on all attributes. Use the sklearn
implementation in documentation.
The format of your output should be:
Cross validation score: x.xx
[11]: ### TEST FUNCTION: test_naive_bayes
from sklearn.naive_bayes import GaussianNB
from sklearn.model_selection import train_test_split
from pandas.core.common import random_state
#Naïve Bayes
def nbClassifier(X, y):
nb=GaussianNB()
#nb.fit(X_train,y_train)
#y_pred=nb.predict(X_test)
#print("Accuracy on test set:{:.3f}".format(accuracy_score(y_test,y_pred)))
scores=cross_val_score(nb,X,y,cv=cvKFold)
#print("Cross_validation scores: {}".format(scores)) #accuracy for each fold
meanscore=scores.mean()
# TODO
return nb, meanscore
6
nb, meanscore=nbClassifier(X_scaled, y)
print("Cross validation score: {:.2f}".format(meanscore)) #mean of all folds
[12]: ### SKIP

There is an issue with our current implementation of Naive Bayes using GaussianNB above. Gaus-
sianNB can still work effectively with these attributes, but a slight improvement can be made by
removing these attributes.
Answer the Part 1 Question on Gaussian Naive Bayes.
Decision Tree
As shown in the tutorials, decision trees can often perform well in classification tasks. Fill in the
function to perform classifier with a decision tree classifier. Test the function with entropy, max
depth of 5 and sqrt max features. Read through documentation
[13]: ### TEST FUNCTION: test_decision_tree_classifier
from sklearn.tree import DecisionTreeClassifier
#Decision Trees
def dtClassifier(X, y, **hyperparams):
dt = DecisionTreeClassifier(**hyperparams,random_state=0)
scores=cross_val_score(dt, X, y, cv=cvKFold)
#print("Cross validation score: {:.2f}".format(scores.mean())) #mean of all␣
↪folds
return dt,meanscore
# TODO
dt,meanscore=dtClassifier(X_scaled, y,␣
↪criterion='entropy',max_depth=5,max_features='sqrt')
print("Cross validation score: {:.2f}".format(meanscore))
[14]: ### SKIP

7
Support Vector Machine
Fill in the function to perform a linear support vector machine classifier. Try utilising
sklearn.svm.LinearSVC. Test the function with lasso regularization, C = 0.005 and set dual
to “auto”.
[15]: ### TEST FUNCTION: test_svm
from sklearn.svm import LinearSVC
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import make_classification
#Support Vector Machine

def svmClassifier(X, y, **hyperparams):
svm = LinearSVC(**hyperparams)
#svm.fit(X, y)
scores=cross_val_score(svm, X, y, cv=cvKFold)
↪folds
# TODO
return svm,meanscore
svm,meanscore=svmClassifier(X_scaled, y,penalty='l1',dual=False,C=0.005)
[16]: ### SKIP

1.3 Ensemble Methods

Ensembles are powerful tools in machine learning that seek to improve predictive performance by
combining predictions from multiple models.
Bagging with logistic regression
Fill in the function to perform bagging with logistic regression as the base classifier. Test the
bagging with 20 estimators and a maximum of half of the samples. The logistic regression should
be set with an C of 2 and using ridge regularisation.
Hint: The hyperparams dict should be split to only pass the relevant hyperparameters to bagging
and the logistic regression.
8
[17]: ### TEST FUNCTION: test_bagging_lr
from sklearn.ensemble import BaggingClassifier
from sklearn.linear_model import LogisticRegression
#Bagging
def baggingClassifier(X, y, **hyperparams):
logistic_regression_params = {
key: value for key, value in hyperparams.items() if key in ['C',␣
↪'penalty']
}
bagging_params = {
key: value for key, value in hyperparams.items() if key in␣
↪['n_estimators', 'max_samples']
model = BaggingClassifier(LogisticRegression(**logistic_regression_params,␣
↪random_state=0), **bagging_params, random_state=0)
scores = cross_val_score(model, X, y, cv=cvKFold)

score = np.mean(scores)
return model, score

bag_clf, score = baggingClassifier(X_scaled, y, n_estimators=20, max_samples=0.
↪5, C=2, penalty='l2')
print("Cross validation score: {:.2f}".format(score))
[18]: ### SKIP

Adaboost on DT
Fill in the function to perform boosting with Adaboost on a decision tree as the base classifier.
Test your function with the Adaboost with 25 estimators and a learning rate of 0.1. The decision
tree should have a max depth of 3 with a log loss criterion.
Hint: The hyperparams dict should only pass the relevant hyperparameters to Adaboost and the
decision tree.
9
[19]: ### TEST FUNCTION: test_adaboost_dt
from sklearn.ensemble import AdaBoostClassifier
from sklearn.tree import DecisionTreeClassifier
#Adaboost
def adaBoostClassifier(X, y, **hyperparams):
DecisionTree_Classifier_params = {
↪['criterion', 'max_depth']
}
AdaBoost_Classifier_params = {
↪['n_estimators', 'learning_rate']
}
adb =␣
↪AdaBoostClassifier(DecisionTreeClassifier(**DecisionTree_Classifier_params,random_state=0),*
↪fit(X, y)
scores=cross_val_score(adb, X, y, cv=cvKFold,error_score='raise')
↪folds
# TODO
return adb,meanscore
adb,meanscore=adaBoostClassifier(X_scaled,␣
↪y,criterion='log_loss',max_depth=3,n_estimators=25,learning_rate=0.1)
[20]: ### SKIP

1.4 Hyperparameter tuning

Cross-validation is an excellent tool for determining the best generalisation performance and de-
termining the best hyperparameters. However, when we are looking at slower algorithms or large
data, we may choose to use a validation set instead of cross-validation. Here you will perform hy-
perparameter tuning using a test set consisting of 25% of the data, and a validation set consisting
of 10% of the training set. Set any random_state parameters to 0.
[21]: ### TEST FUNCTION: test_assert_splitting
# Create this function to use on any subset of this data
from sklearn.model_selection import train_test_split
10
# TODO: uncomment this code to create the initial train test split
# X_train_all, X_test, y_train_all, y_test = train_test_split(X, y,␣
↪random_state=0)
def train_val_test_split(X,y):
X_train_all, X_test, y_train_all, y_test = train_test_split(X, y,␣

test_size=0.25,stratify=y, random_state=0)
↪
X_train, X_val, y_train, y_val = train_test_split(X_train_all, y_train_all,␣

↪test_size=0.1, stratify=y_train_all,random_state=0)
# TODO
return X_train_all, X_train, X_val, X_test, y_train_all, y_train, y_val,␣
↪y_test
# TODO
# return X_train_all, X_train, X_val, X_test, y_train_all, y_train, y_val,␣
↪y_test
#return None
X_train_all, X_train, X_val, X_test, y_train_all, y_train, y_val,␣
↪y_test=train_val_test_split(X_scaled,y)
print(X_train_all, X_train, X_val, X_test, y_train_all, y_train, y_val, y_test)
[[0.88461538 0.85611511 0.23684211 … 0. 0. 1. ]

[0.38461538 0.73381295 0.15789474 … 0. 0. 1. ]
[0.19230769 0.83453237 0.26973684 … 0. 0. 1. ]
…
[1. 0.8057554 0.32236842 … 0. 1. 1. ]
[0.5 0.94244604 0.32236842 … 0. 0. 1. ]
[0.65384615 0.85611511 0.38157895 … 0. 0. 1. ]]
[[0.88461538 0.72661871 0.25657895 … 0. 0. 0. ]
[0.80769231 0.68345324 0.21052632 … 0. 0. 1. ]
[0.57692308 0.8057554 0.21052632 … 1. 0. 0. ]
…
[0.57692308 0.82733813 0.28947368 … 0. 0. 1. ]
[0.61538462 0.70503597 0.30921053 … 0. 0. 1. ]
[0.88461538 0.82733813 0.26973684 … 0. 0. 1. ]]
[[0.65384615 0.86330935 0.23026316 … 1. 0. 1. ]
[0.61538462 0.82014388 0.23026316 … 0. 0. 1. ]
[0.73076923 0.98561151 0.42763158 … 0. 0. 1. ]
…
[0.42307692 0.76978417 0.23684211 … 0. 0. 1. ]
[0.96153846 0.75539568 0.20394737 … 0. 0. 1. ]
[0.92307692 0.9352518 0.36184211 … 1. 0. 1. ]]
[[0.80769231 0.82733813 0.39473684 … 0. 0. 1. ]
[0.26923077 0.82733813 0.24342105 … 0. 0. 1. ]
[0.80769231 0.71942446 0.34210526 … 0. 0. 0. ]
…
[0.34615385 0.82014388 0.30921053 … 0. 0. 1. ]
[0.23076923 0.91366906 0.27631579 … 1. 0. 1. ]
11
[0.19230769 0.86330935 0.42763158 … 1. 0. 1. ]] [0 1 1
… 1 0 0] [0 0 0 … 0 0 0] [0 0 1 0 0 0 1 0 0 0 0 0 1 0 0 0 1 0 0 0 1 1 0 0 1
0 0 1 0 0 1 1 0 1 1 0 0
1 1 1 0 1 0 0 0 1 0 0 1 0 1 0 0 0 0 1 1 0 0 0 0 0 1 1 1 0 0 0 1 0 1 0 1 1
1 0 1 1 0 1 0 0 0 0 1 1 0 0 0 1 1 0 1 1 0 1 1 1 0 1 0 1 0 0 0 1 0 1 1 1 1
0 0 1 1 1 1 1 1 1 0 1 1 1 0 0 0 1 0 0 0 1 0 1 0 1 0 1 0 0 1 1 1 1 1 1 0 1
0 0 1 1 0 0 0 0 1 0 0 1 1 0 1 0 0 1 0 0 0 1 1 1 1 1 1 1 1 1 0 1 1 1 1 0 0
1 0 0 0 0 1 0 1 1 1 0 1 1 1 1 1 0 1 1 1 0 1 1 1 1 1 1 1 1 0 1 0 0 0 0 0 1
1 0 0 1 1 1 1 0 0 1 1 0 1 0 1 1 1 1 1 0 1 0 1 1 1 1 1 1 1 0 1 0 0 0 1 0 1
0 0 0 0 0 0 0 1 0 0 1 0 0 1 1 0 0 1 1 0 0 0 0 1 1 1 0 0 1 1 0 0 0 1 1 0 0
0 0 1 0 0 1 1 0 0 0 0 1 1 0 0 1 0 0 0 1 0 1 1 0 1 1 1 0 1 0 1 1 0 0 1 1 0
1 0 1 1 0 0 1 1 1 1 1 1 1 0 1 0 0 1 0 0 1 1 0 1 1 1 0 1 0 0 1 1 0 1 0 1 0
0 0 1 0 1 0 0 1 1 0 0 1 1 0 0 0 0 0 0 1 0 0 0 1 0 1 1 1 0 1 0 1 1 0 0 0 0
0 0 1 0 1 0 0 0 0 1 0 0 1 1 0 1 0 0 1 1 0 1 1 1 1 1 0 0 0 0 0 1 1 1 0 1 1
1 0 0 1 0 0 1 1 1 1 0 0 0 1 0 0 0 1 1 1 1 1 1 1 0 0 1 0 0 1 0 1 1 0 0 1 1
0 0 0 1 0 0 1 0 0 0 0 0 1 1 1 0 0 1 1 1 0 0 0 0 1 1 0 1 0 1 0 0 0 1 1] [1 0 1
… 0 0 1]
[22]: ### SKIP

Perform a grid search with Adaboost on linear SVM. You should select and handle appropriate
hyperparameters for both methods, and return the best set of hyperparameters found.
In the following cell, you should define a grid with three parameters for the AdaBoost and/or the
linear SVM.
Use the variable names provided in the scaffold. We will ensure your results are reproduced and
that you set reasonable parameter values to achieve a threshold score.
The hyperparameter tuning will also be run on a different subset of the data, so try and make sure
your hyperparameter ranges have good coverage.
Output format:
Best hyperparameter combination: {‘param1’: value, ‘param2’: value, …} Best model’s test set
score: x.xx
[27]: ### TEST FUNCTION: test_parameter_tuning
from sklearn import svm, datasets
from sklearn.svm import LinearSVC
from sklearn.ensemble import AdaBoostClassifier
from sklearn.model_selection import GridSearchCV
def adaBoostGrid(X, y,**param_grid):
svm = LinearSVC(random_state=0)
12
adb = AdaBoostClassifier(svm,random_state=0)
grid = GridSearchCV(adb,param_grid).fit(X,y)
best_model=grid.best_estimator_
best_params=grid.best_params_
best_val_score=grid.best_score_
return best_model, best_params, best_val_score

# TODO
# return model, best_params, best_val_score, score
# return None
param_grid = {
'base_estimator__penalty':['l2'],
'base_estimator__C':[0.001,0.01,0.1,1,2],
#'base_estimator__dual':['auto'],
#'penalty':['l2'],
#'C':[0.001,0.01,0.1,1,2],
#'dual':['auto'],
'n_estimators':[15,20,25,30,50],
'algorithm':['SAMME','SAMME.R'],
'learning_rate':[0.01,0.1,1,2,3]
}
best_model, best_params, best_val_score =␣
↪adaBoostGrid(X_train,y_train,**param_grid)
print(f"Best hyperparameter combination: {best_model, best_params}")

print("Average cross_validation score: {:.2f}".format(best_val_score))
#param_grid = None
#best_model, best_params, best_val_score = None, None, None
#test_score = None
# Print your result as above
/usr/lib/python3.11/site-packages/sklearn/model_selection/_validation.py:720:
FutureWarning: Parameter 'base_estimator' of AdaBoostClassifier is deprecated in
favor of 'estimator'. See AdaBoostClassifier's docstring for more details.
estimator = estimator.set_params(**cloned_parameters)
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
esti
[too much output …]
Best hyperparameter combination: (AdaBoostClassifier(algorithm='SAMME',
estimator=LinearSVC(C=2, dual='auto', random_state=0),
learning_rate=1, n_estimators=25, random_state=0),
{'algorithm': 'SAMME', 'base_estimator__C': 2, 'base_estimator__dual': 'auto',
'base_estimator__penalty': 'l2', 'learning_rate': 1, 'n_estimators': 25})
Average cross_validation score: 0.71
[17]: ### TEST FUNCTION: test_adaboost_grid

[0]: ### SKIP

42

COMP5318

Uploaded by

Copyright:

Available Formats

COMP5318

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

COMP5318

Uploaded by

Copyright:

Available Formats

notebookp1-9-10

September 14, 2023

[2]: # Import all libraries

1.1 Data loading and preprocessing

[[ 51. 162. 82. … 0. 0. 1.]

SMOKE ALCOHOL PHYSICAL_ACTIVITY CARDIO_DISEASE

[4]: ### SKIP

[6]: ### SKIP

1.2 Implementing classification algorithms and comparing with 10-fold strati-

StratifiedKFold(n_splits=10, random_state=0, shuffle=True)

[8]: ### SKIP

print(f"Cross validation score: {score:.2f}")

Cross validation score: 0.71

[10]: ### SKIP

Cross validation score: 0.71

[12]: ### SKIP

print("Cross validation score: {:.2f}".format(meanscore))

Cross validation score: 0.73

[14]: ### SKIP

#Support Vector Machine

Cross validation score: 0.72

[16]: ### SKIP

1.3 Ensemble Methods

scores = cross_val_score(model, X, y, cv=cvKFold)

return model, score

print("Cross validation score: {:.2f}".format(score))

Cross validation score: 0.73

[18]: ### SKIP

print("Cross validation score: {:.2f}".format(meanscore))

Cross validation score: 0.73

[20]: ### SKIP

1.4 Hyperparameter tuning

X_train_all, X_test, y_train_all, y_test = train_test_split(X, y,␣

X_train, X_val, y_train, y_val = train_test_split(X_train_all, y_train_all,␣

print(X_train_all, X_train, X_val, X_test, y_train_all, y_train, y_val, y_test)

[[0.88461538 0.85611511 0.23684211 … 0. 0. 1. ]

[22]: ### SKIP

def adaBoostGrid(X, y,**param_grid):

return best_model, best_params, best_val_score

print(f"Best hyperparameter combination: {best_model, best_params}")

[17]: ### TEST FUNCTION: test_adaboost_grid

[0]: ### SKIP

You might also like