3/15/2020 EDA Assignment

Haberman's Survival Data Set

Survival of patients who had undergone surgery for breast cancer.

About the Data Set: The dataset contains cases from a study that was conducted between 1958 and 1970
at the University of Chicago's Billings Hospital on the survival of patients who had undergone surgery for
breast cancer.

Database Information:
Source of Data Set: https://www.kaggle.com/gilsousa/habermans-survival-data-set
(https://www.kaggle.com/gilsousa/habermans-survival-data-set)

Objective: To predict whether the patient will survive after 5 years or not based upon the patient's age, year
of treatment and the number of positive lymph nodes

In [1]:

import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np

haberman = pd.read_csv('haberman.csv', header = 0, names = ['age', 'year_of_operation',

'positive_axillary_nodes', 'survival_status'])

In [2]:

# 1 = positive, 2 = negative
haberman["survival_status"]=haberman["survival_status"].map({1:'positive',2:'negative'
})
haberman.head()

Out[2]:

age year_of_operation positive_axillary_nodes survival_status

0 30 64 1 positive

1 30 62 3 positive

2 30 65 0 positive

3 31 59 2 positive

4 31 65 4 positive

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 1/15

3/15/2020 EDA Assignment

In [3]:

haberman.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 306 entries, 0 to 305
Data columns (total 4 columns):
age 306 non-null int64
year_of_operation 306 non-null int64
positive_axillary_nodes 306 non-null int64
survival_status 306 non-null object
dtypes: int64(3), object(1)
memory usage: 9.6+ KB

Observations:

306 entries ranging from 0 to 305

There are 4 columns, 3 describing the data and 1 for the output
Each column has 306 non-null values
The output, survival_status column can take only 2 values 1 or 2. Which I changed into positive and
negative.

Conclusions:

There are no missing values in this dataset.

This survival_status column is a binary-classification problem.

In [4]:

print (haberman.shape)

(306, 4)

Number of Rows: 306

Number of Columns: 4

Attribute Information:

Age of patient at time of operation (numerical)

Patient's year of operation (year - 1900, numerical)
Number of positive axillary nodes detected (numerical)
Survival status (class attribute) 1 = the patient survived 5 years or longer 2 = the patient died within
5 year
Missing Attribute Values: None

Positive axillary nodes: https://en.wikipedia.org/wiki/Positive_axillary_lymph_node

(https://en.wikipedia.org/wiki/Positive_axillary_lymph_node)

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 2/15

3/15/2020 EDA Assignment

In [5]:

haberman["survival_status"].value_counts()

Out[5]:

positive 225
negative 81
Name: survival_status, dtype: int64

Observation: In the above data which is an imbalance dataset, the number of patient who will survive after 5
years is 225 and 81 patients will died within 5 year.

In [6]:

print (haberman.describe())

age year_of_operation positive_axillary_nodes

count 306.000000 306.000000 306.000000
mean 52.457516 62.852941 4.026144
std 10.803452 3.249405 7.189654
min 30.000000 58.000000 0.000000
25% 44.000000 60.000000 0.000000
50% 52.000000 63.000000 1.000000
75% 60.750000 65.750000 4.000000
max 83.000000 69.000000 52.000000

Observation:

The Maximum age of the patients is 83 years & Minimum age is 30 years, with a mean value of
52.46 years.
The median age of the patients is 52 years.
All the operation of the given data done between 1958 - 1969.
The Maximum positive axillary nodes found in a patients is 52 & Minimum is 0 years, with an
average of 4.03.

In [7]:

haberman_positive = haberman.loc[haberman.survival_status == 'positive']

haberman_negative = haberman.loc[haberman.survival_status == 'negative']

In [8]:

print (haberman_positive.describe())

age year_of_operation positive_axillary_nodes

count 225.000000 225.000000 225.000000
mean 52.017778 62.862222 2.791111
std 11.012154 3.222915 5.870318
min 30.000000 58.000000 0.000000
25% 43.000000 60.000000 0.000000
50% 52.000000 63.000000 0.000000
75% 60.000000 66.000000 3.000000
max 77.000000 69.000000 46.000000

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 3/15

3/15/2020 EDA Assignment

Observation: Mean value of positive axillary nodes is 2.79

Conclusion: Hence we can assume that less amount of nodes may be signify less amount of health risk.

In [9]:

print (haberman_negative.describe())

age year_of_operation positive_axillary_nodes

count 81.000000 81.000000 81.000000
mean 53.679012 62.827160 7.456790
std 10.167137 3.342118 9.185654
min 34.000000 58.000000 0.000000
25% 46.000000 59.000000 1.000000
50% 53.000000 63.000000 4.000000
75% 61.000000 65.000000 11.000000
max 83.000000 69.000000 52.000000

Observation: Mean value of positive axillary nodes is 7.46

Conclusion: Hence we can assume that higher amount of nodes may be signify higher amount of health
risk.

2-D Scatter plot

In [10]:

sns.set_style("whitegrid");
sns.FacetGrid(haberman, hue="survival_status", size=5) \
.map(plt.scatter, "positive_axillary_nodes", "age") \
.add_legend();
plt.show();

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 4/15

3/15/2020 EDA Assignment

Observation: More amount of positive survival status found where the positive axillary nodes are minimum.

Conclusion: The chance of recovery is greater if the less amount of positive axillary nodes were found.

In [11]:

sns.set_style("whitegrid");
sns.FacetGrid(haberman, hue="survival_status", size=5) \
.map(plt.scatter, "year_of_operation", "age") \
.add_legend();
plt.show();

Conclusion: Most of the operations done in between 40 to 70 years of age.

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 5/15

3/15/2020 EDA Assignment

In [12]:

sns.set_style("whitegrid");
sns.FacetGrid(haberman, hue="survival_status", size=5) \
.map(plt.scatter, "positive_axillary_nodes", "year_of_operation") \
.add_legend();
plt.show();

Conclusion: No significant conclusion can be observerd as the data overlapped.

Pairplots

Here we get 3c2 = 3 pairplots because we have 3 features in which we can select only 2.
In the pairplots we did not take the principle diagonal graphs.

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 6/15

3/15/2020 EDA Assignment

In [13]:

sns.set_style('whitegrid')
sns.pairplot(haberman, hue='survival_status', vars=['age', 'year_of_operation', 'positi
ve_axillary_nodes'], size=4)
plt.show()

Conclusion:

We were unable to get any conclusion from the plot between year of operation vs age and year of
operation vs positive axillary nodes because the data mostly overlapped.
Where as it can be said that from the plot between positive axillary nodes & age it can be assumesd
that the chance of recovery is greater if the less amount of positive axillary nodes were found.

1D scatter

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 7/15

3/15/2020 EDA Assignment

In [14]:

# Patient Age
sns.FacetGrid(haberman,hue="survival_status",size=5)\
.map(sns.distplot,"age")\
.add_legend();
plt.show();

C:\Users\Shiladitya\Anaconda3\lib\site-packages\matplotlib\axes\_axes.py:6
462: UserWarning: The 'normed' kwarg is deprecated, and has been replaced
by the 'density' kwarg.
warnings.warn("The 'normed' kwarg is deprecated, and has been "
C:\Users\Shiladitya\Anaconda3\lib\site-packages\matplotlib\axes\_axes.py:6
462: UserWarning: The 'normed' kwarg is deprecated, and has been replaced
by the 'density' kwarg.
warnings.warn("The 'normed' kwarg is deprecated, and has been "

Observation: Patients with age range 30-40 have survived the most.

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 8/15

3/15/2020 EDA Assignment

In [15]:

# Year of Operation
sns.FacetGrid(haberman,hue="survival_status",size=5)\
.map(sns.distplot,"year_of_operation")\
.add_legend();
plt.show()

C:\Users\Shiladitya\Anaconda3\lib\site-packages\matplotlib\axes\_axes.py:6
462: UserWarning: The 'normed' kwarg is deprecated, and has been replaced
by the 'density' kwarg.
warnings.warn("The 'normed' kwarg is deprecated, and has been "
C:\Users\Shiladitya\Anaconda3\lib\site-packages\matplotlib\axes\_axes.py:6
462: UserWarning: The 'normed' kwarg is deprecated, and has been replaced
by the 'density' kwarg.
warnings.warn("The 'normed' kwarg is deprecated, and has been "

Observation:

Operation year having range (63-66) had highest un-successfull rate.

Operation year (60-62) had highest successfull rate.

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 9/15

3/15/2020 EDA Assignment

In [16]:

# Positive axillary nodes

sns.FacetGrid(haberman,hue="survival_status",size=5)\
.map(sns.distplot,"positive_axillary_nodes")\
.add_legend();
plt.show()

C:\Users\Shiladitya\Anaconda3\lib\site-packages\matplotlib\axes\_axes.py:6
462: UserWarning: The 'normed' kwarg is deprecated, and has been replaced
by the 'density' kwarg.
warnings.warn("The 'normed' kwarg is deprecated, and has been "
C:\Users\Shiladitya\Anaconda3\lib\site-packages\matplotlib\axes\_axes.py:6
462: UserWarning: The 'normed' kwarg is deprecated, and has been replaced
by the 'density' kwarg.
warnings.warn("The 'normed' kwarg is deprecated, and has been "

Observation: positive axillary nodes=0 has the highest Survival rate.

Conclusion: Positive axillary nodes is most significant amonog all as its value is lessly overlapped.

PDF & CDF

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 10/15

3/15/2020 EDA Assignment

In [17]:

counts, bin_edges = np.histogram(haberman_positive['positive_axillary_nodes'], bins=20,

density = True)
pdf = counts/(sum(counts))
print(pdf);
print(bin_edges)

cdf = np.cumsum(pdf)
plt.plot(bin_edges[1:], pdf)
plt.plot(bin_edges[1:], cdf)
plt.xlabel('positive_axillary_nodes')
plt.show()

[0.73333333 0.10222222 0.02666667 0.05333333 0.01333333 0.00888889

0.02222222 0.00444444 0.00888889 0.00888889 0.00444444 0.
0.00444444 0.00444444 0. 0. 0. 0.
0. 0.00444444]
[ 0. 2.3 4.6 6.9 9.2 11.5 13.8 16.1 18.4 20.7 23. 25.3 27.6 29.9
32.2 34.5 36.8 39.1 41.4 43.7 46. ]

Observation: About 85% of the patients have a positive axillary nodes <= 10 who survive more than 5 years
after operation.

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 11/15

3/15/2020 EDA Assignment

In [18]:

counts, bin_edges = np.histogram(haberman_negative['positive_axillary_nodes'], bins=20,

density = True)
pdf = counts/(sum(counts))
print(pdf);
print(bin_edges)

cdf = np.cumsum(pdf)
plt.plot(bin_edges[1:], pdf)
plt.plot(bin_edges[1:], cdf)
plt.xlabel('positive_axillary_nodes')
plt.show()

[0.39506173 0.17283951 0.0617284 0.08641975 0.04938272 0.08641975

0.01234568 0.03703704 0.0617284 0.01234568 0. 0.
0. 0.01234568 0. 0. 0. 0.
0. 0.01234568]
[ 0. 2.6 5.2 7.8 10.4 13. 15.6 18.2 20.8 23.4 26. 28.6 31.2 33.8
36.4 39. 41.6 44.2 46.8 49.4 52. ]

Observation: About 85% of the patients have a positive axillary nodes <= 20 who survive less than 5 years
after operation.

Conclusion: No of positive auxillary nodes is directly related with the chance of survive.

Box Plot & Whiskers

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 12/15

3/15/2020 EDA Assignment

In [19]:

sns.boxplot(x='survival_status',y='positive_axillary_nodes', data=haberman)
plt.show()
sns.boxplot(x='survival_status',y='year_of_operation', data=haberman)
plt.show()
sns.boxplot(x='survival_status',y='age', data=haberman)
plt.show()

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 13/15

3/15/2020 EDA Assignment

In [20]:

sns.violinplot(x='survival_status',y='positive_axillary_nodes', data=haberman)
plt.show()
sns.violinplot(x='survival_status',y='year_of_operation', data=haberman)
plt.show()
sns.violinplot(x='survival_status',y='age', data=haberman)
plt.show()

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 14/15

3/15/2020 EDA Assignment

Observation: Out of the 3 features, positive_auxiliary_nodes has the most significant distinct-distribution
among the two-classes. From the above observation we can only conclude that higher the
positive_axillary_nodes, higher the chances of their death. The age of the patient does not seem to have any
relation with survial status.

Final Conclusion:

From the dataset we can say that the majority of operations are performed on people age between
40 to70. Observing scatter plot between year_of_operation vs age.
We can say that a large number of operation were done in between 1960 and 1965 (From box plot
between year_of_operation vs survival_status)
We get a conclusion that patients with 0 positive axillary nodes are more likely to survive
irrespective to there age. (positive_axillary_nodes vs age)
Patients with age range 30-40 have survived the most.
From the box plot we can say that, the more number of positive axillary nodes, the more chances
that the patients would die.

file:///C:/Users/Shiladitya/Desktop/EDA Assignment.html 15/15