Investigate Titanic Dataset

Table of Contents

Introduction

In this project we will analyse data associated titanic maiden voyage leading to its crash. We will look for trends among passengers who survived and how they differ from passengers who died.

In [1]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline

Data Wrangling

Tip: In this section of the report, you will load in the data, check for cleanliness, and then trim and clean your dataset for analysis. Make sure that you document your steps carefully and justify your cleaning decisions.

General Properties

In [3]:
df = pd.read_csv('titanic_data.csv')
df.head()
Out[3]:
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked
0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S
1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 0 PC 17599 71.2833 C85 C
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S
3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S
4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S
In [4]:
df.shape
Out[4]:
(891, 12)
In [5]:
df.describe()
Out[5]:
PassengerId Survived Pclass Age SibSp Parch Fare
count 891.000000 891.000000 891.000000 714.000000 891.000000 891.000000 891.000000
mean 446.000000 0.383838 2.308642 29.699118 0.523008 0.381594 32.204208
std 257.353842 0.486592 0.836071 14.526497 1.102743 0.806057 49.693429
min 1.000000 0.000000 1.000000 0.420000 0.000000 0.000000 0.000000
25% 223.500000 0.000000 2.000000 20.125000 0.000000 0.000000 7.910400
50% 446.000000 0.000000 3.000000 28.000000 0.000000 0.000000 14.454200
75% 668.500000 1.000000 3.000000 38.000000 1.000000 0.000000 31.000000
max 891.000000 1.000000 3.000000 80.000000 8.000000 6.000000 512.329200

From the above summary statistics we can see that:

  • 38% of passengers survived
  • Over 50% of passengers are in 3rd class
  • Majority of passengers are between the age 20 and 40
  • Most of passengers came without siblings or Spouce or Children
In [6]:
df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 891 entries, 0 to 890
Data columns (total 12 columns):
PassengerId    891 non-null int64
Survived       891 non-null int64
Pclass         891 non-null int64
Name           891 non-null object
Sex            891 non-null object
Age            714 non-null float64
SibSp          891 non-null int64
Parch          891 non-null int64
Ticket         891 non-null object
Fare           891 non-null float64
Cabin          204 non-null object
Embarked       889 non-null object
dtypes: float64(2), int64(5), object(5)
memory usage: 83.6+ KB

We can drop coloumns which are not useful for our analysis

In [7]:
df.drop(['PassengerId', 'Name', 'Ticket', 'Cabin'], axis=1, inplace=True)
In [8]:
df.head()
Out[8]:
Survived Pclass Sex Age SibSp Parch Fare Embarked
0 0 3 male 22.0 1 0 7.2500 S
1 1 1 female 38.0 1 0 71.2833 C
2 1 3 female 26.0 0 0 7.9250 S
3 1 1 female 35.0 1 0 53.1000 S
4 0 3 male 35.0 0 0 8.0500 S
In [16]:
df.hist(figsize=(10,8));
In [15]:
df[df.Age.isnull()].hist(figsize=(10,8));

Comparing above histograms, we can conclude that both datasets are identical except the age so we can replace missing age with average age.

In [18]:
df.fillna(df.mean(), inplace = True)
In [ ]:
df[df.Embarked.isnull()]
In [19]:
df.dropna(inplace = True)
df.info()
<class 'pandas.core.frame.DataFrame'>
Int64Index: 889 entries, 0 to 890
Data columns (total 8 columns):
Survived    889 non-null int64
Pclass      889 non-null int64
Sex         889 non-null object
Age         889 non-null float64
SibSp       889 non-null int64
Parch       889 non-null int64
Fare        889 non-null float64
Embarked    889 non-null object
dtypes: float64(2), int64(4), object(2)
memory usage: 62.5+ KB

Exploratory Data Analysis

Tip: Now that you've trimmed and cleaned your data, you're ready to move on to exploration. Compute statistics and create visualizations with the goal of addressing the research questions that you posed in the Introduction section. It is recommended that you be systematic with your approach. Look at one variable at a time, and then follow it up by looking at relationships between variables.

# Creating masks

In [21]:
survived = df.Survived == True
died = df.Survived == False
In [25]:
survived.head()
Out[25]:
0    False
1     True
2     True
3     True
4    False
Name: Survived, dtype: bool
In [27]:
died.head()
Out[27]:
0     True
1    False
2    False
3    False
4     True
Name: Survived, dtype: bool
In [28]:
df.Fare[survived].mean()
Out[28]:
48.209498235294106
In [29]:
df.Fare[died].mean()
Out[29]:
22.117886885245877
In [37]:
df.Fare[survived].hist(alpha=0.5, label='survived')
df.Fare[died].hist(alpha=0.5, label='died')
plt.legend();

conclusion: We can see from above histogram that people who survived paid more, especially lower class passengers died than survived.

Changing bin size makes the seperation clear.

In [38]:
df.Fare[survived].hist(alpha=0.5, bins=20, label='survived')
df.Fare[died].hist(alpha=0.5, bins=20, label='died')
plt.legend();

How Fare is correlated to Class

In [40]:
df.groupby('Pclass').Survived.mean()
Out[40]:
Pclass
1    0.626168
2    0.472826
3    0.242363
Name: Survived, dtype: float64
In [43]:
df.groupby('Pclass').Survived.mean().plot(kind='bar');

Distribution of ages among passengers who survived and didn't survive

In [44]:
df.Age[survived].hist(alpha=0.5, bins=20, label='survived')
df.Age[died].hist(alpha=0.5, bins=20, label='died')
plt.legend();

conclusion: Looks like younger people survived more compared to older people.

Correlation between gender and survival

In [46]:
df.groupby('Sex').Survived.mean()
Out[46]:
Sex
female    0.740385
male      0.188908
Name: Survived, dtype: float64
In [45]:
df.groupby('Sex').Survived.mean().plot(kind='bar');

conclusion: More female survived than male.

In [47]:
df.Sex.value_counts()
Out[47]:
male      577
female    312
Name: Sex, dtype: int64

conclusion: There are more males than females

In [50]:
df.groupby('Sex')['Pclass'].value_counts()
Out[50]:
Sex     Pclass
female  3         144
        1          92
        2          76
male    3         347
        1         122
        2         108
Name: Pclass, dtype: int64
In [51]:
df.query('Sex == "female"')['Fare'].median(), df.query('Sex == "male"')['Fare'].median()
Out[51]:
(23.0, 10.5)

conclusion: Females spent more money on ticket.

In [54]:
df.groupby(['Pclass','Sex']).Survived.mean().plot(kind='bar');
In [59]:
df.SibSp[survived].value_counts().plot(kind='bar', alpha=0.5, color='blue', label='survived')
df.SibSp[died].value_counts().plot(kind='bar', alpha=0.5, color='orange', label='died')
plt.legend();

conclusion: people having lot of family doesn't appear to be surviving

In [60]:
df.Parch[survived].value_counts().plot(kind='bar', alpha=0.5, color='blue', label='survived')
df.Parch[died].value_counts().plot(kind='bar', alpha=0.5, color='orange', label='died')
plt.legend();

Conclusions

Finally, summarize your findings and the results that have been performed. Make sure to be clear with regards to the limitations of your exploration. If you haven't done any statistical tests, do not imply any statistical conclusions.

General trends are mentioned in each exploration cases.