Clustering using the Mean Shift algorithm

Using MeanShift to find clusters in the Titanic data set and examine each cluster

Clustering is a popular and elegant unsupervised learning technique which helps find patterns in the underlying data. Clustering does not use any Y variables or labels on the data. It looks at the data structure itself.

In [1]:
import pandas as pd

Titanic data set

Download link: https://www.kaggle.com/c/3136/download/train.csv

Summary: Information about passengers who were on the Titanic including whether they survived

In [2]:
titanic_data = pd.read_csv('../data/titanic.csv', quotechar='"')
titanic_data.head()
Out[2]:
PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked
0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S
1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1 0 PC 17599 71.2833 C85 C
2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S
3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S
4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S

Drop columns which are meaningless when attempting to find patterns

In [3]:
titanic_data.drop(['PassengerId', 'Name', 'Ticket', 'Cabin'], 'columns', inplace=True)
titanic_data.head()
Out[3]:
Survived Pclass Sex Age SibSp Parch Fare Embarked
0 0 3 male 22.0 1 0 7.2500 S
1 1 1 female 38.0 1 0 71.2833 C
2 1 3 female 26.0 0 0 7.9250 S
3 1 1 female 35.0 1 0 53.1000 S
4 0 3 male 35.0 0 0 8.0500 S

Convert the gender values to numbers

In [4]:
from sklearn import preprocessing

le = preprocessing.LabelEncoder()
titanic_data['Sex'] = le.fit_transform(titanic_data['Sex'].astype(str))
titanic_data.head()
Out[4]:
Survived Pclass Sex Age SibSp Parch Fare Embarked
0 0 3 1 22.0 1 0 7.2500 S
1 1 1 0 38.0 1 0 71.2833 C
2 1 3 0 26.0 0 0 7.9250 S
3 1 1 0 35.0 1 0 53.1000 S
4 0 3 1 35.0 0 0 8.0500 S

Apply one-hot encoding for the port of Embarkation

In [5]:
titanic_data = pd.get_dummies(titanic_data, columns=['Embarked'])
titanic_data.head()
Out[5]:
Survived Pclass Sex Age SibSp Parch Fare Embarked_C Embarked_Q Embarked_S
0 0 3 1 22.0 1 0 7.2500 0 0 1
1 1 1 0 38.0 1 0 71.2833 1 0 0
2 1 3 0 26.0 0 0 7.9250 0 0 1
3 1 1 0 35.0 1 0 53.1000 0 0 1
4 0 3 1 35.0 0 0 8.0500 0 0 1

Check how many rows in the data frame contain null values

In [6]:
titanic_data[titanic_data.isnull().any(axis=1)]
Out[6]:
Survived Pclass Sex Age SibSp Parch Fare Embarked_C Embarked_Q Embarked_S
5 0 3 1 NaN 0 0 8.4583 0 1 0
17 1 2 1 NaN 0 0 13.0000 0 0 1
19 1 3 0 NaN 0 0 7.2250 1 0 0
26 0 3 1 NaN 0 0 7.2250 1 0 0
28 1 3 0 NaN 0 0 7.8792 0 1 0
29 0 3 1 NaN 0 0 7.8958 0 0 1
31 1 1 0 NaN 1 0 146.5208 1 0 0
32 1 3 0 NaN 0 0 7.7500 0 1 0
36 1 3 1 NaN 0 0 7.2292 1 0 0
42 0 3 1 NaN 0 0 7.8958 1 0 0
45 0 3 1 NaN 0 0 8.0500 0 0 1
46 0 3 1 NaN 1 0 15.5000 0 1 0
47 1 3 0 NaN 0 0 7.7500 0 1 0
48 0 3 1 NaN 2 0 21.6792 1 0 0
55 1 1 1 NaN 0 0 35.5000 0 0 1
64 0 1 1 NaN 0 0 27.7208 1 0 0
65 1 3 1 NaN 1 1 15.2458 1 0 0
76 0 3 1 NaN 0 0 7.8958 0 0 1
77 0 3 1 NaN 0 0 8.0500 0 0 1
82 1 3 0 NaN 0 0 7.7875 0 1 0
87 0 3 1 NaN 0 0 8.0500 0 0 1
95 0 3 1 NaN 0 0 8.0500 0 0 1
101 0 3 1 NaN 0 0 7.8958 0 0 1
107 1 3 1 NaN 0 0 7.7750 0 0 1
109 1 3 0 NaN 1 0 24.1500 0 1 0
121 0 3 1 NaN 0 0 8.0500 0 0 1
126 0 3 1 NaN 0 0 7.7500 0 1 0
128 1 3 0 NaN 1 1 22.3583 1 0 0
140 0 3 0 NaN 0 2 15.2458 1 0 0
154 0 3 1 NaN 0 0 7.3125 0 0 1
... ... ... ... ... ... ... ... ... ... ...
718 0 3 1 NaN 0 0 15.5000 0 1 0
727 1 3 0 NaN 0 0 7.7375 0 1 0
732 0 2 1 NaN 0 0 0.0000 0 0 1
738 0 3 1 NaN 0 0 7.8958 0 0 1
739 0 3 1 NaN 0 0 7.8958 0 0 1
740 1 1 1 NaN 0 0 30.0000 0 0 1
760 0 3 1 NaN 0 0 14.5000 0 0 1
766 0 1 1 NaN 0 0 39.6000 1 0 0
768 0 3 1 NaN 1 0 24.1500 0 1 0
773 0 3 1 NaN 0 0 7.2250 1 0 0
776 0 3 1 NaN 0 0 7.7500 0 1 0
778 0 3 1 NaN 0 0 7.7375 0 1 0
783 0 3 1 NaN 1 2 23.4500 0 0 1
790 0 3 1 NaN 0 0 7.7500 0 1 0
792 0 3 0 NaN 8 2 69.5500 0 0 1
793 0 1 1 NaN 0 0 30.6958 1 0 0
815 0 1 1 NaN 0 0 0.0000 0 0 1
825 0 3 1 NaN 0 0 6.9500 0 1 0
826 0 3 1 NaN 0 0 56.4958 0 0 1
828 1 3 1 NaN 0 0 7.7500 0 1 0
832 0 3 1 NaN 0 0 7.2292 1 0 0
837 0 3 1 NaN 0 0 8.0500 0 0 1
839 1 1 1 NaN 0 0 29.7000 1 0 0
846 0 3 1 NaN 8 2 69.5500 0 0 1
849 1 1 0 NaN 1 0 89.1042 1 0 0
859 0 3 1 NaN 0 0 7.2292 1 0 0
863 0 3 0 NaN 8 2 69.5500 0 0 1
868 0 3 1 NaN 0 0 9.5000 0 0 1
878 0 3 1 NaN 0 0 7.8958 0 0 1
888 0 3 0 NaN 1 2 23.4500 0 0 1

177 rows × 10 columns

Rather than clean up all the missing NaN values, we drop them all

In [7]:
titanic_data = titanic_data.dropna()

Use Mean Shift in order to find clusters in the data

  • The bandwidth parameter specifies the "radius" of each cluster
  • Higher bandwidths will produce fewer clusters
In [8]:
from sklearn.cluster import MeanShift

analyzer = MeanShift(bandwidth=30)
analyzer.fit(titanic_data)
Out[8]:
MeanShift(bandwidth=30, bin_seeding=False, cluster_all=True, min_bin_freq=1,
     n_jobs=None, seeds=None)

Default bandwidth for a data set is calculated using the estimate_bandwidth function

In [9]:
# Getting the size of the bandwidth which MeanShift will have used by default
from sklearn.cluster import estimate_bandwidth
estimate_bandwidth(titanic_data)
Out[9]:
30.44675914497196

Fetch the labels generated by MeanShift for the Titanic data

In [10]:
labels = analyzer.labels_

How many clusters do we have?

In [11]:
import numpy as np

np.unique(labels)
Out[11]:
array([0, 1, 2, 3, 4], dtype=int64)

Create a new cluster_group column in the data frame for these labels

In [12]:
import numpy as np

titanic_data['cluster_group'] = np.nan
data_length = len(titanic_data)
for i in range(data_length):
    titanic_data.iloc[i, titanic_data.columns.get_loc('cluster_group')] = labels[i]
In [13]:
titanic_data.head()
Out[13]:
Survived Pclass Sex Age SibSp Parch Fare Embarked_C Embarked_Q Embarked_S cluster_group
0 0 3 1 22.0 1 0 7.2500 0 0 1 0.0
1 1 1 0 38.0 1 0 71.2833 1 0 0 1.0
2 1 3 0 26.0 0 0 7.9250 0 0 1 0.0
3 1 1 0 35.0 1 0 53.1000 0 0 1 1.0
4 0 3 1 35.0 0 0 8.0500 0 0 1 0.0

Examine the overall data in the data set

In [14]:
titanic_data.describe()
Out[14]:
Survived Pclass Sex Age SibSp Parch Fare Embarked_C Embarked_Q Embarked_S cluster_group
count 714.000000 714.000000 714.000000 714.000000 714.000000 714.000000 714.000000 714.000000 714.000000 714.000000 714.000000
mean 0.406162 2.236695 0.634454 29.699118 0.512605 0.431373 34.694514 0.182073 0.039216 0.775910 0.315126
std 0.491460 0.838250 0.481921 14.526497 0.929783 0.853289 52.918930 0.386175 0.194244 0.417274 0.690647
min 0.000000 1.000000 0.000000 0.420000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
25% 0.000000 1.000000 0.000000 20.125000 0.000000 0.000000 8.050000 0.000000 0.000000 1.000000 0.000000
50% 0.000000 2.000000 1.000000 28.000000 0.000000 0.000000 15.741700 0.000000 0.000000 1.000000 0.000000
75% 1.000000 3.000000 1.000000 38.000000 1.000000 1.000000 33.375000 0.000000 0.000000 1.000000 0.000000
max 1.000000 3.000000 1.000000 80.000000 5.000000 6.000000 512.329200 1.000000 1.000000 1.000000 4.000000

Examine average data for each cluster

In [15]:
titanic_cluster_data = titanic_data.groupby(['cluster_group']).mean()
titanic_cluster_data
Out[15]:
Survived Pclass Sex Age SibSp Parch Fare Embarked_C Embarked_Q Embarked_S
cluster_group
0.0 0.336918 2.525090 0.679211 28.256720 0.439068 0.370968 15.434139 0.121864 0.046595 0.831541
1.0 0.611111 1.296296 0.527778 36.148148 0.814815 0.500000 65.622688 0.333333 0.018519 0.629630
2.0 0.733333 1.000000 0.366667 32.430667 0.600000 0.866667 131.183883 0.500000 0.000000 0.500000
3.0 0.733333 1.000000 0.266667 30.333333 1.000000 1.333333 239.991940 0.533333 0.000000 0.466667
4.0 1.000000 1.000000 0.666667 35.333333 0.000000 0.333333 512.329200 1.000000 0.000000 0.000000

Add the counts for each cluster

In [16]:
titanic_cluster_data['Counts'] = pd.Series(titanic_data.groupby(['cluster_group']).size())
titanic_cluster_data
Out[16]:
Survived Pclass Sex Age SibSp Parch Fare Embarked_C Embarked_Q Embarked_S Counts
cluster_group
0.0 0.336918 2.525090 0.679211 28.256720 0.439068 0.370968 15.434139 0.121864 0.046595 0.831541 558
1.0 0.611111 1.296296 0.527778 36.148148 0.814815 0.500000 65.622688 0.333333 0.018519 0.629630 108
2.0 0.733333 1.000000 0.366667 32.430667 0.600000 0.866667 131.183883 0.500000 0.000000 0.500000 30
3.0 0.733333 1.000000 0.266667 30.333333 1.000000 1.333333 239.991940 0.533333 0.000000 0.466667 15
4.0 1.000000 1.000000 0.666667 35.333333 0.000000 0.333333 512.329200 1.000000 0.000000 0.000000 3

Examine the data in one of the clusters

In [17]:
titanic_data[ titanic_data['cluster_group'] == 1 ].describe()
Out[17]:
Survived Pclass Sex Age SibSp Parch Fare Embarked_C Embarked_Q Embarked_S cluster_group
count 108.000000 108.000000 108.000000 108.000000 108.000000 108.000000 108.000000 108.000000 108.000000 108.000000 108.0
mean 0.611111 1.296296 0.527778 36.148148 0.814815 0.500000 65.622688 0.333333 0.018519 0.629630 1.0
std 0.489771 0.645028 0.501555 14.919607 1.086434 0.971558 15.634315 0.473602 0.135445 0.485155 0.0
min 0.000000 1.000000 0.000000 1.000000 0.000000 0.000000 34.654200 0.000000 0.000000 0.000000 1.0
25% 0.000000 1.000000 0.000000 24.000000 0.000000 0.000000 52.554200 0.000000 0.000000 0.000000 1.0
50% 1.000000 1.000000 1.000000 35.000000 1.000000 0.000000 65.000000 0.000000 0.000000 1.000000 1.0
75% 1.000000 1.000000 1.000000 48.000000 1.000000 1.000000 78.937500 1.000000 0.000000 1.000000 1.0
max 1.000000 3.000000 1.000000 71.000000 5.000000 6.000000 93.500000 1.000000 1.000000 1.000000 1.0

View all the rows of one cluster

In [18]:
titanic_data[ titanic_data['cluster_group'] == 0]
Out[18]:
Survived Pclass Sex Age SibSp Parch Fare Embarked_C Embarked_Q Embarked_S cluster_group
0 0 3 1 22.0 1 0 7.2500 0 0 1 0.0
2 1 3 0 26.0 0 0 7.9250 0 0 1 0.0
4 0 3 1 35.0 0 0 8.0500 0 0 1 0.0
7 0 3 1 2.0 3 1 21.0750 0 0 1 0.0
8 1 3 0 27.0 0 2 11.1333 0 0 1 0.0
9 1 2 0 14.0 1 0 30.0708 1 0 0 0.0
10 1 3 0 4.0 1 1 16.7000 0 0 1 0.0
11 1 1 0 58.0 0 0 26.5500 0 0 1 0.0
12 0 3 1 20.0 0 0 8.0500 0 0 1 0.0
13 0 3 1 39.0 1 5 31.2750 0 0 1 0.0
14 0 3 0 14.0 0 0 7.8542 0 0 1 0.0
15 1 2 0 55.0 0 0 16.0000 0 0 1 0.0
16 0 3 1 2.0 4 1 29.1250 0 1 0 0.0
18 0 3 0 31.0 1 0 18.0000 0 0 1 0.0
20 0 2 1 35.0 0 0 26.0000 0 0 1 0.0
21 1 2 1 34.0 0 0 13.0000 0 0 1 0.0
22 1 3 0 15.0 0 0 8.0292 0 1 0 0.0
23 1 1 1 28.0 0 0 35.5000 0 0 1 0.0
24 0 3 0 8.0 3 1 21.0750 0 0 1 0.0
25 1 3 0 38.0 1 5 31.3875 0 0 1 0.0
30 0 1 1 40.0 0 0 27.7208 1 0 0 0.0
33 0 2 1 66.0 0 0 10.5000 0 0 1 0.0
37 0 3 1 21.0 0 0 8.0500 0 0 1 0.0
38 0 3 0 18.0 2 0 18.0000 0 0 1 0.0
39 1 3 0 14.0 1 0 11.2417 1 0 0 0.0
40 0 3 0 40.0 1 0 9.4750 0 0 1 0.0
41 0 2 0 27.0 1 0 21.0000 0 0 1 0.0
43 1 2 0 3.0 1 2 41.5792 1 0 0 0.0
44 1 3 0 19.0 0 0 7.8792 0 1 0 0.0
49 0 3 0 18.0 1 0 17.8000 0 0 1 0.0
... ... ... ... ... ... ... ... ... ... ... ...
852 0 3 0 9.0 1 1 15.2458 1 0 0 0.0
853 1 1 0 16.0 0 1 39.4000 0 0 1 0.0
854 0 2 0 44.0 1 0 26.0000 0 0 1 0.0
855 1 3 0 18.0 0 1 9.3500 0 0 1 0.0
857 1 1 1 51.0 0 0 26.5500 0 0 1 0.0
858 1 3 0 24.0 0 3 19.2583 1 0 0 0.0
860 0 3 1 41.0 2 0 14.1083 0 0 1 0.0
861 0 2 1 21.0 1 0 11.5000 0 0 1 0.0
862 1 1 0 48.0 0 0 25.9292 0 0 1 0.0
864 0 2 1 24.0 0 0 13.0000 0 0 1 0.0
865 1 2 0 42.0 0 0 13.0000 0 0 1 0.0
866 1 2 0 27.0 1 0 13.8583 1 0 0 0.0
869 1 3 1 4.0 1 1 11.1333 0 0 1 0.0
870 0 3 1 26.0 0 0 7.8958 0 0 1 0.0
872 0 1 1 33.0 0 0 5.0000 0 0 1 0.0
873 0 3 1 47.0 0 0 9.0000 0 0 1 0.0
874 1 2 0 28.0 1 0 24.0000 1 0 0 0.0
875 1 3 0 15.0 0 0 7.2250 1 0 0 0.0
876 0 3 1 20.0 0 0 9.8458 0 0 1 0.0
877 0 3 1 19.0 0 0 7.8958 0 0 1 0.0
880 1 2 0 25.0 0 1 26.0000 0 0 1 0.0
881 0 3 1 33.0 0 0 7.8958 0 0 1 0.0
882 0 3 0 22.0 0 0 10.5167 0 0 1 0.0
883 0 2 1 28.0 0 0 10.5000 0 0 1 0.0
884 0 3 1 25.0 0 0 7.0500 0 0 1 0.0
885 0 3 0 39.0 0 5 29.1250 0 1 0 0.0
886 0 2 1 27.0 0 0 13.0000 0 0 1 0.0
887 1 1 0 19.0 0 0 30.0000 0 0 1 0.0
889 1 1 1 26.0 0 0 30.0000 1 0 0 0.0
890 0 3 1 32.0 0 0 7.7500 0 1 0 0.0

558 rows × 11 columns