import pandas as pd
Text images of 28x28 pixels represented as flattened array of 784 pixels
Each pixel is represented by a pixel intensity value from 0-255
Download Link: https://www.kaggle.com/c/3004/download/train.csv
mnist_data = pd.read_csv("../data/mnist/train.csv")
mnist_data.tail()
The pixel intensities are divided by 255 so that they're all between 0 and 1
from sklearn.model_selection import train_test_split
features = mnist_data.columns[1:]
X = mnist_data[features]
Y = mnist_data['label']
X_train, X_test, Y_train, y_test = train_test_split(X/255., Y, test_size=0.1, random_state=0)
from sklearn.svm import LinearSVC
clf_svm = LinearSVC(penalty="l2", dual=False, tol=1e-5)
clf_svm.fit(X_train, Y_train)
from sklearn.metrics import accuracy_score
y_pred_svm = clf_svm.predict(X_test)
acc_svm = accuracy_score(y_test, y_pred_svm)
print ('SVM accuracy: ',acc_svm)
When your model has a number of hyper parameters, we've spoken earlier of the need to tune them to find the best possible model on your data set. Scikit-learn offers some specialized tools to perform exactly this tuning. It will help you choose the best possible model by using a few different values of the hyperparameters that you specify. This is done using the GridSearchCV.
When we instantiate our LinearSVC estimator which is going to be trained with various combinations of the parameters that we've specified in the grid, we can also pass in other arguments, which will remain constant during training.
The second argument to our GridSearchCV is the grid which contains our hyperparameter values. GridSearchCV will now run training on our data with every possible model parameter combination.
The CV parameter specifies that we want this model to be cross validated to mitigate over fitting. CV is equal to three means that the input data set will be split into three different parts. This is threefold cross validation. The training data will be two out of three parts and the validation data will be the third part.
from sklearn.model_selection import GridSearchCV
penalties = ['l1', 'l2']
tolerances = [1e-3, 1e-4, 1e-5]
param_grid = {'penalty': penalties, 'tol': tolerances}
grid_search = GridSearchCV(LinearSVC(dual=False), param_grid, cv=3)
grid_search.fit(X_train, Y_train)
grid_search.best_params_
clf_svm = LinearSVC(penalty="l1", dual=False, tol=1e-3)
clf_svm.fit(X_train, Y_train)
y_pred_svm = clf_svm.predict(X_test)
acc_svm = accuracy_score(y_test, y_pred_svm)
print ('SVM accuracy: ',acc_svm)