In [23]:
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, Flatten, Dense

Formula: Number of Parameters in a Convolutional Layer

  • K - the number of filters in the convolutional layer
  • F - the height and width of the convolutional filters
  • D_in - the depth of the previous layer

Since there are F F D_in weights per filter, and the convolutional layer is composed of K filters, the total number of weights in the convolutional layer is K F F D_in. Since there is one bias term per filter, the convolutional layer has K biases. Thus, the number of parameters in the convolutional layer is given by K F F D_in + K.

K F F * D_in + K

Formula: Shape of a Convolutional Layer

  • K - the number of filters in the convolutional layer
  • F - the height and width of the convolutional filters
  • S - the stride of the convolution
  • H_in - the height of the previous layer
  • W_in - the width of the previous layer

Notice that H_in and W_in are the first and second value of the input_shape tuple, respectively.

The depth of the convolutional layer will always equal the number of filters K.

If padding = 'same', then the spatial dimensions of the convolutional layer are the following:

height = ceil(float(H_in) / float(S))

width = ceil(float(W_in) / float(S))

If padding = 'valid', then the spatial dimensions of the convolutional layer are the following:

height = ceil(float(H_in - F + 1) / float(S))

width = ceil(float(W_in - F + 1) / float(S))

In [13]:
model = Sequential()
model.add(Conv2D(filters=16, kernel_size=2, strides=2, padding='valid', 
    activation='relu', input_shape=(200, 200, 1)))
model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_5 (Conv2D)            (None, 100, 100, 16)      80        
=================================================================
Total params: 80
Trainable params: 80
Non-trainable params: 0
_________________________________________________________________
In [14]:
model = Sequential()
model.add(Conv2D(filters=32, kernel_size=3, strides=2, padding='same', 
    activation='relu', input_shape=(128, 128, 3)))
model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_6 (Conv2D)            (None, 64, 64, 32)        896       
=================================================================
Total params: 896
Trainable params: 896
Non-trainable params: 0
_________________________________________________________________

Increasing the depth of the array

Input has depth of 3 and it is increased to 16-32-64 in subsequent layers.

In [21]:
model = Sequential()
model.add(Conv2D(filters=16, kernel_size=2, padding='same', 
    activation='relu', input_shape=(32, 32, 3)))
model.add(Conv2D(filters=32, kernel_size=2, padding='same', activation='relu'))
model.add(Conv2D(filters=64, kernel_size=2,  padding='same', activation='relu'))
model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_16 (Conv2D)           (None, 32, 32, 16)        208       
_________________________________________________________________
conv2d_17 (Conv2D)           (None, 32, 32, 32)        2080      
_________________________________________________________________
conv2d_18 (Conv2D)           (None, 32, 32, 64)        8256      
=================================================================
Total params: 10,544
Trainable params: 10,544
Non-trainable params: 0
_________________________________________________________________

Max Pooling Layers in Keras

MaxPooling2D(pool_size, strides, padding) You must include the following argument:

  • pool_size - Number specifying the height and width of the pooling window.

  • strides - The vertical and horizontal stride. If you don't specify anything, strides will default to pool_size.

  • padding - One of 'valid' or 'same'. If you don't specify anything, padding is set to 'valid'.

In [9]:
model = Sequential()
model.add(MaxPooling2D(pool_size=2, strides=2, input_shape=(100, 100, 15)))
model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
max_pooling2d_3 (MaxPooling2 (None, 50, 50, 15)        0         
=================================================================
Total params: 0
Trainable params: 0
Non-trainable params: 0
_________________________________________________________________

MaxPool layer decrease the spatial dimention from 32->16->8->4

In [22]:
model = Sequential()

model.add(Conv2D(filters=16, kernel_size=2, padding='same', 
    activation='relu', input_shape=(32, 32, 3)))
model.add(MaxPooling2D(pool_size=2))
model.add(Conv2D(filters=32, kernel_size=2, padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=2))
model.add(Conv2D(filters=64, kernel_size=2,  padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=2))

model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_19 (Conv2D)           (None, 32, 32, 16)        208       
_________________________________________________________________
max_pooling2d_4 (MaxPooling2 (None, 16, 16, 16)        0         
_________________________________________________________________
conv2d_20 (Conv2D)           (None, 16, 16, 32)        2080      
_________________________________________________________________
max_pooling2d_5 (MaxPooling2 (None, 8, 8, 32)          0         
_________________________________________________________________
conv2d_21 (Conv2D)           (None, 8, 8, 64)          8256      
_________________________________________________________________
max_pooling2d_6 (MaxPooling2 (None, 4, 4, 64)          0         
=================================================================
Total params: 10,544
Trainable params: 10,544
Non-trainable params: 0
_________________________________________________________________

Flatten the final maxpooling layer to vector, softmax in last layer returns probablities, commmon to have hidden layers with relu activation

In [24]:
model = Sequential()

model.add(Conv2D(filters=16, kernel_size=2, padding='same', 
    activation='relu', input_shape=(32, 32, 3)))
model.add(MaxPooling2D(pool_size=2))
model.add(Conv2D(filters=32, kernel_size=2, padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=2))
model.add(Conv2D(filters=64, kernel_size=2,  padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=2))
model.add(Flatten())
model.add(Dense(500, activation='relu'))
model.add(Dense(10, activation='softmax'))

model.summary()
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_22 (Conv2D)           (None, 32, 32, 16)        208       
_________________________________________________________________
max_pooling2d_7 (MaxPooling2 (None, 16, 16, 16)        0         
_________________________________________________________________
conv2d_23 (Conv2D)           (None, 16, 16, 32)        2080      
_________________________________________________________________
max_pooling2d_8 (MaxPooling2 (None, 8, 8, 32)          0         
_________________________________________________________________
conv2d_24 (Conv2D)           (None, 8, 8, 64)          8256      
_________________________________________________________________
max_pooling2d_9 (MaxPooling2 (None, 4, 4, 64)          0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 1024)              0         
_________________________________________________________________
dense_1 (Dense)              (None, 500)               512500    
_________________________________________________________________
dense_2 (Dense)              (None, 10)                5010      
=================================================================
Total params: 528,054
Trainable params: 528,054
Non-trainable params: 0
_________________________________________________________________
In [ ]: