Poling Layer is always succeed the Convolution Layer post activation function like in example shown below:
Now lets understand why we need Pooling Layer:
- There is a limitation of output from the convolutional layers that they record the precise position of features in the input. This information is huge and unwanted, so pooling layer down sample the data by reducing the resolution of the feature map but still retaining features of the map required.
- Pooling reduces the computation cost drastically
The most common pooling layer filter is of size 2×2, which discards three forth of the activations.
Common Pooling Techniques are –
Max pooling is where we take largest of the pixel values of a segment defined by pooling size
TensorFlow Core:
tf.nn.max_pool(
input, ksize, strides, padding, data_format=None, name=None
)
tf.nn.max_pool(input_tensor, ksize=2, strides=2, padding="SAME")
TensorFlow with Keras
tf.keras.layers.MaxPool2D(
pool_size=(2, 2),
strides=None,
padding='valid',
data_format=None,
**kwargs
)
tf.keras.layers.MaxPooling2D(pool_size=(2, 2),strides=(1, 1), padding='valid')
input – for the input tensor
ksize or pool_size- for the size of the kernel/mask/filter
strides – for the jump of each filter after every iteration
padding – for the padding algorithm – either same or valid
Avg pooling is where we take average of all pixel values of a segment defined by pooling size
TensorFlow Core:
tf.nn.avg_pool2d(
input, ksize, strides, padding, data_format='NHWC', name=None
)tf.nn.avg_pool2d(input_tensor, ksize=2, strides=2, padding='NHWC')
TensorFlow with Keras
tf.keras.layers.AveragePooling2D(
pool_size=(2, 2),
strides=None,
padding='valid',
data_format=None,
**kwargs
)tf.keras.layers.AveragePooling2D(pool_size=(2, 2),strides=(1, 1), padding='valid')
input – for the input tensor
ksize or pool_size- for the size of the kernel/mask/filter
strides – for the jump of each filter after every iteration
padding – for the padding algorithm – either same or valid
Global average pooling operation for spatial data and used as replacement of Fully Connected Layer
Instead of adopting the traditional fully connected layers for classification in CNN, we directly output the spatial average of the feature maps from the last conv layer as the confidence of categories via a global average pooling layer, and then the resulting vector is fed into the softmax layer.
In traditional CNN, it is difficult to interpret how the category level information from the objective cost layer is passed back to the previous convolution layer due to the fully connected layers which act as a black box in between. In contrast, global average pooling is more meaningful and interpretable as it enforces correspondance between feature maps and categories, which is made possible by a stronger local modeling using the micro network.
Furthermore, the fully connected layers are prone to overfitting and heavily depend on dropout regularization [4] [5], while global average pooling is itself a structural regularizer, which natively prevents overfitting for the overall structure.
tf.keras.layers.GlobalAveragePooling2D(
data_format=None, keepdims=False, **kwargs
)tf.keras.layers.GlobalAveragePooling2D()(x)
