Simple Explanation for Calculating the Number of Parameters in Convolutional Neural Network

Total number of parameters, output shape in the Convolution layer

In this article, we will go through two important concepts in the convolution layer.

1. How to calculate the number of parameters?
2. How the shape of output is calculated?

---

First, let’s understand the important terms used in the convolution layer.

Important terms

1. input_shape

input_shape = (batch_size, height, width, depth)
batch_size= number of training examples in one forward/backward pass

2. output_shape

output_shape = (batch_size, height, width, depth)

3. filter

In a convolution neural network, input data is convolved over with a filter which is used to extract features. Filter/kernel is a matrix that will move over the image pixel data (input) and will perform a dot product with that particular region of that input data and the output will be the matrix of the dot product.

 

---

How to calculate number of parameters and shape of output in convolution layer

Scenario 1:

Input:

• filters = 1
• kernel_size=(3,3)
• input_shape=(10,10,1)

Let’s calculate the number of parameters and output shape in Conv2D.

 

1. How to calculate the number of parameters in the convolution layer?

• Weights in one filter of size(3,3)= 3*3 =9
• Bias =1 
   [One bias will be added to each filter. Since only one filter kernel is used, bias =1]
• Total parameters for one filter kernel of size (3,3) = 9+1 =10

2. How to calculate the Output shape?

 

s →stride, p →padding, n →input size, f →filter size

Stride by default =1 , padding is not mentioned (so,p=0)

Output shape = n-f+1 = 10–3+1 =8

After applying convolution on the input image using a convolution filter, the output will be a feature map. The number of channels in the feature map depends on the number of filters used. Here, in this example, only one filter is used. So, the number of channels in the feature map is 1.

So, Output_shape of feature map= (8,8,1)

Code Snippet

model1=keras.models.Sequential()
model1.add(Conv2D(filters=1,kernel_size=3,input_shape=(10,10,1),activation='relu'))
model1.summary()

Output:

---

Scenario 2:

Input:

• filters = 5
• kernel_size=(3,3)
• input_shape=(10,10,1)

 

1. How to calculate the number of parameters in the convolution layer?

• Parameters in one filter of size(3,3)= 3*3 =9
• Bias =1 
   [One bias will be added to each filter]
• Total parameters for filter kernel of size (3,3) = 9+1 =10
• The total number of filters= 5.
• Total parameters for 5 filter kernel of size (3,3) = 10*5=50

2. How to calculate the Output shape?

n=10,f=3,s=1,p=0

Stride by default =1 , padding is not mentioned (so,p=0)

Output shape = n-f+1 = 10–3+1 =8

After applying convolution on the input image using a convolution filter, the output will be a feature map. The number of channels in the feature map depends on the number of filters used. Here, in this example, 5 filters are used. So, the number of channels in the feature map is 5.

So, Output_shape of feature map= (8,8,5)

Code Snippet

model2=keras.models.Sequential()
model2.add(Conv2D(filters=5,kernel_size=3,input_shape=(10,10,1),activation='relu'))
model2.summary()

Output:

---

Scenario 3:

Input:

• filters = 5
• kernel_size=(3,3)
• input_shape=(10,10,3)

 

1. How to calculate the number of parameters in the convolution layer?

• Parameters in one filter of size(3,3)= 3*3 =9
• The filter will convolve over all three channels concurrently(input_image depth=3). So parameters in one filter will be 3*3*3=27 
  [filter size * input_data depth]
• Bias =1 
   [One bias will be added to each filter]
• Total parameters for one filter kernel of size (3,3) for depth 3=(3*3*3)+1=28
• The total number of filters= 5.
• Total parameters for 5 filter kernel of size (3,3) , input_image depth(3)= 28*5=140

2. How to calculate the Output shape?

n=10,f=3,s=1,p=0

Stride by default =1 , padding is not mentioned (so,p=0)

Output shape = n-f+1 = 10–3+1 =8

After applying convolution on the input image using a convolution filter, the output will be a feature map. The number of channels in the feature map depends on the number of filters used. Here, in this example, 5 filters are used. So, the number of channels in the feature map is 5.

So, Output_shape of feature map= (8,8,5)

Code Snippet

model3=keras.models.Sequential()
model3.add(Conv2D(filters=5,kernel_size=3,input_shape=(10,10,3),activation='relu'))
model3.summary()

Output:

Conclusion

In this article, I have covered how to calculate the number of parameters in the convolution layer. And also how to calculate the shape of the convolved image.

Thanks for reading!

References:

---

If you like to read more of my tutorials on Python and Data Science,
follow me on Medium, Twitter