Simple Explanation for Calculating the Number of Parameters in Convolutional Neural Network

Total number of parameters, output shape in the Convolution layer

Photo by Yan Krukov from Pexels

In this article, we will go through two important concepts in the convolution layer.

  1. How to calculate the number of parameters?
  2. How the shape of output is calculated?

First, let’s understand the important terms used in the convolution layer.

Important terms

1. input_shape

input_shape = (batch_size, height, width, depth)
batch_size= number of training examples in one forward/backward pass

2. output_shape

output_shape = (batch_size, height, width, depth)

3. filter

In a convolution neural network, input data is convolved over with a filter which is used to extract features. Filter/kernel is a matrix that will move over the image pixel data (input) and will perform a dot product with that particular region of that input data and the output will be the matrix of the dot product.

 

Gif — by Author

How to calculate number of parameters and shape of output in convolution layer

Scenario 1:

Input:

  • filters = 1
  • kernel_size=(3,3)
  • input_shape=(10,10,1)

Let’s calculate the number of parameters and output shape in Conv2D.

 

Image by Author

1. How to calculate the number of parameters in the convolution layer?

  • Weights in one filter of size(3,3)= 3*3 =9
  • Bias =1 
     [One bias will be added to each filter. Since only one filter kernel is used, bias =1]
  • Total parameters for one filter kernel of size (3,3) = 9+1 =10

2. How to calculate the Output shape?

 

Formula

s →stride, p →padding, n →input size, f →filter size

Stride by default =1 , padding is not mentioned (so,p=0)

Output shape = n-f+1 = 10–3+1 =8

After applying convolution on the input image using a convolution filter, the output will be a feature map. The number of channels in the feature map depends on the number of filters used. Here, in this example, only one filter is used. So, the number of channels in the feature map is 1.

So, Output_shape of feature map= (8,8,1)

Code Snippet

model1=keras.models.Sequential()
model1.add(Conv2D(filters=1,kernel_size=3,input_shape=(10,10,1),activation='relu'))
model1.summary()

Output:


Scenario 2:

Input:

  • filters = 5
  • kernel_size=(3,3)
  • input_shape=(10,10,1)

 

Image by Author

1. How to calculate the number of parameters in the convolution layer?

  • Parameters in one filter of size(3,3)= 3*3 =9
  • Bias =1 
     [One bias will be added to each filter]
  • Total parameters for filter kernel of size (3,3) = 9+1 =10
  • The total number of filters= 5.
  • Total parameters for 5 filter kernel of size (3,3) = 10*5=50

2. How to calculate the Output shape?

n=10,f=3,s=1,p=0

Stride by default =1 , padding is not mentioned (so,p=0)

Output shape = n-f+1 = 10–3+1 =8

After applying convolution on the input image using a convolution filter, the output will be a feature map. The number of channels in the feature map depends on the number of filters used. Here, in this example, 5 filters are used. So, the number of channels in the feature map is 5.

So, Output_shape of feature map= (8,8,5)

Code Snippet

model2=keras.models.Sequential()
model2.add(Conv2D(filters=5,kernel_size=3,input_shape=(10,10,1),activation='relu'))
model2.summary()

Output:


Scenario 3:

Input:

  • filters = 5
  • kernel_size=(3,3)
  • input_shape=(10,10,3)

 

Image by Author

1. How to calculate the number of parameters in the convolution layer?

  • Parameters in one filter of size(3,3)= 3*3 =9
  • The filter will convolve over all three channels concurrently(input_image depth=3). So parameters in one filter will be 3*3*3=27 
    [filter size * input_data depth]
  • Bias =1 
     [One bias will be added to each filter]
  • Total parameters for one filter kernel of size (3,3) for depth 3=(3*3*3)+1=28
  • The total number of filters= 5.
  • Total parameters for 5 filter kernel of size (3,3) , input_image depth(3)= 28*5=140

2. How to calculate the Output shape?

n=10,f=3,s=1,p=0

Stride by default =1 , padding is not mentioned (so,p=0)

Output shape = n-f+1 = 10–3+1 =8

After applying convolution on the input image using a convolution filter, the output will be a feature map. The number of channels in the feature map depends on the number of filters used. Here, in this example, 5 filters are used. So, the number of channels in the feature map is 5.

So, Output_shape of feature map= (8,8,5)

Code Snippet

model3=keras.models.Sequential()
model3.add(Conv2D(filters=5,kernel_size=3,input_shape=(10,10,3),activation='relu'))
model3.summary()

Output:

Conclusion

In this article, I have covered how to calculate the number of parameters in the convolution layer. And also how to calculate the shape of the convolved image.

Thanks for reading!

References:


If you like to read more of my tutorials on Python and Data Science,
follow me on
Medium, Twitter

One-Time
Monthly
Yearly

Make a one-time donation

Make a monthly donation

Make a yearly donation

Choose an amount

$5.00
$15.00
$100.00
$5.00
$15.00
$100.00
$5.00
$15.00
$100.00

Or enter a custom amount

$

Your contribution is appreciated.

Your contribution is appreciated.

Your contribution is appreciated.

Buy Me a CoffeeBuy Me a CoffeeBuy Me a Coffee

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s