How to calculate receptive field size?

Question

I'm reading paper about using CNN(Convolutional neural network) for object detection.

Rich feature hierarchies for accurate object detection and semantic segmentation

Here is a quote about receptive field:

The pool5 feature map is 6x6x256 = 9216 dimensional. Ignoring boundary effects, each pool5 unit has a receptive field of 195x195 pixels in the original 227x227 pixel input. A central pool5 unit has a nearly global view,
while one near the edge has a smaller, clipped support.

My questions are:

1. What is definition of receptive field?
2. How they compute size and location of receptive field?
3. How we can compute bounding rect of receptive field using caffe/pycaffe?

Answer 1

1) It is the size of the area of pixels that impact the output of the last convolution.

2) For each convolution and pooling operation, compute the size of the output. Now find the input size that results in an output size of 1x1. Thats the size of the receptive field

3) You don't need to use a library to do it. For every 2x2 pooling the output size is reduced by half along each dimension. For strided convolutions, you also divide the size of each dimension by the stride. You may have to shave off some of the dimension depending on if you use padding for your convolutions. The simplest case is to use padding = floor(kernel size/2), so that a convolution dose not have any extra change on the output size.

Answer 2

Here's the python script that calculates the RF size in addition to the stride and the size of the output.

    # [filter size, stride, padding]

convnet =[[11,4,0],[3,2,0],[5,1,2],[3,2,0],[3,1,1],[3,1,1],[3,1,1],[3,2,0],[6,1,0]]
layer_name = ['conv1','pool1','conv2','pool2','conv3','conv4','conv5','pool5','fc6-conv']
imsize = 227


def outFromIn(isz, layernum = 9, net = convnet):
    if layernum>len(net): layernum=len(net)

    totstride = 1
    insize = isz
    #for layerparams in net:
    for layer in range(layernum):
        fsize, stride, pad = net[layer]
        outsize = (insize - fsize + 2*pad) / stride + 1
        insize = outsize
        totstride = totstride * stride

    RFsize = isz - (outsize - 1) * totstride

    return outsize, totstride, RFsize

if __name__ == '__main__':

    print "layer output sizes given image = %dx%d" % (imsize, imsize)
    for i in range(len(convnet)):
        p = outFromIn(imsize,i+1)
        print "Layer Name = %s, Output size = %3d, Stride = % 3d, RF size = %3d" % (layer_name[i], p[0], p[1], p[2])

Answer 3

Assume that we have a network architecture that only comprise of multiple convolution layers. For each convolution layer, we define a square kernel size and a dilation rate. Also, Assume that the stride is 1. Thus, you can compute the receptive field of the network by the following piece of python code:

K=[3,3]   # Kernel Size
R=[1,2]  # Dilation Rate

RF=1
d=1 # Depth
for k,r in zip(K,R):
    support=k+(k-1)*(r-1) # r-dilated conv. adds r-1 zeros among coefficients
    RF=support+(RF-1)
    print('depth=%d, K=%d, R=%d, kernel support=%d'%(d,k,r,support))
    d=d+1
print('Receptive Field: %d'%RF)

As an example, let's compute the receptive field (RF) of the well-known DnCNN (denoising convolutional neural network) [1]. Use the above piece of code with the following inputs to compute the RF of that network. (you will get RF=35).

# In DnCNN-S, the network has 17 convolution layers.
K=[3]*17  # Kernel Size
R=[1]*17  # Dilation Rate

[1] Zhang, Kai, et al. "Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising." IEEE Transactions on Image Processing 26.7 (2017): 3142-3155.

Answer 4

UPDATE Dec 11, 2019:

The TF library was moved to https://github.com/google-research/receptive_field

See also the Distill paper "Computing Receptive Fields of Convolutional Neural Networks": https://distill.pub/2019/computing-receptive-fields/

OLD:

Tensorflow now supports receptive field computation, by simply using tf.contrib.receptive_field

See https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/receptive_field for details.

Answer 5

Here is another way to computes receptive field directly. Stackoverflow does not support math formula, for a more readable version, please refer to Calculating Receptive Field of CNN

The receptive field (RF) $l_k$ of layer $k$ is:

$$ l_k = l_{k-1} + ((f_k - 1) * \prod_{i=1}^{k-1}s_i) $$

where $l_{k-1}$ is the receptive field of layer $k-1$, $f_k$ is the filter size (height or width, but assuming they are the same here), and $s_i$ is the stride of layer $i$.

The formula above calculates receptive field from bottom up (from layer 1). Intuitively, RF in layer $k$ covers $(f_k - 1) * s_{k-1}$ more pixels relative with layer $k-1$. However, the increment needs to be translated to the first layer, so the increments is a factorial --- a stride in layer $k-1$ is exponentially more strides in the lower layers.

Hope this is helpful.

Answer 6

As above, with possibly correct computation of RF:

#Compute input size that leads to a 1x1 output size, among other things   

# [filter size, stride, padding]

convnet =[[11,4,0],[3,2,0],[5,1,2],[3,2,0],[3,1,1],[3,1,1],[3,1,1],[3,2,0],[6,1,0]]
layer_name = ['conv1','pool1','conv2','pool2','conv3','conv4','conv5','pool5','fc6-conv']
imsize = 227

def outFromIn(isz, layernum = 9, net = convnet):
    if layernum>len(net): layernum=len(net)

    totstride = 1
    insize = isz
    #for layerparams in net:
    for layer in range(layernum):
        fsize, stride, pad = net[layer]
        outsize = (insize - fsize + 2*pad) / stride + 1
        insize = outsize
        totstride = totstride * stride
    return outsize, totstride

def inFromOut( layernum = 9, net = convnet):
    if layernum>len(net): layernum=len(net)
    outsize = 1
    #for layerparams in net:
    for layer in reversed(range(layernum)):
        fsize, stride, pad = net[layer]
        outsize = ((outsize -1)* stride) + fsize
    RFsize = outsize
    return RFsize

if __name__ == '__main__':

    print "layer output sizes given image = %dx%d" % (imsize, imsize)
    for i in range(len(convnet)):
        p = outFromIn(imsize,i+1)
        rf = inFromOut(i+1)
        print "Layer Name = %s, Output size = %3d, Stride = % 3d, RF size = %3d" % (layer_name[i], p[0], p[1], rf)