I implemented a convolutional neural network in TensorFlow. However, instead of using the stride operator for the convolutional layers, I use tf.gather to pick every second element. Using this implementation I receive significantly worse results on cifar10 compared to the usual strides operator. Does somebody have an idea, how this can happen?
So basically what I do is:
input -> convolution -> custom_stride_operation -> convolution -> etc
and I compare this to:
Input -> convolution (with stride 2) -> convolution (with stride 2) -> etc
I use the following code as a replacement for the stride operation on a tensor x of shape (batch,height,width,channels):
i = tf.range(0,height//2,dtype=tf.int32)*2
i = tf.expand_dims(i,axis=0)
i = tf.repeat(i,tf.shape(input)[0],axis=0)
x = tf.gather(x, i, axis=1,batch_dims=1)
x = tf.gather(x, i, axis=2,batch_dims=1)