Are there alternatives to constant_initializer when assigning weights in tf.contrib.layers

Question

I want to pass weights to tensorflow.contrib.layers.conv2d. The layers have the parameter weights_initializer. When passing the tensor via weights_initializer=tf.constant_initializer(tensor), the tensor is additionally added as a node to the graph, causing the size of the model to increase.

Is there an alternative to this weight initialization?

I know that tf.nn.conv2d accepts the weights as a parameter. The current model I am working with, however, uses the contrib-layers.

Answer 1

If you want to initialize the weights to some constant but you don't want to store that constant in the graph, can use a placeholder and feed a value for it on initialization. Just have something like:

weight_init = tf.placeholder(tf.float32, <shape>)
# As a parameter to your layer
weights_initializer=lambda *a, **k: weight_init

Note the shape of weight_init must match the size of the weights tensor. Then, on initialization:

init_op = tf.global_variables_initializer()
with tf.Session() as sess:
    sess.run(init_op, feed_dict={weight_init: <initial weight value>})

Alternatively, you can use no initializer and, instead of calling an initialization op, use the load method of the weight variable. For this you would have to access that variable first:

with tf.Session() as sess:
    weight_var.load(<initial weight value>, session=sess)