Tensorboard embeddings does not display tensors?

Question

Tensorboard provides embedding visualization of tensorflow variables by using tf.train.Saver(). The following is a working example (from this answer)

import os
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
from tensorflow.contrib.tensorboard.plugins import projector


LOG_DIR = '/tmp/emb_logs/'
metadata = os.path.join(LOG_DIR, 'metadata.tsv')

mnist = input_data.read_data_sets('MNIST_data')

#Variables
images = tf.Variable(mnist.test.images, name='images')
weights=tf.Variable(tf.random_normal([3,3,1,16]))
biases=tf.Variable(tf.zeros([16]))

#Tensor from the variables
x = tf.reshape(images, shape=[-1, 28, 28, 1])
conv_layer=tf.nn.conv2d(x, weights, [1,1,1,1], padding="SAME")
conv_layer=tf.add(conv_layer, biases)
y = tf.reshape(conv_layer, shape=[-1, 28*28*16])


with open(metadata, 'wb') as metadata_file:
    for row in mnist.test.labels:
        metadata_file.write('%d
' % row)

with tf.Session() as sess:
    saver = tf.train.Saver([images])

    sess.run(images.initializer)
    saver.save(sess, os.path.join(LOG_DIR, 'images.ckpt'))

    config = projector.ProjectorConfig()
    # One can add multiple embeddings.
    embedding = config.embeddings.add()
    embedding.tensor_name = images.name
    # Link this tensor to its metadata file (e.g. labels).
    embedding.metadata_path = metadata
    # Saves a config file that TensorBoard will read during startup.
    projector.visualize_embeddings(tf.summary.FileWriter(LOG_DIR), config)

How can I visualize embeddings from a tensorflow tensor, like y in the code above?

Simply replacing

saver = tf.train.Saver([images])

with

saver = tf.train.Saver([y])

doesn't work, because of the following error:

 474         var = ops.convert_to_tensor(var, as_ref=True)
 475         if not BaseSaverBuilder._IsVariable(var):
 476           raise TypeError("Variable to save is not a Variable: %s" % var)
 477         name = var.op.name
 478         if name in names_to_saveables:

 TypeError: Variable to save is not a Variable: Tensor("Reshape_11:0", shape=(10000, 12544), dtype=float32)

Does anyone know of an alternative to generate tensorboard embedding visualizations of a tf.tensor?

Answer 1

You could create a new variable you can assign the value of y to.

y_var = tf.Variable(tf.shape(y))
saver = tf.train.Saver([y_var])
assign_op = y_var.assign(y)
...
# Training
sess.run((loss, assign_op))
saver.save(...)

Answer 2

tf.summary.tensor_summary will save a summary for an arbitrary Tensor.