What is the effect of calling TensorArray.close()?

Question

(tensorflow version: '0.12.head')

The documentation of TensorArray.close says that it close the current TensorArray. What does it mean for the status of TensorArray? I try the following code

import tensorflow as tf
sess = tf.InteractiveSession()
a1 = tf.TensorArray(tf.int32, 2)
a1.close().run()
a2 = a1.write(0, 0)
a2.close().run()
print(a2.read(0).eval())

And there are no errors. What is the usage of close?

Learning-to-learn includes TensorArray.close in the reset operations of the network. I can't figure out what the comment Empty array as part of the reset process means.

---

Update

For examples,

import tensorflow as tf

sess = tf.InteractiveSession()

N = 3

def cond(i, arr):
    return i < N

def body(i, arr):
    arr = arr.write(i, i)
    i += 1
    return i, arr

arr = tf.TensorArray(tf.int32, N)
_, result_arr = tf.while_loop(cond, body, [0, arr])
reset = arr.close() # corresponds to https://github.com/deepmind/learning-to-learn/blob/6ee52539e83d0452051fe08699b5d8436442f803/meta.py#L370

NUM_EPOCHS = 3
for _ in range(NUM_EPOCHS):
    reset.run() # corresponds to https://github.com/deepmind/learning-to-learn/blob/6ee52539e83d0452051fe08699b5d8436442f803/util.py#L32
    print(result_arr.stack().eval())

Why arr.close() doesn't make the while loop fail? What are the advantages of calling arr.close() at the beginning of each epoch?

Answer 1

The TensorArray that is being closed in the Learning-to-learn example is not the original TensorArray that's being passed to the while-loop.

# original array (fx_array) declared here
fx_array = tf.TensorArray(tf.float32, size=len_unroll + 1,
                          clear_after_read=False)
# new array (fx_array) returned here
_, fx_array, x_final, s_final = tf.while_loop(
    cond=lambda t, *_: t < len_unroll,
    body=time_step,
    loop_vars=(0, fx_array, x, state),
    parallel_iterations=1,
    swap_memory=True,
    name="unroll")

Any subsequent calls to fx_array.close() from here close the new array returned by the while-loop and not the original array passed to the loop in the first iteration.

If you want to see how close behaves as expected then run:

session.run([reset, loss])

This will fail with TensorArray has already been closed. since the loss op tries to run pack() on the closed array.

Answer 2

This is a Python op which wraps a native op and both have help strings, but the native op help string is more informative. If you look at inspect.getsourcefile(fx_array.close) it'll point you to tensorflow/python/ops/tensor_array_ops.py. Inside the implementation you see that it defers to _tensor_array_close_v2. So you can do this

> from tensorflow.python.ops import gen_data_flow_ops
> help(gen_data_flow_ops._tensor_array_close_v2)
Delete the TensorArray from its resource container.  This enables
the user to close and release the resource in the middle of a step/run.

That same doc string is also in tensorflow/core/ops/ops.pbtxt under TensorArrayCloseV2

Looking at tensorflow/core/kernels/tensor_array_ops.cc you see that TensorArrayCloseOp is the implementation registered for TensorArrayCloseV2 in , and has more info

// Delete the TensorArray from its resource container.  This enables
// the user to close and release the resource in the middle of a step/run.
// TODO(ebrevdo): decide whether closing the grad op should happen
// here or on the python side.
class TensorArrayCloseOp : public OpKernel {
 public:
  explicit TensorArrayCloseOp(OpKernelConstruction* context)
      : OpKernel(context) {}

  void Compute(OpKernelContext* ctx) override {
    TensorArray* tensor_array;
    OP_REQUIRES_OK(ctx, GetTensorArray(ctx, &tensor_array));
    core::ScopedUnref unref(tensor_array);
    // Instead of deleting this TA from the ResourceManager, we just
    // clear it away and mark it as closed.  The remaining memory
    // consumed store its mutex and handle Tensor.  This will be
    // cleared out at the end of the step anyway, so it's fine to keep
    // it around until the end of the step.  Further calls to the
    // TensorArray will fail because TensorArray checks internally to
    // see if it is closed or not.

The description seems inconsistent with the behavior you are seeing, could be a bug.