Accessing tensor numpy array using `dataset.map()` in tensorflow

Question

I am trying to access the numpy array from a tensor object that is processed with https://www.tensorflow.org/api_docs/python/tf/data/Dataset#map.

I get the error: AttributeError: 'Tensor' object has no attribute 'numpy'

When I try to access the tensor as: np_array = tensor.numpy()

While if I use: dataset.take(n), i am able to access the numpy array.

For more clarity on the situation I am facing, here is a short reproducible example of the error in a google colab:

https://colab.research.google.com/drive/13ectGEMDSygcyuW4ip9zrWaHO3pSxc3p?usp=sharing

Tensorflow version: 2.4.1

Update: Adding code in addition to the colab above:

import os
import numpy as np
import tensorflow as tf

# This works
def get_spectrogram_and_label_id(file_path):
    spectrogram, label = get_spectrogram(audio) # not showing the function here since it is irrelevant
    return spectrogram, label

# This doesn't!
def get_spec_and_label_time(spectrogram, label):
    time_step_spectrogram = generate_time_step_samples(spectrogram)
    return time_step_spectrogram, label

# I want to manipulate the Tensor by extracting the numpy array as part of the map function
def generate_time_step_samples(tensor):
    np_array = tensor.numpy() # ERROR: AttributeError: 'Tensor' object has no attribute 'numpy'
    # Do something with the numpy array
    return np_array

filenames = ['file1.wav', 'file2.wav', ...]
files_ds = tf.data.Dataset.from_tensor_slices(filenames)
spectrogram_ds = files_ds.map(get_spectrogram_and_label_id) # this works
spectrogram_time_ds = spectrogram_ds.map(get_spec_and_label_time) # this doesn't

More details in the google colab.

Answer 1

You cannot access .numpy() inside a .map() function.

This is not a bug, it is how TensorFlow works with static graphs behind the scenes.

Read my answer here for a more comprehensive explanation.

AttributeError: 'Tensor' object has no attribute 'numpy' in Tensorflow 2.1