Getting error in model summary and extracting feature for Vision Transformer model

Question

I am writing a code for vision transformers for image feature extraction. I had defined a ViT model from this github site.

image_model = ViT(
    image_size=224,
    patch_size=32,
    num_classes=1000,
    dim=1024,
    depth=6,
    heads=16,
    mlp_dim=2048,
    dropout=0.1,
    emb_dropout=0.1
)

# Image Shape is 224x224 RGB
input_shape = (1, 3, 224, 224)
output_shape = (1, 1024)

new_input = tf.keras.Input(shape=input_shape)
new_input = tf.squeeze(new_input, axis=0)
new_input = tf.zeros(input_shape)
new_input = tf.convert_to_tensor(new_input)
print(f'Input Shape : {new_input.shape}')
hidden_layer = image_model(new_input)
^^^^^     
Here I am getting error saying: AttributeError: EagerTensor object has no attribute 'reshape'. 
        If you are looking for numpy-related methods, please run the following:
        from tensorflow.python.ops.numpy_ops import np_config
        np_config.enable_numpy_behavior()

print(f'\nOutput Shape : {hidden_layer.shape}')

image_features_extract_model = tf.keras.Model(new_input, output_shape, name="image_features_extract_model")

Can anyone suggest how do i write the code to implement this. And after this generation of model I am doing image_features_extract_model.summary() <--- I want this as well

Answer 1

Did you achieve what you wanted? I'm pretending to extract deep features from pretrained Visual transformers, I think that maybe it's similar to your task!