tensorflow Lite Segmentation fault

Question

System information

• OS Platform and Distribution :CentOS Linux release 7.7.1908 -TensorFlow version:2.3.0

I try to convert the tensorflow offical image caption model to TFLite model

And Now I have successfully convert the model using tf.lite.TFLiteConverter.from_concrete_functions as following:

@tf.function
def evaluate(img_tensor_val):
    temp_input = tf.expand_dims(img_tensor_val, 0)
    img_tensor_val = image_features_extract_model(temp_input)
    img_tensor_val = tf.reshape(img_tensor_val, (img_tensor_val.shape[0], -1, img_tensor_val.shape[3]))
    hidden = decoder.reset_states(batch_size=1)

    features = encoder(img_tensor_val)

    dec_input = tf.expand_dims([tokenizer.word_index['<start>']], 0)
    result = []

    for i in range(max_length):
        predictions, hidden, attention_weights = decoder(dec_input, features, hidden)
        print(predictions.shape)
        # result.append(predictions)
        predicted_id = tf.random.categorical(predictions, 1)[0][0]
        #
        #
        result.append(predicted_id)
        #
        #
        # if predicted_id == 3:
        #     return result
        # # result.append(tf.gather(tokenizer.index_word, predicted_id))
        # #
        # # if tf.gather(tokenizer.index_word, predicted_id) == '<end>':
        # #     return result
        #
        dec_input = tf.expand_dims([predicted_id], 0)
    return result

export_dir = "./"
tflite_enc_input = ''
ckpt.f = evaluate
to_save = evaluate.get_concrete_function(tf.TensorSpec(shape=(299, 299, 3),dtype=tf.dtypes.float32))

converter = tf.lite.TFLiteConverter.from_concrete_functions([to_save])
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS,
                                       tf.lite.OpsSet.SELECT_TF_OPS]
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)

And I Visualize the converted_model.tflite by Netorn: But when I invoke the interpreter the problem came: LOG:

2020-10-03 12:11:24.049222: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-10-03 12:11:30.184705: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1
2020-10-03 12:11:30.213363: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
pciBusID: 0000:af:00.0 name: Tesla V100-SXM2-32GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 31.72GiB deviceMemoryBandwidth: 836.37GiB/s
2020-10-03 12:11:30.213414: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-10-03 12:11:30.219666: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2020-10-03 12:11:30.223018: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2020-10-03 12:11:30.224419: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2020-10-03 12:11:30.227861: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2020-10-03 12:11:30.230195: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2020-10-03 12:11:30.236320: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2020-10-03 12:11:30.239374: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-10-03 12:11:30.239829: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-10-03 12:11:30.248265: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 2600000000 Hz
2020-10-03 12:11:30.249524: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5615faa7fa90 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-10-03 12:11:30.249552: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-10-03 12:11:30.381691: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5615faaec0c0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-10-03 12:11:30.381734: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Tesla V100-SXM2-32GB, Compute Capability 7.0
2020-10-03 12:11:30.383860: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
pciBusID: 0000:af:00.0 name: Tesla V100-SXM2-32GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 31.72GiB deviceMemoryBandwidth: 836.37GiB/s
2020-10-03 12:11:30.383900: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-10-03 12:11:30.383930: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2020-10-03 12:11:30.383944: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2020-10-03 12:11:30.383959: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2020-10-03 12:11:30.383973: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2020-10-03 12:11:30.383987: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2020-10-03 12:11:30.384002: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2020-10-03 12:11:30.387738: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-10-03 12:11:30.387786: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-10-03 12:11:31.156790: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-10-03 12:11:31.156840: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0 
2020-10-03 12:11:31.156853: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N 
2020-10-03 12:11:31.160006: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 30098 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-32GB, pci bus id: 0000:af:00.0, compute capability: 7.0)
**(299, 299, 3)**
INFO: Created TensorFlow Lite delegate for select TF ops.
2020-10-03 12:11:31.760387: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
pciBusID: 0000:af:00.0 name: Tesla V100-SXM2-32GB computeCapability: 7.0
coreClock: 1.53GHz coreCount: 80 deviceMemorySize: 31.72GiB deviceMemoryBandwidth: 836.37GiB/s
2020-10-03 12:11:31.760470: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-10-03 12:11:31.760523: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2020-10-03 12:11:31.760551: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2020-10-03 12:11:31.760577: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2020-10-03 12:11:31.760601: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2020-10-03 12:11:31.760625: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2020-10-03 12:11:31.760647: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2020-10-03 12:11:31.763282: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-10-03 12:11:31.763329: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-10-03 12:11:31.763346: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0 
2020-10-03 12:11:31.763360: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N 
2020-10-03 12:11:31.766083: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 30098 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-32GB, pci bus id: 0000:af:00.0, compute capability: 7.0)
**INFO: TfLiteFlexDelegate delegate: 51 nodes delegated out of 2014 nodes with 51 partitions.**

**Segmentation fault (core dumped)**

The invoke of Interpreter

def load_image(image_path):
    img = tf.io.read_file(image_path)
    img = tf.image.decode_jpeg(img, channels=3)
    img = tf.image.resize(img, (299,299))
    img = tf.keras.applications.inception_v3.preprocess_input(img)
    return img, image_path

image = load_image('./test.jpg')[0]
print(image.shape)


interpreter = tf.lite.Interpreter(model_path='./converted_model.tflite')
input_details = interpreter.get_input_details()
interpreter.allocate_tensors()
interpreter.set_tensor(input_details[0]['index'], image)
interpreter.invoke()

raw_prediction = interpreter.tensor(
    interpreter.get_output_details()[0]['index'])()
print(raw_prediction)

Please tell me what 's the problem of the program?What's the meaning of 'Segmentation fault (core dumped)' ?

Answer 1

After I change the GPU I use to CPU, the program is now correct.But I don't know the Why the theraw_prediction=291 .But In the evaluation function, the return is a list[].How can this be?