what ImagePairData layers means in Caffe?

Question

I have a Caffe .prototxt file and i want to convert Caffe layers in Keras or TensorFlow. There is one layer type: "ImagePairData", i don't understand what this means and what's its conversion to Keras or TensorFlow? Here is the Layer:

layer {
  name: "pairdata"
  type: "ImagePairData"
  top: "data"
  top: "label"
  image_pair_data_param {
    image_dir: "benchmark_val/train/images"
    label_dir: "benchmark_val/train/gt"
    batch_size: 10
    h_img: 256
    w_img: 256
    h_map: 256
    w_map: 256
    channels: 3
    mean: 0
    scale: 1
    multiclass: false
  }

  include: {phase: TRAIN}
}

What layer is similar to this layer in Keras or TensorFlow?

Answer 1

This layer is not part of caffe's "basic" layers (the layers caffe is "shipped" with, see a list of caffe layers here). It is probably a custom layer written by whomever you are trying to take this model from.

Without looking at the code I cannot tell you exactly what this layer does, but my guess is that this layer provides two inputs to the net:
1. "data" of size 10-3-256-256 (batch_size: 10, channels: 3 and h_img, w_img: 256)
2. "label" of size 10-1-256-256 (since multiclass: false I assume only one "channel" here)

I suppose this layer is an input layer to segmentation/pixel labeling task providing both input ("data") and reference ground truth of the same spatial size ("label").

I think you should write your own input layer in Keras/TensorFlow to have similar functionality.