Tensorflow Intel MKL Optimization with NHWC data format

Question

TensorFlow is compiled with the Intel MKL optimizations, many operations will be optimized and support NCHW.

Can someone please explain, why does Intel MKL support NCHW format more than NHWC?

Answer 1

TensorFlow default NHWC format is not the most efficient data layout for CPU and it results in some additional conversion overhead.Hence Intel MKL support NCHW format

Answer 2

MKLDNN will run most of its compute-heavy math on a custom (Blocked) memory layout, and the conversion from NCHW --> blocked takes similar time to NHWC --> blocked. So there should be no difference for pure MKLDNN primitive accelerations. The difference is in some of the GEMM-based math, which works on the NCHW format by default, meaning that some matmul calls will NOT require layout conversions if input format is NCHW.