How to Use Graph Cut for the Statistics of Patch Offsets for Image Completion Model in MATLAB

Question

I am trying to use Shai Bagon's (@Shai) Graph Cut MATLAB Toolbox GCMex for the model from Statistics of Patch Offsets for Image Completion (Image Completion Approaches Using the Statistics of Similar Patches).

The model is given by:

Let's say the number of labels (Shifts) is 4 (Up by 1, down by 2, right by 3 or left by 4) to make things simpler.
I am still not sure how to set the data structures (Arrays) to utilize the solver.

Does anyone have idea?

Answer 1

I'm not sure my GCMex implementation supports this type of smoothness term.

If I understand correctly, you need, for each neighboring i, j to compute a different LxL matrix of weights. The underlying c++ package (by Boykov et al) does support this, but I did not create an interface for this option.

Update:

Let's say you have n pixels and l possible labels.
The current GCMex interface supports pair-wise weights of the form:

E(i, L(i), j, L(j)) = w(i,j) * S(L(i), L(j)) 

That is, the main "source" of pair wise penalty is a fixed lxl matrix S(L(i), L(j)) that defines the penalty of assigning neighboring pixels i and j to labels L(i) and L(j) respectively. This lxl weight does not depend on the location of the pixels i and j only on their labels L(i) and L(j).
The only spatial dependency comes from a scalar w(i,j) that modulates S by a scalar that depends on the locaiton of i and j.

However, it seems like in your case, what you actually need is a lxl matrix for each pair i and j.

I think you should look into void setSmoothness(smoothFnCoord cost);.