Why no feature is being tracked using KLT-tracking in Matlab?

Question

I am trying to track some features (extracted using multiscale-harrys detector) between two frames using the Kanade-Lucas-Tomasi (KLT) algorithm using the functions you can find here (Mathworks documentation).

I cannot understand what goes wrong. None of the points can be tracked. I tried making bigger the number of iterations and changing the size of the window around the features but the result is always the same, no feature is tracked.

• Is it a problem in the data (images resolution is too low (240x180 pixels))?

• Is the problem in the selected features?

These are the two images I am using:

This is my code:

img = single(imread('img.png'));
end_img = single(imread('end_img.png'));

coord_first = [24,21;25,97;29,134;37,25;37,55;37,64;38,94;38,103;40,131;41,139;43,14;44,22;44,54;44,63;46,93;46,101;47,111;49,131;49,140;52,166;55,52;62,151;76,51;78,89;81,151;81,165;83,13;92,165;111,18;111,96;155,42;155,62;155,81;155,100;156,129;163,133;168,126;170,40;170,65;172,26;173,134;174,59;174,84;174,103;174,116;175,73;178,97;186,142;186,149;190,119;190,132;194,75;209,99;210,42;210,66;212,133;212,152;215,61;215,79;218,119];

% display of the target image and all the features I want to track
figure
imshow(img,[]), 
colormap gray
hold on
plot(coord_first(:,1), coord_first(:,2), 'r*');

% point tracker creation
% the paramters reported here are the default ones
pointTracker = vision.PointTracker('MaxIterations', 30, 'BlockSize', [31,31]);
% point tracker initialization
initialize(pointTracker,coord_first,img);

% actual tracking
[coord_end, point_validity] = step(pointTracker, end_img);

% display of all the correctly tracked featrures
figure
imshow(end_img,[]), 
colormap gray
hold on
plot(coord_end(point_validity,1), coord_end(point_validity,2), 'r*');

Answer 1

Check the contents of point_validity. If all elements of points_validity are false, then you would not see any points. If that is the case, the next question is why points were not tracked.

For an image of this size, try setting 'NumPyramidLevels' to 1.

Answer 2

Actually I have just solved the problem. Of course the problem was the fact that no point was tracked.

The problem is that the images given in input must have grayscale values in [0, 1] and not in [0, 255] (as I was doing).

There is no specific need to tune any of the parameter once the data are passed the right way (at least in my case with these low resolution grayscale images).