Track eye balls using tracking.js

Question

I am working with a red eye remover effect with html5 canvas. I used tracking.js to track the eyes. Check the following code.

Html

<!-- Include tracking js libraries -->
<script src="js/tracking-min.js"></script>
<script src="js/eye-min.js"></script>

<img id="sourceImg" src="red_eye1.jpg"  width="202" height="187">
<canvas id="myCanvas" width="202" height="187" style="border:1px solid #d3d3d3;"></canvas>

Javascript

document.getElementById("sourceImg").onload = function() {
    var c = document.getElementById("myCanvas");
    var ctx = c.getContext("2d");
    var img = document.getElementById("sourceImg");
    ctx.drawImage(img, 0, 0);

    // track eyes
    var tracker = new tracking.ObjectTracker(['eye']);
    tracker.setStepSize(1.7);
    tracking.track('#sourceImg', tracker);
    tracker.on('track', function(event) {
        event.data.forEach(function(rect) {
            // rect consists area of each eyes.
            var imgData = ctx.getImageData(rect.x, rect.y, rect.width, rect.height);
            // remove red eye
            var i;
            for (i = 0; i < imgData.data.length; i += 4) {
                // find and check red intensity
                var redIntensity = (imgData.data[i] / ((imgData.data[i+1] + imgData.data[i+2]) / 2));
                if (redIntensity > 1.5)  // 1.5 because it gives the best results
                {
                    // reduce red to the average of blue and green
                    imgData.data[i] = (imgData.data[i+1] + imgData.data[i+2]) / 2;
                }
            }
            ctx.putImageData(imgData, rect.x, rect.y);

        });
    });  
};

I got the following result.

Here the red eye removed successfully. But it affected the color around the eye area. Because the tracking.js returned the eye area including the surrounding area. How to track the eye balls, which will solve this issue.

Answer 1

A quick look at TrackingJS shows that it doesn't offer to take your red-bordered "eye region" and refine it to just the Iris.

So you will have to find a tracking app that does offer this refinement or code the refinement yourself.

Recommending an iris tracking library is off-topic in Stackoverflow and a code answer which does iris tracking is too broad to qualify as a good Stackoverflow answer.

Having said that, here is a verrrrrry broad outline for finding the Iris:

• Start with the "eye region" detected by TrackingJS,

• Get the bounding box of the sclera by color testing for the whitest pixels in that eye region,

• In this bounding box get the circular pupil area by color testing for the darkest pixels between the 2 sclera regions.

• Using the circular pupil as a basis, find the iris by expanding that pupil circle's radius outward until you reach the white sclera.

[ Additional thoughts ]

I continued thinking about the many nonstandard situations that need to be handled:

• The eyelid is closed or squinting,

• The iris is fully left or right leaving no whitish boundary encapsulating the iris,

• The sclera is not colored distinctly enough due to being jaundiced or bloodshot,

• Eye cosmetics that might be whiter than the sclara,

• Photo imperfections like lens flare and reflections,

• And more I'm not thinking of!

If your project allows, it would be much easier if the user clicks on the 2 pupils and expand the pupil radius to define the iris. Pupils are likely to be both darker than other eye structures and will be round (but possibly clipped by the eyelids).

If your project can't allow the user to select the pupils, you might use an "edge detection" algorithm to isolate the pupil (maybe Sobel Edge detection).