Drawing a Box Around Face To Existed Photos with Google Face Detection ML Kit

Question

We implemented Android ML Kit for face detection in Android. It works like charm, detect faces.

The problem: We want to draw rectangles around detected faces when multiple faces detected

What we have done:

Implemented

 implementation 'com.google.android.gms:play-services-mlkit-face-detection:16.1.5'

Created a custom View :

class FaceView(val theContext : Context, val bounds : Rect) : View(theContext) {

    override fun onDraw(canvas: Canvas?) {
        super.onDraw(canvas)

        val myPaint = Paint()

        myPaint.color = Color.BLACK
        myPaint.style = Paint.Style.STROKE
        myPaint.strokeWidth = 10f

        canvas?.drawRect(bounds, myPaint)
    }
}

Tried to draw a rectangle to the bound we got from the face object ML kit created

val result = detector.process(image).addOnSuccessListener { faces ->

for (face in faces) {
val bounds = face.boundingBox

val view = FaceView(requireContext(), bounds)
binding.actionRoot.addView(view)

val lp : ConstraintLayout.LayoutParams =
 ConstraintLayout.LayoutParams(bounds.width(),bounds.height())

lp.startToStart = binding.actionPhoto.id
lp.topToTop = binding.actionPhoto.id

lp.marginStart = bounds.right
lp.topMargin = bounds.bottom
        
view.layoutParams = lp
}}

Result :

How can we draw a rectangle for each face that we produced from URI(not from CameraX) and make them clickable?

Answer 1

-- While ML kit gives Rect of detected Faces. I think Google Developers make it more easier for us and draw rectangle automatically and provide faces as standalone bitmap objects.

For my solution :

I used the example project that @anonymous suggested to draw lines around the face.

• First I got start, end, bottom and top points from provided face rect. ( This rect is created from original bitmap, not from the imageview. So the rect points are belong to the original bitmap I was using).

• As Rect points are not belong to ImageView but the bitmap itself, We need to find related rect points on ImageView (or create a scaled bitmap and detect faces on it). We calculated this points as percentages in original Bitmap.

• While we use original points as in the example project to draw lines. We implemented a view with Transparent Background with using calculated points to make face rectangles clickable.

-As the last thing we created bitmaps for each face.

    detector.process(theImage)
                    .addOnSuccessListener { faces ->
    
           val bounds = face.boundingBox
            val screenWidth = GetScreenWidth().execute()
    
            // val theImage = InputImage.fromBitmap(tempBitmap,0)
    
            val paint = Paint()
            paint.strokeWidth = 1f
            paint.color = Color.RED
            paint.style = Paint.Style.STROKE
    
            val theStartPoint = if(bounds.left < 0) 0 else{ bounds.left}
            val theEndPoint = if(bounds.right > tempBitmap.width) tempBitmap.width else { bounds.right}
            val theTopPoint = if(bounds.top < 0) 0 else { bounds.top }
            val theBottomPoint = if(bounds.bottom > tempBitmap.height) tempBitmap.height else { bounds.bottom }
    
            val faceWidth = theEndPoint - theStartPoint
            val faceHeight = theBottomPoint - theTopPoint
    
            Log.d(Statics.LOG_TAG, "Face width : ${faceWidth} Face Height $faceHeight")
    
            val startPointPercent = theStartPoint.toFloat() / tempBitmap.width.toFloat()
            val topPointPercent = theTopPoint.toFloat() / tempBitmap.height.toFloat()
    
            Log.d(Statics.LOG_TAG, "Face start point percent : ${startPointPercent} Face top percent $topPointPercent")
    
            val faceWidthPercent = faceWidth / tempBitmap.width.toFloat()
            val faceHeightPercent = faceHeight / tempBitmap.height.toFloat()
    
            Log.d(Statics.LOG_TAG, "Face width  percent: ${faceWidthPercent} Face Height Percent $faceHeightPercent")
    
            val faceImage = ConstraintLayout(requireContext())
            faceImage.setBackgroundColor(Color.TRANSPARENT)
            binding.actionRoot.addView(faceImage)
    
            val boxWidth = screenWidth.toFloat()*faceWidthPercent
            val boxHeight = screenWidth.toFloat()*faceHeightPercent
    
            Log.d(Statics.LOG_TAG, "Box width : ${boxWidth} Box Height $boxHeight")
    
            val lp : ConstraintLayout.LayoutParams =
                ConstraintLayout.LayoutParams(
                    boxWidth.toInt(),
                    boxHeight.toInt()
                )
    
    
            lp.startToStart = binding.actionPhoto.id
            lp.topToTop = binding.actionPhoto.id
    
            lp.marginStart = (screenWidth * startPointPercent).toInt()
            lp.topMargin = (screenWidth * topPointPercent).toInt()
    
            faceImage.layoutParams = lp
    
            val theFaceBitmap = Bitmap.createBitmap(
                tempBitmap,
                theStartPoint,
                theTopPoint,
                faceWidth,
                faceHeight)
    
    
            if (face.trackingId != null) {
                val id = face.trackingId
    
                faceImage.setOnClickListener {
                    
                
                    binding.actionPhoto.setImageBitmap(theFaceBitmap)
                    
                 
                }
            }
}

Result :

Answer 2

you can reference the project here, but it is the java code

https://github.com/kkdroidgit/FaceDetect