Why does python produce a more accurate result than kotlin when using tensorflow?

44 Views Asked by At

I am making an app that will detect different handwritten mathematical expressions in different number systems. As of right now, the main thing that is stopping any progress is the inaccuracies kotlin is producing when using tensorflow lite - roughly 10% correct. My python code is very similar, but it uses regular tensorflow, and gets it a lot more accurate - about 70% correct.

My thoughts are the way the image is converted from an OpenCV Mat to a Tensorbuffer is causing some issues, that or the way the pre-processing is handled is causing the discrepancies.

Snippets of my code are as follows:

  1. After extracting the bounding rectangles, perform preprocessing and normalisation.
val image_roi = Mat(tmp, boundRect)
Imgproc.cvtColor(image_roi, image_roi, Imgproc.COLOR_RGB2GRAY)                Imgproc.GaussianBlur(image_roi, image_roi, Size(3.0,3.0), 0.0)
Imgproc.dilate(image_roi, image_roi, Imgproc.getStructuringElement(Imgproc.MORPH_RECT, Size(4.0, 4.0)))
Imgproc.threshold(image_roi, image_roi, 90.0, 255.0, Imgproc.THRESH_BINARY);
Imgproc.resize(image_roi, image_roi, Size(28.0,28.0))
Core.normalize(image_roi, image_roi, 0.0, 255.0, Core.NORM_MINMAX);
image_roi.convertTo(image_roi, CvType.CV_8UC1)
extracted.add(image_roi) 
  1. Run Predicitions, convert OpenCV Mat to Tensorbuffer (step 3)
for (img in extracted) {
       val tensorBuffer = extractBytes(img)
       val outputs = model.process(tensorBuffer)
       val outputFeature0 = outputs.outputFeature0AsTensorBuffer
       val conf = outputFeature0.floatArray
       out += getLanguageText(conf, numbers)
 
}
  1. Convert Mat to Tensorbuffer
private fun extractBytes(img: Mat): TensorBuffer{
        val inputFeature = TensorBuffer.createFixedSize(intArrayOf(1, 28, 28, 1), DataType.FLOAT32)
        val byteBuffer = ByteBuffer.allocateDirect(28 * 28 * 4) // 4 bytes per float
        byteBuffer.order(ByteOrder.nativeOrder())
        byteBuffer.rewind()
 
        for (i in 0 until 28) {
            for (j in 0 until 28) {
                val temp = img.get(i, j)[0].toFloat() // Assuming single-channel (gray)
                byteBuffer.putFloat(temp)
            }
        }
 
        inputFeature.loadBuffer(byteBuffer)
        return inputFeature
    }

In the paste bin below I have included my pythin code also. I would like some help figuriring out why my model wont predict accurately through Kotlin, but will through python.

https://pastebin.com/BACzTkq6

Here are examples of the differences using the same image in Python and Kotlin:

Image showing Predicitions through Kotlin

Image showing Predicitions through python

I have tried different methods of conversion for the Matrix to Tensorbuffer, I have tried removing most, if not all, preprocessing on the images, I have tried getting python to work in Android studio (However that hasnt worked.)

0

There are 0 best solutions below