DCT method of MATLAB doesn't work in Kotlin Android

Question

I am passing an array of 64 bits/elements to DCT function. But when I send the dctResult (obtained from DCT) to IDCT for inverse, I don't get the same values what I passed to DCT function.

Below is the code for both the functions

DCT:

private fun applyDCT(block: Array<Int>): Array<Int> {
    val dctResult = Array(8) { DoubleArray(8) }
    for (u in 0 until 8) {
        for (v in 0 until 8) {
            var sum = 0.0
            for (x in 0 until 8) {
                for (y in 0 until 8) {
                    val cu = if (u == 0) 1 / kotlin.math.sqrt(2.0) else 1.0
                    val cv = if (v == 0) 1 / kotlin.math.sqrt(2.0) else 1.0
                    sum += cu * cv * block[(x * 8) + y] *
                            kotlin.math.cos((2 * x + 1) * u * PI / 16) *
                            kotlin.math.cos((2 * y + 1) * v * PI / 16)
                }
            }
            val alphaU = if (u == 0) 1 / kotlin.math.sqrt(2.0) else 1.0
            val alphaV = if (v == 0) 1 / kotlin.math.sqrt(2.0) else 1.0
            dctResult[u][v] = 0.25 * alphaU * alphaV * sum
        }
    }
    return applyIDCT(dctResult)
}

IDCT:

private fun applyIDCT(dctCoefficients: Array<DoubleArray>): Array<Int> {
    val block = Array(64) { 0 }
    for (x in 0 until 8) {
        for (y in 0 until 8) {
            var sum = 0.0
            for (u in 0 until 8) {
                for (v in 0 until 8) {
                    val cu = if (u == 0) 1 / kotlin.math.sqrt(2.0) else 1.0
                    val cv = if (v == 0) 1 / kotlin.math.sqrt(2.0) else 1.0
                    sum += cu * cv * dctCoefficients[u][v] *
                            kotlin.math.cos((2 * x + 1) * u * PI / 16) *
                            kotlin.math.cos((2 * y + 1) * v * PI / 16)
                }
            }
            block[x * 8 + y] = (0.25 * sum).toInt()
        }
    }
    return block
}

I want the same values in return after applying DCT and IDCT to an array of 64 elements.

Answer 1

I'm pretty sure you want to rebuild the two-dimensional MATLAB function dct2 and idct2 and not, as written, the column-wise one-dimensional function dct or idct. If so, then you are using the wrong formulas. For dct2 you must calculate

sum += block[(x * 8) + y] * kotlin.math.cos((2 * x + 1) * u * PI / 16) * kotlin.math.cos((2 * y + 1) * v * PI / 16)

and then divide this output matrix by 4 and also divide the first row and the first column by kotlin.math.sqrt(2.0) - the element [0][0] is thus finally divided by 8.

Do the reverse for idct2: Before you start the transformation, divide the input matrix by 4 and the first row and first column additionally by kotlin.math.sqrt(2.0) and then calculate the final result

sum += block[(x * 8) + y] * kotlin.math.cos((2 * x + 1) * u * PI / 16) * kotlin.math.cos((2 * y + 1) * v * PI / 16)

Source: Section "More About" of dct2 and idct2.