Android Baseline Profiles - No difference in App Startup Time (StartupTimingMetric)

Question

This is my Macrobenchmark test to measure StartupTimingMetric

@RunWith(AndroidJUnit4::class)
@LargeTest
class Startup {

@get:Rule
val rule = MacrobenchmarkRule()

@Test
fun measureScrollWithoutBaseline() {
    startup(CompilationMode.None())
}

@Test
fun measureScrollWithBaseline() {
    startup(CompilationMode.Partial(baselineProfileMode = BaselineProfileMode.Require))
}

private fun startup(mode: CompilationMode) {
    rule.measureRepeated(
        packageName = PACKAGE_NAME,
        metrics = listOf(StartupTimingMetric()),
        startupMode = StartupMode.COLD,
        compilationMode = mode,
        iterations = 5,
        setupBlock = {
            killProcess()
            pressHome()
        },
        measureBlock = {
            startActivityAndWait()
        }
    )
}


companion object {
    const val PACKAGE_NAME = "com.xyz"
}
}

I've collected the Baseline Profiles using

@RunWith(AndroidJUnit4::class)
class StartupProfile {

@get:Rule
val rule = BaselineProfileRule()

@OptIn(ExperimentalStableBaselineProfilesApi::class)
@Test
fun collectBaselineProfileAppStartup() {
    rule.collectStableBaselineProfile(
        packageName = PACKAGE_NAME,
        includeInStartupProfile = true,
        maxIterations = 1
    ) {
        startActivityAndWait()
    }
}

companion object {
    const val PACKAGE_NAME = "com.xzy"
 }
}

I've confirmed that Baseline Profiles are present in the APK using APK Explorer in Studio

assets > dexopt > baseline.prof & baseline.profm

Also, I've confirmed that Baseline Profile are being applied by checking the log

Installing profile for com.xzy

My app Gradle includes as suggested by https://developer.android.com/topic/performance/baselineprofiles/dex-layout-optimizations

experimentalProperties["android.experimental.art-profile-r8-rewriting"] = true
experimentalProperties["android.experimental.r8.dex-startup-optimization"] = true

---

Results

WithOutBaselineProfiles

timeToInitialDisplayMs
min 384.5,
median 430.3,
max 643.0 Traces: Iteration 0 1 2 3 4

WithBaselineProfiles

timeToInitialDisplayMs
min 395.7,
median 445.7,
max 662.7 Traces: Iteration 0 1 2 3 4

Tried the same thing again where the median - withOutBaseline is 523 and withBaseline is 549 for the same iterations (5 times)

---

The good part is FrameTimingMetric where I've seen consistent gains when using Baseline Profiles.

Testing on Pixel 6A, Android 13

What mistakes might I be making in this situation? Alternatively, is this the highest level of optimization I can achieve?

Answer 1

I also faced the same issue in pixel device , in your mobile go to settings > turn off adaptive battery after that do the benchmark definitely u will see improvements because it fixed for me and

Answer 2

I think maxIterations = 1 might not be enough to capture all the code for Baseline Profiles. You might check the baseline-prof.txt file to see if there are classes/methods from your package (as a sanity checkt).

You also can check the first iteration of the system tracing recorded with Macrobenchmark and check JitThreadpool for JITting activity. There should be none (or very small amount).

Alternatively, you might add TraceSectionMetric("Jit compiling %", Mode.Sum) as a metric to the benchmark to see how much jittin is occuring.