Android with Nexus 6 -- how to avoid decreased OpenSL audio thread priority relating to app focus?

Question

I'm encountering a strange problem when trying to implement low-latency streaming audio playback on a Nexus 6 running Android 6.0.1 using OpenSL ES.

My initial attempt seemed to be suffering from starvation issues, so I added some basic timing benchmarks in the buffer completion callback function. What I've found is that audio plays back fine if I continually tap the screen while my app is open, but if I leave it alone for a few seconds, the callback starts to take much longer. I'm able to reproduce this behavior consistently. A couple of things to note:

• "a few seconds" ~= 3-5 seconds, not long enough to trigger a screen change
• My application's activity sets FLAG_KEEP_SCREEN_ON, so no screen changes should occur anyway
• I have taken no action to try to increase the audio callback thread's priority, since I was under the impression that Android reserves high priority for these threads already
• The behavior occurs on my Nexus 6 (Android 6.0.1), but not on a Galaxy S6 I also have available (Android 5.1.1).

The symptoms I'm seeing really seem like the OS kicks down the audio thread priority after a few seconds of non-interaction with the phone. Is this right? Is there any way I can avoid this behavior?

Answer 1

While watching the latest Google I/O 2016 audio presentation, I finally found the cause and the (ugly) solution for this problem.

Just watch the around one minute of this you tube clip (starting at 8m56s): https://youtu.be/F2ZDp-eNrh4?t=8m56s

It explains why this is happening and how you can get rid of it.

In fact, Android slows the CPU down after a few seconds of touch inactivity to reduce the battery usage. The guy in the video promises a proper solution for this soon, but for now the only way to get rid of it is to send fake touches (that's the official recommendation).

Instrumentation instr = new Instrumentation();
instr.sendKeyDownUpSync(KeyEvent.KEYCODE_BACKSLASH); // or whatever event you prefer

Repeat this with a timer every 1.5 seconds and the problem will vanish.

I know, this is an ugly hack, and it might have ugly side effects which must be handled. But for now, it is simply the only solution.

Update: Regarding your latest comment ... here's my solution. I'm using a regular MotionEvent.ACTION_DOWN at a location outside of the screen bounds. Everything else interfered in an unwanted way with the UI. To avoid the SecurityException, initialize the timer in the onStart() handler of the main activity and terminate it in the onStop() handler. There are still situations when the app goes to the background (depending on the CPU load) in which you might run into a SecurityException, therefore you must surround the fake touch call with a try catch block.

Please note, that I'm using my own timer framework, so you have to transform the code to use whatever timer you want to use.

Also, I cannot ensure yet that the code is 100% bulletproof. My apps have that hack applied, but are currently in beta state, therefore I cannot give you any guarantee if this is working correctly on all devices and Android versions.

Timer fakeTouchTimer = null;
Instrumentation instr;
void initFakeTouchTimer()
{
    if (this.fakeTouchTimer != null)
    {
        if (this.instr == null)
        {
            this.instr = new Instrumentation();
        }
        this.fakeTouchTimer.restart();
    }
    else
    {
        if (this.instr == null)
        {
            this.instr = new Instrumentation();
        }
        this.fakeTouchTimer = new Timer(1500, Thread.MIN_PRIORITY, new TimerTask()
        {
            @Override
            public void execute()
            {
                if (instr != null && fakeTouchTimer != null && hasWindowFocus())
                {
                    try
                    {
                        long downTime = SystemClock.uptimeMillis();

                        MotionEvent event = MotionEvent.obtain(downTime, downTime, MotionEvent.ACTION_DOWN, -100, -100, 0);
                        instr.sendPointerSync(event);
                        event.recycle();
                    }
                    catch (Exception e)
                    {
                    }
                }
            }
        }, true/*isInfinite*/);
    }
}
void killFakeTouchTimer()
{
    if (this.fakeTouchTimer != null)
    {
        this.fakeTouchTimer.interupt();
        this.fakeTouchTimer = null;
        this.instr = null;
    }
}

@Override
protected void onStop()
{
    killFakeTouchTimer();
    super.onStop();

    .....
}

@Override
protected void onStart()
{
    initFakeTouchTimer();
    super.onStart();

    .....
}

Answer 2

It is well known that the audio pipeline in Android 6 has been completely rewritten. While this improved latency-related issues in most cases, it is not impossible that it generated a number of undesirable side-effects, as is usually the case with such large-scale changes.

While your issue does not seem to be a common one, there are a few things you might be able to try:

• Increase the audio thread priority. The default priority for audio threads in Android is -16, with the maximum being -20, usually only available to system services. While you can't assign this value to you audio thread, you can assign the next best thing: -19 by using the ANDROID_PRIORITY_URGENT_AUDIO flag when setting the thread's priority.

• Increase the number of buffers to prevent any kind of jitter or latency (you can even go up to 16). However on some devices the callback to fill a new buffer isn’t always called when it should.

• This SO post has several suggestions to improve audio latency on Anrdoid. Of particular interest are points 3, 4 and 5 in the accepted answer.

• Check whether the current Android system is low-latency-enabled by querying whether hasSystemFeature(FEATURE_AUDIO_LOW_LATENCY) or hasSystemFeature(FEATURE_AUDIO_PRO).

---

Additionally, this academic paper discusses strategies for improving audio latency-related issues in Android/OpenSL, including buffer- and callback interval-related approaches.

Answer 3

Force resampling to native device sample rate on Android 6.

Use the device's native sample rate of 48000. For example:

SLDataFormat_PCM dataFormat;

dataFormat.samplesPerSec = 48000;