Bluetoothsocket affects thread that plays and record audio

Question

I have an app that runs 3 threads at the same time. One thread is for stablishing a bluetooth connection between the phone and another bluetooth device (Arduino). Thread 2 plays audio incoming from another phone via bluetooh. Thread 3 records and sends audio to the other phone via bluetooth.

The audio communication works with lots of glitches if phone is trying to stablish a connection with the Arduino (when thread 1 is running bluetoothsocket.connect();). However, when phone does not try to stablish a connection with the Arduino or the connection is already stablished and thread 1 is done, then the communication is good.

Here is the code for thread 1 - arduino (this code is with a class)

public class ConnectThread extends Thread {
        private final BluetoothSocket mmSocket;
        private final BluetoothDevice mmDevice;

        public ConnectThread(BluetoothDevice device) {
            // As mmSocket is final, we use a temporary socket variable
            BluetoothSocket tmp = null;
            mmDevice = device;

            // Get a BluetoothSocket to connect with the given BluetoothDevice
            try {
                // MY_UUID is the app's UUID string, also used by the server code
                tmp = device.createRfcommSocketToServiceRecord(UUID);
            } catch (IOException e) { }
            mmSocket = tmp;
        }

        public void run() {
            android.os.Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);
            // Cancel discovery because it will slow down the connection
            mBluetoothAdapter.cancelDiscovery();

            try {
                // Connect the device through the socket. This will block
                // until it succeeds or throws an exception
                mmSocket.connect();
            } catch (IOException connectException) {
                // Unable to connect; close the socket and get out
                try {
                    mmSocket.close();
                } catch (IOException closeException) { }
                // Send the name of the disconnected device back to the UI Activity
                sendDeviceConnectionToActivity(deviceMAC, false);
                Log.d("Bluetoot connected -->", "NNNNNNNN" + connectException);
                return;
            }

            // Do work to manage the connection (in a separate thread)
            manageConnectedSocket(mmSocket, mmDevice);
//            mConnectedThread = new ConnectedThread(mmSocket);
//            mConnectedThread.start();
            Log.d("Bluetoot connected -->", mmDevice.getName());
        }

        /** Will cancel an in-progress connection, and close the socket */
        public void cancel() {
            try {
                mmSocket.close();
            } catch (IOException e) { }
        }
    }

the code for audio in thread 2 and 3 (this code is with a another class)

public void audioCreate() {
        // Audio track object
        track = new AudioTrack(AudioManager.STREAM_VOICE_CALL,
                16000, AudioFormat.CHANNEL_OUT_MONO,
                encoding, minSize, AudioTrack.MODE_STREAM);
        // Audio record object
        recorder = new AudioRecord(MediaRecorder.AudioSource.VOICE_COMMUNICATION, 16000,
                AudioFormat.CHANNEL_IN_MONO, encoding,
                bufferSize);
    }

public void initiateBluetoothConexion(BluetoothDevice deviceSelected) {
//        Toast.makeText(getApplicationContext(), "Service On", Toast.LENGTH_SHORT).show();
        deviceMAC = deviceSelected.getAddress();

        mBluetoothAdapter.cancelDiscovery();
        // Cancel any thread attempting to make a connection
        if (mConnectThread != null) {
            mConnectThread.cancel();
            mConnectThread = null;
        }
        mConnectThread = new ConnectThread(deviceSelected);
        mConnectThread.setPriority(10);
        mConnectThread.start();
    }

    public class ConnectThread extends Thread {
        private final BluetoothSocket mmSocket;
        private final BluetoothDevice mmDevice;

        public ConnectThread(BluetoothDevice device) {
            // As mmSocket is final, we use a temporary socket variable
            BluetoothSocket tmp = null;
            mmDevice = device;

            // Get a BluetoothSocket to connect with the given BluetoothDevice
            try {
                // MY_UUID is the app's UUID string, also used by the server code
                tmp = device.createRfcommSocketToServiceRecord(UUID);
            } catch (IOException e) { }
            mmSocket = tmp;
        }

        public void run() {
            android.os.Process.setThreadPriority(Process.THREAD_PRIORITY_AUDIO);
            // Cancel discovery because it will slow down the connection
            mBluetoothAdapter.cancelDiscovery();

            try {
                // Connect the device through the socket. This will block
                // until it succeeds or throws an exception
                mmSocket.connect();
            } catch (IOException connectException) {
                // Unable to connect; close the socket and get out
                try {
                    mmSocket.close();
                } catch (IOException closeException) { }
                // Send the name of the disconnected device back to the UI Activity
                sendDeviceConnectionToActivity(deviceMAC, false);
                return;
            }

            // Do work to manage the connection (in a separate thread)
            manageConnectedSocket(mmSocket, mmDevice);
//            mConnectedThread = new ConnectedThread(mmSocket);
//            mConnectedThread.start();
            Log.d("Bluetoot connected -->", mmDevice.getName());
        }

        /** Will cancel an in-progress connection, and close the socket */
        public void cancel() {
            try {
                mmSocket.close();
            } catch (IOException e) { }
        }
    }

    private void manageConnectedSocket(BluetoothSocket mmSocket, BluetoothDevice mmDevice) {
        // Cancel the thread that completed the connection
//        if (mConnectThread != null) {
//            mConnectThread.cancel();
//            mConnectThread = null;
//        }
        // Cancel any thread currently running a connection
        if (mConnectedThread != null) {
            mConnectedThread.cancel();
            mConnectedThread = null;
        }
        // Start the thread to manage the connection and perform transmissions
        mConnectedThread = new ConnectedThread(mmSocket);
        mConnectedThread.setPriority(10);
        mConnectedThread.start();
        // Send the name of the connected device back to the UI Activity
        Log.d(TAG, "Connected to " + mmDevice.getName());
        sendDeviceConnectionToActivity(mmDevice.getAddress(), true);
//        setState(STATE_CONNECTED);
    }

    public class ConnectedThread extends Thread {
        private final BluetoothSocket mmSocket;
        private final InputStream mmInStream;
        private final OutputStream mmOutStream;
        private byte buffer[] = null;
        private byte playBuffer[] = null;
        private boolean intercomm = false;

        public ConnectedThread(BluetoothSocket socket) {
            mmSocket = socket;
            InputStream tmpIn = null;
            OutputStream tmpOut = null;

            // Get the input and output streams; using temp objects because
            // member streams are final.
            try {
                tmpIn = socket.getInputStream();
            } catch (IOException e) {
                Log.e(TAG, "Error occurred when creating input stream", e);
            }
            try {
                tmpOut = socket.getOutputStream();
            } catch (IOException e) {
                Log.e(TAG, "Error occurred when creating output stream", e);
            }

            mmInStream = tmpIn;
            mmOutStream = tmpOut;
            intercomm = true;
        }

        public void run() {
            android.os.Process.setThreadPriority(Process.THREAD_PRIORITY_URGENT_AUDIO);
            playBuffer = new byte[minSize];
            // Playback received audio
            track.play();

            startRecording();

            // receive recording until an exception occurs.
            while (intercomm) {
                try {
                    if (mmInStream.available() == 0) {
                        //Do nothing
                    } else {
                        mmInStream.read(playBuffer);
                        track.write(playBuffer, 0, playBuffer.length);
                    }
                } catch (IOException e) {
                    Log.d("AUDIO", "Error when receiving recording");
                    sendDeviceConnectionToActivity(deviceMAC, false);
                    break;
                }
            }
        }

        // Record Audio
        public void startRecording() {
            Log.d("AUDIO", "Assigning recorder");
            buffer = new byte[bufferSize];

            // Start Recording
            recorder.startRecording();
            Log.d("startRecording", "passed");
            // Start a thread
            recordingThread = new Thread(new Runnable() {
                @Override
                public void run() {
                    android.os.Process.setThreadPriority(Process.THREAD_PRIORITY_URGENT_AUDIO);
                    Log.d("startRecording", "sendRecording");
                    sendRecording();
                }
            }, "AudioRecorder Thread");
            recordingThread.setPriority(10);
            recordingThread.start();
        }
        // Method for sending Audio
        public void sendRecording() {
            // Infinite loop until microphone button is released
            while (intercomm) {
                try {
                    recorder.read(buffer, 0, bufferSize);
                    mmOutStream.write(buffer);
                } catch (IOException e) {
                    Log.d("AUDIO", "Error when sending recording");
                    sendErrorsToActivity("Error sending audio");
                }
            }
        }

        // Call this method from the main activity to shut down the connection.
        public void cancel() {
            intercomm = false;
            stopPlaying();
            stopRecording();
            destroyProcesses();
            try {
                mmSocket.close();
            } catch (IOException e) {
                Log.e(TAG, "Could not close the connect socket", e);
            }
        }

        // Stop playing and free up resources
        public void stopPlaying() {
            if (track != null) {
                track.stop();
                track.flush();
            }
        }

        // Stop Recording and free up resources
        public void stopRecording() {
            if (recorder != null) {
                recorder.stop();
            }
        }

        public void destroyProcesses() {
            //Release resources for audio objects
            track.release();
            recorder.release();
        }
    }

I tested the code in an octacore android oreo. However, when I did it in an phone sdk 23, it was worst.

Answer 1

Your AudioTrack is starving because it is not receiving data quickly enough from the arduino. This is most likely due to increased network contention during the BT connection process.

You appear to be configuring your AudioTrack with the smallest possible play-buffer. On most devices, this is only a few ms of audio, so if the AudioTrack isn't fed more data every few ms, it will starve, and you will hear a glitch.

One solution is to increase the AudioTrack's buffer size (perhaps to around 8000 samples or more).

In addition, you have not checked the return value from mmInStream.read(), which means you may be trying to "play" a playBuffer that is only partially filled.

Altering the thread priorities, as you are, is unlikely to make a qualitative difference.