Clicking/distortion noise at start of mixed audio in java

Question

I am attempting to mix multiple .wav sound files (merging musical notes to create a chord) in Java. The end result is mostly great, except during the first 23ms of the mixed audio where there's a loud clicking noise. It does not matter which soundfiles I mix, the issue always persists and the waveforms always look identical during the clicking. Below is a screenshot of 2 mixed audio clips for different chords:

And below is the waveform of one of the individual audiofiles used in the mixing process:

This is my mixing code in Java:

private byte[] mixAudioFiles(List<String> fileNames) throws Exception {
    // NOTE: For simplification, assume all files are mono, 16-bit PCM, 16kHz sample rate and 4 seconds long
    int sampleRate = 16000;
    int bytesPerSample = 2;
    long numSamples = sampleRate * 4;
    int totalBytes = (int) (numSamples * bytesPerSample);
    int[] mixedSamples = new int[(int) numSamples];

    for (String fileName : fileNames) {
        int resId = reactContext.getResources().getIdentifier(fileName, "raw", reactContext.getPackageName());
        InputStream inputStream = reactContext.getResources().openRawResource(resId);
        byte[] audioBytes = toByteArray(inputStream);

        for (int i = 0; i < numSamples && i < audioBytes.length / 2; i++) {
            int byteIndex = i * bytesPerSample;
            short sample = (short) ((audioBytes[byteIndex + 1] << 8) | (audioBytes[byteIndex] & 0xFF));
            mixedSamples[i] += sample;
        }
    }

    byte[] mixedAudio = new byte[totalBytes];
    for (int i = 0; i < numSamples; i++) {
        if (mixedSamples[i] > Short.MAX_VALUE || mixedSamples[i] < Short.MIN_VALUE) {
            mixedSamples[i] = Math.min(Math.max(mixedSamples[i], Short.MIN_VALUE), Short.MAX_VALUE);
        }
        mixedAudio[i * 2] = (byte) (mixedSamples[i] & 0xFF);
        mixedAudio[i * 2 + 1] = (byte) ((mixedSamples[i] >> 8) & 0xFF);
    }

    return mixedAudio;
}

private byte[] toByteArray(InputStream inputStream) throws Exception {
    ByteArrayOutputStream buffer = new ByteArrayOutputStream();
    int nRead;
    byte[] data = new byte[16384];

    while ((nRead = inputStream.read(data, 0, data.length)) != -1) {
        buffer.write(data, 0, nRead);
    }

    buffer.flush();
    return buffer.toByteArray();
}

All audio files used during mixing are mono channel, 16-bit PCM, 16kHz sample rate and 4 seconds long. It does not matter if I play the buffer directly in my app, or first export it to a wav file. The clicking noise always persists.

I've also tried fading in the amplitude from zero. But even then I hear the clicking noise faintly.

Does anyone have any idea what the culprit could be? I'm not very knowledgable in audio processing. As a last resort I could just trim the first 23ms off the mixed clip, but I would prefer to maintain the complete audio clip.