Using Rubberband with oboe on Android in Stereo

27 Views Asked by At

I'm building a music player in an Android App using Oboe. For the decoding of mp3-Files I'm using FFMPEG. Now I'm working on a time stretch option for my player and thought about using Rubberband. Everything works fine if I resample my Audio to mono. But as soon, as I'm using Stereo (audioProperties->channelCount=2) the result contains a lot of noise and it seems, that the Rubberband Algorithm introduces a lot of empty samples. Here is my Algorithm to read Frames from an mp3 File and put the processed Frames into a Fifo-Buffer to be read by oboe:

    int result = 0;


    if (avPacket->stream_index == stream->index && avPacket->size > 0) {
        // Pass our compressed data into the codec
        result = avcodec_send_packet(codecContext, avPacket);
        if (result != 0) {
            LOGE("avcodec_send_packet error: %s", av_err2str(result));
            cleanup();
            return -1;
        }

        // Retrieve our raw data from the codec
        result = avcodec_receive_frame(codecContext, decodedFrame);
        if (result == AVERROR(EAGAIN)) {
            // The codec needs more data before it can decode
            LOGI("avcodec_receive_frame return EAGAIN");
            av_packet_unref(avPacket);
            return 0;
        } else if (result != 0){
            LOGE("avcodec_receive_frame error: %s", av_err2str(result));
            cleanup();
            return -1;
        }

        // Do resampling
        auto dst_nb_samples = (int32_t) av_rescale_rnd(
                swr_get_delay(swr, decodedFrame->sample_rate) + decodedFrame->nb_samples,
                audioProperties->sampleRate,
                decodedFrame->sample_rate,
                AV_ROUND_UP);


        float *buffer1;
        av_samples_alloc(
                (uint8_t **) &buffer1,
                nullptr,
                audioProperties->channelCount,
                dst_nb_samples,
                AV_SAMPLE_FMT_FLT,
                0);
        int frame_count = swr_convert(
                swr,
                (uint8_t **) &buffer1,
                dst_nb_samples,
                (const uint8_t **) decodedFrame->data,
                decodedFrame->nb_samples);


        stretcher->process(&buffer1, int2size_t(frame_count), false);

        int available_Samples = stretcher->available();
        if(available_Samples>0) {
            int min_Samples = stretcher->getSamplesRequired();

            float *outp;
            av_samples_alloc(
                    (uint8_t **) &outp,
                    nullptr,
                    audioProperties->channelCount,
                    available_Samples,
                    AV_SAMPLE_FMT_FLT,
                    0);

            size_t samples_retrieved = stretcher->retrieve(&outp, int2size_t(available_Samples) );
            LOGD("Frames: %i, dst_nb_sampes: %i, frame_count: %i, Samples: available - %i, min- %i, retreived - %i", frame_count, dst_nb_samples, frame_count, available_Samples, min_Samples, samples_retrieved);


            av_audio_fifo_write(fifo, (void **)&outp, size_t2int(samples_retrieved) );
            av_freep(&outp);
        }

        av_freep(&buffer1);

        av_packet_unref(avPacket);
    }

I tried using the same code in mono (audioProperties->channelCount=1) and it worked fine. I also tried different ways of allocating the buffers.

I hope that you have some ideas what the source of the problem could be.

0

There are 0 best solutions below