using audio flinger in shared library to capture audio buffer in android system

Question

I'm working on a native shared library that will be built inside aosp source tree, in this path "sdk-version/external"

in this shared library i'm trying to use audio flinger to use some of its function so i can capture audio buffer or track that will played in android system on device.

how can i use the audioflinger inside my shared library

i tried doing this

#include <system/audio.h>
#include <media/AudioSystem.h>
#include <media/AudioFlinger.h>

but i got this error

i also tried using the audio flinger with binder since audio flinger is part of ipc mechanism like this

// Get a reference to the AudioFlinger service using the IServiceManager
    android::sp<android::IBinder> binder =
        android::defaultServiceManager()->getService(android::String16("media.audio_flinger"));

if (binder != nullptr) {
     
        
        gAudioFlinger = android::interface_cast<android::IAudioFlinger>(binder);
    } 

i got this error

Answer 1

Keep in mind that Android native dev is analogous to Linux native dev if you new to it, though be aware that Google likes to break things.

If you are going to use Android Binder to do IPC on Android, you most likely want to use AIDL/HIDL instead of doing everything manually. AIDL/HIDL are just source code generation mechanisms, and the generated code can be found in out/soong/.intermediates.

In your case, you want to call AudioFlinger's Android Binder interface, in other words, you want to use the interface defined in IAudioFlignerService.aidl, so your code should look like this if you are using the cpp backend.

#include <android/media/IAudioPolicyService.h>
...
android::interface_cast<android::media::IAudioFlingerService>(binder);

Okay, where does the compiler find android/media/IAudioPolicyService.h and since interface_cast is a template function that just calls asInterface of the type argument type, where can it find the definition and declaration of IAudioFlingerService?

template<typename INTERFACE>
inline sp<INTERFACE> interface_cast(const sp<IBinder>& obj)
{
    return INTERFACE::asInterface(obj);
} 

Well the source code for the definition and declaration for IAudioFlingerService is in out/soong/.intermediates/frameworks/av/media/libaudioclient/audioflinger-aidl-${cpp_or_ndk}-source, and in the AOSP source tree the generated AIDL/HIDL code are generally compiled into (shared) libraries and linked to form an ELF, so the compiler needs to know the location of the libraries too. They are in out/soong/.intermediates/frameworks/av/media/libaudioclient/audioflinger-aidl-${cpp_or_ndk}.

In your Android.bp or Android.mk file, just include audioflinger-aidl-${cpp_or_ndk} as a shared library dependency and things should be fine at build time. At runtime, you may run into /dev/binder issues and you WILL runinto selinux issues. Once those are fixed, you still have to face this on Android 13:

status_t AudioFlinger::onTransactWrapper(TransactionCode code,
                                         [[maybe_unused]] const Parcel& data,
                                         [[maybe_unused]] uint32_t flags,
                                         const std::function<status_t()>& delegate) {
    // make sure transactions reserved to AudioPolicyManager do not come from other processes
    switch (code) {
        case TransactionCode::SET_STREAM_VOLUME:
        case TransactionCode::SET_STREAM_MUTE:
        case TransactionCode::OPEN_OUTPUT:
        case TransactionCode::OPEN_DUPLICATE_OUTPUT:
        case TransactionCode::CLOSE_OUTPUT:
        case TransactionCode::SUSPEND_OUTPUT:
        case TransactionCode::RESTORE_OUTPUT:
        case TransactionCode::OPEN_INPUT:
        case TransactionCode::CLOSE_INPUT:
        case TransactionCode::SET_VOICE_VOLUME:
        case TransactionCode::MOVE_EFFECTS:
        case TransactionCode::SET_EFFECT_SUSPENDED:
        case TransactionCode::LOAD_HW_MODULE:
        case TransactionCode::GET_AUDIO_PORT:
        case TransactionCode::CREATE_AUDIO_PATCH:
        case TransactionCode::RELEASE_AUDIO_PATCH:
        case TransactionCode::LIST_AUDIO_PATCHES:
        case TransactionCode::SET_AUDIO_PORT_CONFIG:
        case TransactionCode::SET_RECORD_SILENCED:
        case TransactionCode::AUDIO_POLICY_READY:
        case TransactionCode::SET_DEVICE_CONNECTED_STATE:
        case TransactionCode::SET_REQUESTED_LATENCY_MODE:
        case TransactionCode::GET_SUPPORTED_LATENCY_MODES:
        case TransactionCode::INVALIDATE_TRACKS:
        case TransactionCode::GET_AUDIO_POLICY_CONFIG:
            ALOGW("%s: transaction %d received from PID %d",
                  __func__, code, IPCThreadState::self()->getCallingPid());
            // return status only for non void methods
            switch (code) {
                case TransactionCode::SET_RECORD_SILENCED:
                case TransactionCode::SET_EFFECT_SUSPENDED:
                    break;
                default:
                    return INVALID_OPERATION;
            }
            // Fail silently in these cases.
            return OK;
        default:
            break;
    }

    // make sure the following transactions come from system components
    switch (code) {
        case TransactionCode::SET_MASTER_VOLUME:
        case TransactionCode::SET_MASTER_MUTE:
        case TransactionCode::MASTER_MUTE:
        case TransactionCode::GET_SOUND_DOSE_INTERFACE:
        case TransactionCode::SET_MODE:
        case TransactionCode::SET_MIC_MUTE:
        case TransactionCode::SET_LOW_RAM_DEVICE:
        case TransactionCode::SYSTEM_READY:
        case TransactionCode::SET_AUDIO_HAL_PIDS:
        case TransactionCode::SET_VIBRATOR_INFOS:
        case TransactionCode::UPDATE_SECONDARY_OUTPUTS:
        case TransactionCode::SET_BLUETOOTH_VARIABLE_LATENCY_ENABLED:
        case TransactionCode::IS_BLUETOOTH_VARIABLE_LATENCY_ENABLED:
        case TransactionCode::SUPPORTS_BLUETOOTH_VARIABLE_LATENCY: {
            if (!isServiceUid(IPCThreadState::self()->getCallingUid())) {
                ALOGW("%s: transaction %d received from PID %d unauthorized UID %d",
                      __func__, code, IPCThreadState::self()->getCallingPid(),
                      IPCThreadState::self()->getCallingUid());
                // return status only for non-void methods
                switch (code) {
                    case TransactionCode::SYSTEM_READY:
                        break;
                    default:
                        return INVALID_OPERATION;
                }
                // Fail silently in these cases.
                return OK;
            }
        } break;
        default:
            break;
    }

    // List of relevant events that trigger log merging.
    // Log merging should activate during audio activity of any kind. This are considered the
    // most relevant events.
    // TODO should select more wisely the items from the list
    switch (code) {
        case TransactionCode::CREATE_TRACK:
        case TransactionCode::CREATE_RECORD:
        case TransactionCode::SET_MASTER_VOLUME:
        case TransactionCode::SET_MASTER_MUTE:
        case TransactionCode::SET_MIC_MUTE:
        case TransactionCode::SET_PARAMETERS:
        case TransactionCode::CREATE_EFFECT:
        case TransactionCode::SYSTEM_READY: {
            requestLogMerge();
            break;
        }
        default:
            break;
    }

    const std::string methodName = getIAudioFlingerStatistics().getMethodForCode(code);
    mediautils::TimeCheck check(
            std::string("IAudioFlinger::").append(methodName),
            [code, methodName](bool timeout, float elapsedMs) { // don't move methodName.
        if (timeout) {
            mediametrics::LogItem(mMetricsId)
                .set(AMEDIAMETRICS_PROP_EVENT, AMEDIAMETRICS_PROP_EVENT_VALUE_TIMEOUT)
                .set(AMEDIAMETRICS_PROP_METHODCODE, int64_t(code))
                .set(AMEDIAMETRICS_PROP_METHODNAME, methodName.c_str())
                .record();
        } else {
            getIAudioFlingerStatistics().event(code, elapsedMs);
        }
    }, mediautils::TimeCheck::kDefaultTimeoutDuration,
    mediautils::TimeCheck::kDefaultSecondChanceDuration,
    true /* crashOnTimeout */);

    // Make sure we connect to Audio Policy Service before calling into AudioFlinger:
    //  - AudioFlinger can call into Audio Policy Service with its global mutex held
    //  - If this is the first time Audio Policy Service is queried from inside audioserver process
    //  this will trigger Audio Policy Manager initialization.
    //  - Audio Policy Manager initialization calls into AudioFlinger which will try to lock
    //  its global mutex and a deadlock will occur.
    if (IPCThreadState::self()->getCallingPid() != getpid()) {
        AudioSystem::get_audio_policy_service();
    }

    return delegate();
}