GNAudioSourceMic raw audio location

Question

I'm currently developing an app which uses Gracenote Mobile Client to create a fingerprint as well as identify which music I'm listening to. I've successfully implemented it on my project but now due to a business requirement I've to use the audio recorded by Gracenote for a different processing.

The point is: As GNAudioSourceMic encapsulates the whole microphone recording operations such as startRecording/stopRecording so I've no access to Microphone raw audio.

This is the code I'm using:

- (void)viewDidLoad
{
    [super viewDidLoad];
    [self setNeedsStatusBarAppearanceUpdate];
    [self setupUI];

    @try {
        self.config = [GNConfig init:GRACENOTE_CLIENTID];
    }
    @catch (NSException * e) {
        NSLog(@"%s clientId can't be nil or the empty string",__PRETTY_FUNCTION__);
        [self.view setUserInteractionEnabled:FALSE];
        return;
    }

    // Debug is disabled in the GUI by default
#ifdef DEBUG
    [self.config setProperty:@"debugEnabled" value:@"1"];
#else
    [self.config setProperty:@"debugEnabled" value:@"0"];
#endif
    [self.config setProperty:@"lookupmodelocalonly" value:@"0"];

    // -------------------------------------------------------------------------------
    //Init AudioSource to Start Recording.
    // -------------------------------------------------------------------------------

    self.recognizeFromPCM = [GNRecognizeStream gNRecognizeStream:self.config];
    self.audioConfig = [GNAudioConfig gNAudioConfigWithSampleRate:44100 bytesPerSample:2 numChannels:1];

    self.objAudioSource = [GNAudioSourceMic gNAudioSourceMic:self.audioConfig];
    self.objAudioSource.delegate=self;

    NSError *err;

    RecognizeStreamOperation *op = [RecognizeStreamOperation recognizeStreamOperation:self.config];
    op.viewControllerDelegate = self;
    err = [self.recognizeFromPCM startRecognizeSession:op audioConfig:self.audioConfig];

    if (err) {
        NSLog(@"ERROR: %@",[err localizedDescription]);
    }

    [self.objAudioSource startRecording];

    [self performSelectorInBackground:@selector(setUpRecognizePCMSession) withObject:nil];

}

-(void) startRecordMicrophone{
    #ifdef DEBUG
        NSLog(@"%s startRecording",__PRETTY_FUNCTION__);
    #endif

    NSError *error;
    error = [self.recognizeFromPCM idNow];

    if (error) {
        NSLog(@"ERROR: %@",[error localizedDescription]);
    }

}

Does someone have been exposed to the same need as explained above ?

Thanks in advance

Answer 1

After much googling yesterday I came up with a solution which isn't what I was previously expecting but it works as good as I want to. I've decided to record the iOS microphone myself and then call a method on Grancenote SDK to recognise what I've just recorded.

Here's what has worked for me.

MicrophoneInput.h

#import <Foundation/Foundation.h>
#import <UIKit/UIKit.h>
#import <AVFoundation/AVFoundation.h>

@interface MicrophoneInput : UIViewController {
    AVAudioPlayer *audioPlayer;
    AVAudioRecorder *audioRecorder;
    int recordEncoding;
    enum
    {
        ENC_AAC = 1,
        ENC_ALAC = 2,
        ENC_IMA4 = 3,
        ENC_ILBC = 4,
        ENC_ULAW = 5,
        ENC_PCM = 6,
    } encodingTypes;
}

-(IBAction) startRecording;
-(IBAction) stopRecording;

@end

MicrophoneInput.m

#import "MicrophoneInput.h"


@implementation MicrophoneInput

- (void)viewDidLoad
{
    [super viewDidLoad];
    recordEncoding = ENC_PCM;
}

-(IBAction) startRecording
{
    NSLog(@"startRecording");
    [audioRecorder release];
    audioRecorder = nil;

    // Init audio with record capability
    AVAudioSession *audioSession = [AVAudioSession sharedInstance];
    [audioSession setCategory:AVAudioSessionCategoryRecord error:nil];

    NSMutableDictionary *recordSettings = [[NSMutableDictionary alloc] initWithCapacity:10];
    recordSettings[AVFormatIDKey] = @(kAudioFormatLinearPCM);
    recordSettings[AVSampleRateKey] = @8000.0f;
    recordSettings[AVNumberOfChannelsKey] = @1;
    recordSettings[AVLinearPCMBitDepthKey] = @16;
    recordSettings[AVLinearPCMIsBigEndianKey] = @NO;
    recordSettings[AVLinearPCMIsFloatKey] = @NO;   

    //set the export session's outputURL to <Documents>/output.caf
    NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
    NSString *documentsDirectory = paths[0];
    NSURL* outURL = [NSURL fileURLWithPath:[documentsDirectory stringByAppendingPathComponent:@"output.caf"]];
    [[NSFileManager defaultManager] removeItemAtURL:outURL error:nil];
    NSLog(@"url loc is %@", outURL);

    NSError *error = nil;
    audioRecorder = [[ AVAudioRecorder alloc] initWithURL:outURL settings:recordSettings error:&error];

    if ([audioRecorder prepareToRecord] == YES){
        [audioRecorder record];
    }else {
        int errorCode = CFSwapInt32HostToBig ([error code]); 
        NSLog(@"Error: %@ [%4.4s])" , [error localizedDescription], (char*)&errorCode); 

    }
    NSLog(@"recording");
}

-(IBAction) stopRecording
{
    NSLog(@"stopRecording");
    [audioRecorder stop];
    NSLog(@"stopped");
}


- (void)dealloc
{
    [audioPlayer release];
    [audioRecorder release];
    [super dealloc];
}


@end

Obs.: If you're using ARC don't forget to add -fno-objc-arc compiler flag on Compiling BuildPhase as shown below.

YourViewController.h

//Libraries
#import <AVFoundation/AVFoundation.h>
#import <AudioToolbox/AudioToolbox.h>

//Echonest Codegen
#import "MicrophoneInput.h"

//GracenoteMusic
#import <GracenoteMusicID/GNRecognizeStream.h>
#import <GracenoteMusicID/GNAudioSourceMic.h>
#import <GracenoteMusicID/GNAudioConfig.h>
#import <GracenoteMusicID/GNCacheStatus.h>
#import <GracenoteMusicID/GNConfig.h>
#import <GracenoteMusicID/GNSampleBuffer.h>
#import <GracenoteMusicID/GNOperations.h>
#import <GracenoteMusicID/GNSearchResponse.h>

@interface YourViewController : UIViewController<GNSearchResultReady>


@end

YourViewController.m

#import "YourViewController.h"

@interface YourViewController ()
//Record
@property(strong,nonatomic) MicrophoneInput* recorder;
@property (strong,nonatomic) GNConfig *config;
@end

@implementation YourViewController


#pragma mark - UIViewController lifecycle

- (void)viewDidLoad
{
    [super viewDidLoad];
    self.recorder = [[MicrophoneInput alloc] init];

    @try {
        self.config = [GNConfig init:GRACENOTE_CLIENTID];
    }
    @catch (NSException * e) {
        NSLog(@"%s clientId can't be nil or the empty string",__PRETTY_FUNCTION__);
        [self.view setUserInteractionEnabled:FALSE];
        return;
    }

    // Debug is disabled in the GUI by default
#ifdef DEBUG
    [self.config setProperty:@"debugEnabled" value:@"1"];
#else
    [self.config setProperty:@"debugEnabled" value:@"0"];
#endif
    [self.config setProperty:@"lookupmodelocalonly" value:@"0"];
}    

-(void)viewDidAppear:(BOOL)animated{
    [self performSelectorInBackground:@selector(startRecordMicrophone) withObject:nil];
}

-(void) startRecordMicrophone{
    #ifdef DEBUG
        NSLog(@"%s startRecording",__PRETTY_FUNCTION__);
    #endif
    [self.recorder startRecording];
    [self performSelectorOnMainThread:@selector(makeMyProgressBarMoving) withObject:nil waitUntilDone:NO];
}

-(void) stopRecordMicrophone{
#ifdef DEBUG
    NSLog(@"%s stopRecording",__PRETTY_FUNCTION__);
#endif
    [self.recorder stopRecording];

    NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
    NSString *documentsDirectory = paths[0];
    NSString *filePath =[documentsDirectory stringByAppendingPathComponent:@"output.caf"];

    NSData* sampleData = [[NSData alloc] initWithContentsOfFile:filePath];
        GNSampleBuffer *sampleBuffer = [GNSampleBuffer gNSampleBuffer:sampleData numChannels:1 sampleRate:8000];
    [GNOperations recognizeMIDStreamFromPcm:self config:self.config sampleBuffer:sampleBuffer];
}

#pragma mark - UI methods

-(void)makeMyProgressBarMoving {

    float actual = [self.progressBar progress];
    if (actual < 1) {
        [self.loadingAnimationView showNextLevel];
        self.progressBar.progress = actual + 0.0125;
        [NSTimer scheduledTimerWithTimeInterval:0.25f target:self selector:@selector(makeMyProgressBarMoving) userInfo:nil repeats:NO];
    }
    else{
        self.progressBar.hidden = YES;
        [self stopRecordMicrophone];
    }

}            

#pragma mark - GNSearchResultReady methods
- (void) GNResultReady:(GNSearchResult*)result{
    NSLog(@"%s",__PRETTY_FUNCTION__);
}

@end

Credits go to Brian Whitman and Echo Nest Library for the MicrophoneInput solution.

Hope it helps someone out who is facing the same situation.

Cheers

Answer 2

The Gracenote SDK does provide access to the audio data even when using the included GnMic class. The GnMic class defines a GnMicDelegate protocol, which you can use to get notified when a new audio buffer is available. You must:

GnViewController.h

Add the GnMicDelegate protocol to your class definition

@interface GnViewController : UIViewController<CLLocationManagerDelegate, UITableViewDataSource, UITableViewDelegate, UINavigationBarDelegate, UIActionSheetDelegate, GnMicDelegate>

GnViewController.m

Assign your class as the delegate for the GnMic instance

self.gnMic = [[GnMic alloc] initWithSampleRate: 44100 bitsPerChannel: 16 numberOfChannels: 1 delegate:nil];
self.gnMic.gnMicDelegate = self;

Implement the protocol method. This will get called each time a new audio buffer is available for processing

- (void) audioBufferDidBecomeReady:(NSData *)samples {
    // Do something with the audio samples
}