get working Microsoft Speech API with Angular

Question

hi im trying to find a way to get working Angular 5 with Microsoft Speech API i used microsoft-speech-browser-sdk for javascript

https://github.com/Azure-Samples/SpeechToText-WebSockets-Javascript

i just import the SDK import * as SDK from 'microsoft-speech-browser-sdk'; and i tried to use the same code on the example

but i have this error SDK.Recognizer.CreateRecognizer is not a function I know that the skd is imported because it executes the first functions

also i cant find the API reference Is there anyone who has got work this cognitive service with angular?

Answer 1

use this code SDK.CreateRecognizer(recognizerConfig,authentication);

Answer 2

I had this same issue and seems to be a typo in the blogpost, so I compared with the SDK sample here: https://github.com/Azure-Samples/cognitive-services-speech-sdk/tree/master/samples/js/browser

Smael's answer is essentially the fix - remove the .Recognizer from the function call and that should fix it (also ensure that the SDK reference you're returning has the same name as the one you're importing:

import { Component } from '@angular/core';
import { environment } from 'src/environments/environment';
import * as SpeechSDK from 'microsoft-speech-browser-sdk';

@Component({
  selector: 'app-home',
  templateUrl: './home.component.html',
})
export class HomeComponent {

  speechAuthToken: string;
  recognizer: any;

  constructor() {
    this.recognizer = this.RecognizerSetup(SpeechSDK, SpeechSDK.RecognitionMode.Conversation, 'en-US',
      SpeechSDK.SpeechResultFormat.Simple, environment.speechSubscriptionKey);
  }

  RecognizerSetup(SDK, recognitionMode, language, format, subscriptionKey) {
    const recognizerConfig = new SDK.RecognizerConfig(
        new SDK.SpeechConfig(
            new SDK.Context(
                new SDK.OS(navigator.userAgent, 'Browser', null),
                new SDK.Device('SpeechSample', 'SpeechSample', '1.0.00000'))),
        recognitionMode, // SDK.RecognitionMode.Interactive  (Options - Interactive/Conversation/Dictation)
        language, // Supported languages are specific to each recognition mode Refer to docs.
        format); // SDK.SpeechResultFormat.Simple (Options - Simple/Detailed)

    // Alternatively use SDK.CognitiveTokenAuthentication(fetchCallback, fetchOnExpiryCallback) for token auth
    const authentication = new SDK.CognitiveSubscriptionKeyAuthentication(subscriptionKey);

    return SpeechSDK.CreateRecognizer(recognizerConfig, authentication);
  }

  RecognizerStart() {
    this.recognizer.Recognize((event) => {
        /*
            Alternative syntax for typescript devs.
            if (event instanceof SDK.RecognitionTriggeredEvent)
        */
        switch (event.Name) {
            case 'RecognitionTriggeredEvent' :
                console.log('Initializing');
                break;
            case 'ListeningStartedEvent' :
                console.log('Listening');
                break;
            case 'RecognitionStartedEvent' :
                console.log('Listening_Recognizing');
                break;
            case 'SpeechStartDetectedEvent' :
                console.log('Listening_DetectedSpeech_Recognizing');
                console.log(JSON.stringify(event.Result)); // check console for other information in result
                break;
            case 'SpeechHypothesisEvent' :
                // UpdateRecognizedHypothesis(event.Result.Text);
                console.log(JSON.stringify(event.Result)); // check console for other information in result
                break;
            case 'SpeechFragmentEvent' :
                // UpdateRecognizedHypothesis(event.Result.Text);
                console.log(JSON.stringify(event.Result)); // check console for other information in result
                break;
            case 'SpeechEndDetectedEvent' :
                // OnSpeechEndDetected();
                console.log('Processing_Adding_Final_Touches');
                console.log(JSON.stringify(event.Result)); // check console for other information in result
                break;
            case 'SpeechSimplePhraseEvent' :
                // UpdateRecognizedPhrase(JSON.stringify(event.Result, null, 3));
                break;
            case 'SpeechDetailedPhraseEvent' :
                // UpdateRecognizedPhrase(JSON.stringify(event.Result, null, 3));
                break;
            case 'RecognitionEndedEvent' :
                // OnComplete();
                console.log('Idle');
                console.log(JSON.stringify(event)); // Debug information
                break;
        }
    })
    .On(() => {
        // The request succeeded. Nothing to do here.
    },
    (error) => {
        console.error(error);
    });
  }

  RecognizerStop() {
    // recognizer.AudioSource.Detach(audioNodeId) can be also used here. (audioNodeId is part of ListeningStartedEvent)
    this.recognizer.AudioSource.TurnOff();
  }

}

Answer 3

(Angular 10)

Make sure you have the services installed by running:

npm install microsoft-cognitiveservices-speech-sdk

What I did is create a service for all the cognitive services and keep the logic within that service.

The Service: I have used a promise to get the result in the component

import { HttpClient } from '@angular/common/http';
import { Injectable } from '@angular/core';

import { CancellationDetails, CancellationReason, PhraseListGrammar, ResultReason, SpeechConfig, SpeechRecognizer, SpeechSynthesizer } from 'microsoft-cognitiveservices-
speech-sdk';

export class CognitiveService {
  private speechConfig = SpeechConfig.fromSubscription("subscriptionKeyHere", "areaHere");
  private speechRecognizer = new SpeechRecognizer(this.speechConfig);
  private speechSynthesizer = new SpeechSynthesizer(this.speechConfig)

  constructor(private httpClient: HttpClient) {}

  public speechToText() {
    return new Promise((resolve, reject) => {
      this.speechRecognizer.recognizeOnceAsync(result => {
        let text = "";
        switch (result.reason) {
          case ResultReason.RecognizedSpeech:
            text = result.text;
            break;
          case ResultReason.NoMatch:
            text = "Speech could not be recognized.";
            reject(text);
            break;
          case ResultReason.Canceled:
            var cancellation = CancellationDetails.fromResult(result);
            text = "Cancelled: Reason= " + cancellation.reason;
            if (cancellation.reason == CancellationReason.Error) {
              text = "Canceled: " + cancellation.ErrorCode;
            }
            reject(text);
            break;
        }
        resolve(text);
      });
    });
  }

  public textToSpeech(text: string) {
    this.speechSynthesizer.speakTextAsync(text);
  }

The Component: I have used the await/async approach

public async Speech2Text() {

if (this.listening) // Used for UI effects
  return;
this.listening = true;

await this.cognitiveService.speechToText().then((res: string) => {
  console.log(res);
  // use text 
})
  .catch((res: string) => {
    this._snackBar.open(res, "okay", { duration: 3000 });
  })
  .finally(() => this.listening = false);
}