Twilio - Transcribe voice answer from IVR

Question

I have set up an IVR in Twilio Studio and I have a voice prompt where the user states some information and I would like to have the IVR produce a transcript of that audio or produce a media file of the full audio (that I will use elsewhere) - I am unable to find this functionality in the docs but didn't know if there is a workaround or another solution.

Edit for more context - I want to ask the user to say their street address when prompted (which I will then use to look up their account and scheduled appointments).

Currently I can only set logic for things like "audio contains/does not contain X" but I have more advanced logic that I would like to implement and need the entirety of the user answer.

Answer 1

A good starting point for transcription is here: https://www.twilio.com/docs/glossary/what-is-call-transcription.

When using Gather with input as speech, the "action" URL is the endpoint where you will receive the transcribed text after the caller finished speaking:

The action attribute takes an absolute or relative URL as a value. When the caller finishes speaking the information (or the timeout is reached), Twilio will make an HTTP request to this URL. That request will include the user's data and Twilio's standard request parameters.

If you specify speech as an input with input="speech", Twilio will include SpeechResult and Confidence:

SpeechResult contains the transcribed result of your caller's speech.

Confidence contains a confidence score between 0.0 and 1.0. A higher confidence score means a better likelihood that the transcription is accurate.

Note: Your code should not expect confidence as a required field as it is not guaranteed to be accurate, or even set, in any of the results.

The "partialResultsCallback" is the URL where you would receive the transcription in real-time as soon as it is processed:

If you provide a partialResultCallback URL, Twilio will make requests to this URL in real-time as it recognizes speech. These requests will contain a parameter labeled UnstableSpeechResult which contains partial transcriptions. These transcriptions may change as the speech recognition progresses.

Here is some C# based on the example for gathering speech:

var response = new VoiceResponse();
var gather = new Gather(
    input: new []{Gather.InputEnum.Speech}.ToList(),
    action: new Uri("/completed"),
    partialResultCallback: new Uri("/partial"));
gather.Say("What is your street address?");
response.Append(gather);

The /completed endpoint will receive the transcribed text after the input is finished or timesout.

The /partial endpoint will receive the transcribed text as soon as it is available.

You would then need to receive the data from either the completed or partial endpoints and generate further Twiml to continue the call using that information.

As a caution, you may have more accurate results and possibly a better user experience asking for their phone number (which is only digits) or using the phone number from the caller id to lookup the account. Voice-to-text is more accurate for numbers than free form words. What you are seeking may not be possible in real-time with a high level of accuracy due to the accuracy of transcribing the wide variety of words that are in the street name. Or, you could do a combination of using caller id plus transcribed speech to achieve a higher level of accuracy.