How to connect a voice stream websocket (goal is Google Voice API) to a Twimlets Conference

Question

In my existing app we use the https://www.twilio.com/labs/twimlets/conference feature :

var strURI = "http://twimlets.com/conference?Name=" + channelId.ToString() + "&Message=Welcome&Music=http://bter.com/radio/radio2.xml";
TwilioClient.Init(_twilioConfiguration.AccountSid, _twilioConfiguration.AuthToken);
var application = await ApplicationResource.CreateAsync(voiceUrl:new Uri(strURI), friendlyName: channelId.ToString());

My app then allows users in their browser to join in a special audio conference bridge to participate in training. It does not involve any phone number.

We have a need to route the stream to the Google Speech to Text API so we can use a custom dictionary.

In the examples I have found, and exercised

twilio phone-numbers:update 14805551212 --voice-url http://b74c35a792ff.ngrok.io

It seems like the phone number is configured in Twilio so that on connecting it opens a web socket stream to the endpoint (a ngrok endpoint proxy is common in the examples) .

In my situation, I wish to programmatically tell the dynamically created conference room to open a voice stream to a specific endpoint (for example http://b74c35a792ff.ngrok.io).

Is there an API call I can make to Twilio or additional settings I can add to the twimlets.com/conference to achieve this?

Answer 1

Twilio developer evangelist here.

When you set up a phone number with a voice URL in Twilio it doesn't connect via web sockets. Instead, when a phone call is made to the Twilio number it will make an HTTP request, a webhook, to the voice URL you provide. Similarly, when you create a new call via the REST API and provide a URL, Twilio will make an HTTP request to that URL when the call connects.

In both instances, Twilio is asking your application for instructions for what to do next. Those instructions are supplied by responding to the HTTP request with a subset of XML called TwiML. The XML elements and attributes describe what Twilio should do next with the call, like speaking a message, playing an audio file, gathering user input or dialling another number or into a conference.

When you use the Conference Twimlet, the URL you are providing returns TwiML to direct the call into a conference.

The Conference Twimlet does not provide any way to stream the audio from it. In fact, conferences in general do not make it easy to get the audio unless you literally dial into the conference itself. You could set up to stream the audio of each of the conference participants to your own websocket server by forking their audio before they join the conference using <Start><Stream> before redirecting to the conference Twimlet.

Sorry this isn't a full answer, but hope it helps.