I have a text to speech engine that reads exam questions. The reader seems to be working fine, but I have picked up an issue when there is a large text to continuously read.
After 12 Min (every time) the reader just stops reading. I have traced this to have something to do with the Naudio BufferedWaveProvider's BufferDuration. Here is my code:
Shared waveOut As WaveOut
Shared waveFormat As WaveFormat
Shared provider As BufferedWaveProvider
Private Shared Function InitializeAudio(Freq As Integer) As Boolean
Try
bNoAudioDected = False
waveOut = New WaveOut()
waveFormat = New WaveFormat(Freq, 16, 1)
provider = New BufferedWaveProvider(waveFormat)
provider.BufferLength = 31457280
''this was added for temporary duration fix
Dim ts = New TimeSpan(0, 30, 0)
provider.BufferDuration = ts
'''''''''''''''''''''''''''''''''''''''''''
provider.DiscardOnBufferOverflow = True
Try
waveOut.Init(provider)
Catch mmEx As NAudio.MmException
ErrorMessage("Audio device not detected!" & vbNewLine & "Please connect an audio device to use the Text-To-Speech system!")
bNoAudioDected = True
bAudioInitialised = True
Return False
Catch ex As Exception
WriteError(ex, "TextToSpeech -> InitializeAudio")
bNoAudioDected = True
bAudioInitialised = True
Return False
End Try
Return True
Catch ex As Exception
End Try
End Function
Private Shared Sub ReleaseAudio()
Try
If provider IsNot Nothing Then
provider.ClearBuffer()
End If
If waveOut IsNot Nothing Then
waveOut.[Stop]()
End If
Catch ex As Exception
End Try
End Sub
I noticed that the default BufferDuration is set to : 11:53 min. I changed this in code to 10 seconds using TimeSpan. After 10 seconds the system did exactly the same as the 12 min issue.
I have since changed the duration manualy as per my code above, first to 6 hours (just to over compensate), this cause the system to bomb out instantly. Then to 2 hours which seemed to be working fine, but cause issue with smaller text selections. Then to 30 min, what it is now currently and this seems to be stable but I'm concerned with the reliability since my other two attempts seemed to have caused more issues.
My questions is basically if this is a limitation to NAudio's dll that it can only handle that amount of continuous reading?
Should the BufferDuration be changed as I'm currently doing it or should the default value stay the same?
EDIT
Here is the Text-To-Speech wrapper class that I use. Note that this was provided by the Text-To-Speech provider I use.
Imports NAudio.Wave
Imports VENET
Imports System.Threading.Thread
Public Class TextToSpeech
#Region "Variables"
#Region "Shared"
Shared sText As String
Shared sDir As String()
Public Shared iSpeechRate As Integer = 100
Shared hTtsInstance As IntPtr
Shared hTtsCl As IntPtr
Public Shared bUploadTypePaper As Boolean = False
#End Region
#Region "Global"
Dim lstText As List(Of String)
Dim nErr As NUAN_ERROR
Dim tts As New VETts
Dim Lang As VE_LANGUAGE()
Dim Voice As VE_VOICEINFO()
Dim thread As Threading.Thread
#End Region
#End Region
#Region "Properties"
Private _newSpeed As Integer
Public Property NewSpeed() As Integer
Get
Return _newSpeed
End Get
Set(ByVal value As Integer)
_newSpeed = value
End Set
End Property
#End Region
Private SpeechHandler As New VENET.VE_OUTPUTDEVICE(AddressOf StreamSpeech)
Private Function StreamSpeech(hTtsInst As IntPtr, Msg As VE_MSG, Param As VE_LPARAM, OutData As VE_OUTDATA) As NUAN_ERROR
Try
If OutData.pOutPcmBuf IsNot Nothing Then
provider.AddSamples(OutData.pOutPcmBuf, 0, CInt(OutData.ulPcmBufLen))
End If
Catch ex As Exception
MessageBox.Show("StreamSpeech FAILED: " & nErr)
MessageBox.Show("StreamSpeech FAILED: " & ex.InnerException.ToString())
End Try
Return NUAN_ERROR.NUAN_OK
End Function
Private Sub ExecuteTts()
Try
NewSpeed = TtsSpeechSpeed
nErr = tts.ve_ttsGetLanguageList(hTtsCl, Lang)
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsGetLanguageList FAILED: " & nErr)
Exit Sub
End If
If Lang.Length = 0 Then
MessageBox.Show("There are no available languages!")
Exit Sub
End If
nErr = tts.ve_ttsGetVoiceList(hTtsCl, Lang(iLang).szLanguage, 0, Voice)
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsGetVoiceList FAILED: " & nErr)
Exit Sub
End If
If Voice.Length = 0 Then
MessageBox.Show("There are no available voices!")
Exit Sub
End If
Dim paramList() As VE_PARAM
paramList = New VE_PARAM() {New VE_PARAM(VE_PARAMID.VE_PARAM_LANGUAGE, Lang(iLang).szLanguage), New VE_PARAM(VE_PARAMID.VE_PARAM_SPEECHRATE), New VE_PARAM(VE_PARAMID.VE_PARAM_VOICE, Voice(iVoice).szVoiceName)}
If NewSpeed <> 0 Then
iSpeechRate = NewSpeed
ElseIf NewSpeed = 0 Then
paramList(1).usValue = iSpeechRate
End If
paramList(1).usValue = iSpeechRate
Try
nErr = tts.ve_ttsSetParamList(hTtsInstance, paramList)
Catch ex As Exception
MsgBox(ex.Message)
End Try
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsSetParamList FAILED: " & nErr)
Exit Sub
End If
Dim FreqParam As VE_PARAM() = {New VE_PARAM(VE_PARAMID.VE_PARAM_FREQUENCY)}
nErr = tts.ve_ttsGetParamList(hTtsInstance, FreqParam)
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsGetParamList FAILED: " & nErr)
Exit Sub
End If
tts.tts_setOutDevice(SpeechHandler)
' Start Streaming Audio
Try
waveOut.Play()
Catch ex As Exception
End Try
nErr = tts.ve_ttsProcessText2Speech(hTtsInstance, VE_TEXTFORMAT.VE_NORM_TEXT, sText)
If nErr <> NUAN_ERROR.NUAN_OK And nErr <> NUAN_ERROR.NUAN_E_TTS_USERSTOP Then
MessageBox.Show("ve_ttsProcessText2Speech FAILED: " & nErr)
Exit Sub
End If
Catch ex As Exception
MessageBox.Show("ExecuteTts FAILED: " & nErr)
MessageBox.Show("ExecuteTts FAILED: " & ex.Message.ToString())
MessageBox.Show("ExecuteTts FAILED: " & ex.InnerException.ToString())
End Try
End Sub
Public Function InitialiseTtsEngine() As Boolean
Try
sDir = {sTtsEnginePath}
nErr = tts.ve_ttsInitialize(sDir, hTtsCl)
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsInitialize FAILED: " & nErr)
Exit Function
End If
nErr = tts.ve_ttsOpen(hTtsCl, hTtsInstance)
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsOpen FAILED: " & nErr)
Exit Function
End If
If Not InitializeAudio(22050) Then
MessageBox.Show("Could not initialize audio with sampling frequency")
End If
Catch ex As Exception
End Try
End Function
Public Sub CleanUp()
If hTtsInstance <> 0 Then
nErr = tts.ve_ttsStop(hTtsInstance)
Application.DoEvents()
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsStop FAILED: " & nErr)
Exit Sub
End If
Sleep(1500)
nErr = tts.ve_ttsClose(hTtsInstance)
Application.DoEvents()
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsClose FAILED: " & nErr)
Exit Sub
End If
End If
End Sub
Public Function UnInitialiseTtsEngineForTest() As Boolean
Try
If hTtsInstance <> 0 Then
nErr = tts.ve_ttsStop(hTtsInstance)
Application.DoEvents()
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsStop FAILED: " & nErr)
End If
Sleep(1500)
nErr = tts.ve_ttsClose(hTtsInstance)
Application.DoEvents()
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsClose FAILED: " & nErr)
Exit Function
End If
nErr = tts.ve_ttsUnInitialize(hTtsCl)
Application.DoEvents()
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsUnInitialize FAILED: " & nErr)
Exit Function
End If
ReleaseAudio()
GC.Collect()
End If
Catch ex As Exception
End Try
End Function
Public Function UnInitialiseTtsEngineForQTCTest() As Boolean
Try
If hTtsInstance <> 0 Then
nErr = tts.ve_ttsStop(hTtsInstance)
Application.DoEvents()
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsStop FAILED: " & nErr)
End If
nErr = tts.ve_ttsClose(hTtsInstance)
Application.DoEvents()
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsClose FAILED: " & nErr)
Exit Function
End If
nErr = tts.ve_ttsUnInitialize(hTtsCl)
Application.DoEvents()
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsUnInitialize FAILED: " & nErr)
Exit Function
End If
ReleaseAudio()
GC.Collect()
End If
Catch ex As Exception
End Try
End Function
Public Function UnInitialiseTtsEngine() As Boolean
Try
If hTtsInstance <> 0 Then
nErr = tts.ve_ttsClose(hTtsInstance)
Application.DoEvents()
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsClose FAILED: " & nErr)
Exit Function
End If
nErr = tts.ve_ttsUnInitialize(hTtsCl)
Application.DoEvents()
If nErr <> NUAN_ERROR.NUAN_OK Then
MessageBox.Show("ve_ttsUnInitialize FAILED: " & nErr)
Exit Function
End If
ReleaseAudio()
GC.Collect()
End If
Catch ex As Exception
End Try
End Function
Public Sub PlayTts(ByVal text As String)
Try
sText = text
If thread IsNot Nothing Then
tts.ve_ttsStop(hTtsInstance)
ReleaseAudio()
thread.Join()
thread = Nothing
End If
thread = New Threading.Thread(AddressOf ExecuteTts)
thread.Start()
Catch ex As Exception
End Try
End Sub
Public Sub PauseResumeTts()
Try
If waveOut.PlaybackState = PlaybackState.Playing Then
waveOut.Pause()
ElseIf waveOut.PlaybackState = PlaybackState.Paused Then
waveOut.Resume()
End If
Catch ex As Exception
WriteError(ex, "TextToSpeech -> PlayPauseTts")
End Try
End Sub
Shared waveOut As WaveOut
Shared waveFormat As WaveFormat
Shared provider As BufferedWaveProvider
Private Shared Function InitializeAudio(Freq As Integer) As Boolean
If bUploadTypePaper = True Then
Try
bNoAudioDected = False
waveOut = New WaveOut()
waveFormat = New WaveFormat(Freq, 16, 1)
provider = New BufferedWaveProvider(waveFormat)
provider.BufferLength = 31457280
Dim ts = New TimeSpan(0, 30, 0)
provider.BufferDuration = ts
provider.DiscardOnBufferOverflow = True
Try
waveOut.Init(provider)
Catch mmEx As NAudio.MmException
ErrorMessage("Audio device not detected!" & vbNewLine & "Please connect an audio device to use the Text-To-Speech system!")
bNoAudioDected = True
bAudioInitialised = True
Return False
Catch ex As Exception
WriteError(ex, "TextToSpeech -> InitializeAudio")
bNoAudioDected = True
bAudioInitialised = True
Return False
End Try
Return True
Catch ex As Exception
End Try
End If
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
''This was provided
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Try
bNoAudioDected = False
waveOut = New WaveOut()
waveFormat = New WaveFormat(Freq, 16, 1)
provider = New BufferedWaveProvider(waveFormat)
provider.BufferLength = 31457280
provider.DiscardOnBufferOverflow = True
Try
waveOut.Init(provider)
Catch mmEx As NAudio.MmException
ErrorMessage("Audio device not detected!" & vbNewLine & "Please connect an audio device to use the Text-To-Speech system!")
bNoAudioDected = True
bAudioInitialised = True
Return False
Catch ex As Exception
WriteError(ex, "TextToSpeech -> InitializeAudio")
bNoAudioDected = True
bAudioInitialised = True
Return False
End Try
Return True
Catch ex As Exception
End Try
End Function
Private Shared Sub ReleaseAudio()
Try
If provider IsNot Nothing Then
provider.ClearBuffer()
End If
If waveOut IsNot Nothing Then
waveOut.[Stop]()
End If
Catch ex As Exception
End Try
End Sub
End Class
This is the code I use when I click on the "read text" button:
If CStr(currentSelection.type) <> "None" Then
Dim range As IHTMLTxtRange = TryCast(currentSelection.createRange(), IHTMLTxtRange)
If IsNothing(range.text) Then
MessageBox.Show("There is no available text to be read!", "No Text Available", MessageBoxButtons.OK, MessageBoxIcon.Hand)
Exit Sub
End If
If range IsNot Nothing Then
CallTts(range.text)
End If
ElseIf currentSelection Is Nothing And String.IsNullOrEmpty(browserContents) Then
MessageBox.Show("There is no available text to be read!", "No Text Available", MessageBoxButtons.OK, MessageBoxIcon.Hand)
Exit Sub
Else
CallTts(browserContents)
End If
And finally the CallTts function code which then goes to the TextToSpeech class provided:
Try
If thread IsNot Nothing Then
tts.ve_ttsStop(hTtsInstance)
ReleaseAudio()
thread.Join()
thread = Nothing
End If
thread = New Threading.Thread(AddressOf ExecuteTts)
thread.Start()
Catch ex As Exception
End Try
The whole idea behind
BufferedWaveProvideris that one thread is filling it up while another thread is reading from it. The rate of filling and reading needs to be roughly the same, otherwise the filling thread will overflow the buffer or the reading thread will have dropouts. If you're trying to set the size of the buffer to several hours, it sounds like your use case is not a good fit forBufferedWaveProviderat all. How are you adding the audio toBufferedWaveProvider?