Python Audio over Network Problems

Question

Hello I am having problems with audio being sent over the network. On my local system with no distance there is no problems but whenever I test on a remote system there is audio but its not the voice input i want its choppy/laggy etc. I believe its in how I am handling the sending of the audio but I have tried now for 4 days and can not find a solution.

I will post all relevant code and try and explain it the best I can

these are the constant/global values

    #initilaize Speex
    speex_enc = speex.Encoder()
    speex_enc.initialize(speex.SPEEX_MODEID_WB)
    speex_dec = speex.Decoder()
    speex_dec.initialize(speex.SPEEX_MODEID_WB)

    #some constant values
    chunk = 320
    FORMAT = pyaudio.paInt16
    CHANNELS = 1
    RATE = 44100

I found adjusting the sample rate value would allow for more noise

Below is the pyAudio code to initialize the audio device this is also global

    #initalize PyAudio
    p = pyaudio.PyAudio()
    stream = p.open(format = FORMAT,
                    channels = CHANNELS,
                    rate = RATE,
                    input = True,
                    output = True,
                    frames_per_buffer = chunk)

This next function is the keypress function which writes the data from the mic and sends it using the client function This is where I believe I am having problems.

I believe how I am handling this is the problem because if I press and hold to get audio it loops and sends on each iteration. I am not sure what to do here. (Ideas!!!)

    def keypress(event):
        #chunklist = []
        #RECORD_SECONDS = 5
        if event.keysym == 'Escape':
            root.destroy()
        #x = event.char
        if event.keysym == 'Control_L':   
            #for i in range(0, 44100 / chunk * RECORD_SECONDS):
            try:
                #get data from mic
                data = stream.read(chunk)
            except IOError as ex:
                if ex[1] != pyaudio.paInputOverflowed:
                    raise
                data = '\x00' * chunk
            encdata = speex_enc.encode(data)        #Encode the data.
            #chunklist.append(encdata)
            #send audio
            client(chr(CMD_AUDIO), encrypt_my_audio_message(encdata))

The server code to handle the audio

    ### Server function ###
    def server():
        PORT = 9001
        ### Initialize socket 
        server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        server_socket.bind((socket.gethostbyname(socket.gethostname()), PORT))
        # socket.gethostbyname(socket.gethostname())
        server_socket.listen(5)
        read_list = [server_socket]
        ### Start receive loop
        while True:
            readable, writable, errored = select.select(read_list, [], [])
            for s in readable:
                if s is server_socket:
                    conn, addr = s.accept()
                    read_list.append(conn)
                    print "Connection from ", addr
                else:
                    msg = conn.recv(2048)
                    if msg:                
                        cmd, msg = ord(msg[0]),msg[1:]
                        ## get a text message from GUI
                        if cmd == CMD_MSG:
                            listb1.insert(END, decrypt_my_message(msg).strip() + "\n")
                            listb1.yview(END)
                        ## get an audio message
                        elif cmd == CMD_AUDIO:
                            # make sure length is 16 --- HACK ---
                            if len(msg) % 16 != 0:
                                msg += '\x00' * (16 - len(msg) % 16)
                            #decrypt audio
                            data = decrypt_my_message(msg)
                            decdata = speex_dec.decode(data)
                            #Write the data back out to the speaker
                            stream.write(decdata, chunk)
                    else:
                        s.close()
                        read_list.remove(s)

and for completion the binding of the keyboard in Tkinter

    root.bind_all('', keypress)

Any ideas are greatly appreciated how I can make that keypress method work as needed or suggest a better way or maybe I am doing something wrong altogether

*cheers

Please note I have tested it without the encryption methods also and same thing :-)

Answer 1

Did you run ping or ttcp to test network performance between the 2 hosts?

If you have latency spikes or if some packets are dropped your approach to sending voice stream will suffer badly. TCP will wait for missing packet, report it being lost, wait for retransmit, etc.

You should be using UDP over lossy links and audio compression that handles missing packets gracefully. Also in this case you have to timestamp outgoing packets.