Python consumes 99% of CPU running eventlet

Question

I have posted to the python and eventlet mailing list already so I apologize if I seem impatient.

I am running eventlet 0.9.16 on a Small (not micro) reserved ubuntu 11.10 aws instance.

I have a socketserver that is similar to the echo server from the examples in the eventlet documentation. When I first start running the code, everything seems fine, but I have been noticing that after 10 or 15 hours the cpu usage goes from about 1% to 99+%. At that point I am unable to make further connections to the socketserver.

This is the code that I am running:

    def socket_listener(self, port, socket_type): 
        L.LOGG(self._CONN, 0, H.func(), 'Action:Starting|SocketType:%s' % socket_type)   
        listener = eventlet.listen((self._host, port)) 
        listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        pool = eventlet.GreenPool(20000)
        while True: 
            connection, address = listener.accept() 
            connection.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
            L.LOGG(self._CONN, 0, H.func(), 'IPAddress:%s|GreenthreadsFree:%s|GreenthreadsRunning:%s' % (str(address[0]), str(pool.free()),str(pool.running())))
            pool.spawn_n(self.spawn_socketobject, connection, address, socket_type)
        listener.shutdown(socket.SHUT_RDWR)
        listener.close()

The L.LOGG method simply logs the supplied parameters to a mysql table.

I am running the socket_listener in a thread like so:

def listen_phones(self):  
    self.socket_listener(self._port_phone, 'phone') 

t_phones = Thread(target = self.listen_phones)
t_phones.start() 

From my initial google searches I thought the issue might be similar to the bug reported at https://lists.secondlife.com/pipermail/eventletdev/2008-October/000140.html but I am using a new version of eventlet so surely that cannot be it?

Answer 1

Sorry for late reply.

There was no code like listener.setblocking(0), therefore, it MUST behave as blocking and no sleep must be required.

Also, please use a tool like ps or top to at least ensure that it's python process who eats all CPU.

If the issue still persists, please, report it to one of these channels, whichever you like:

• https://bitbucket.org/which_linden/eventlet/issues/new
• https://github.com/eventlet/eventlet/issues/new
• email to [email protected]

Answer 2

If listener.accept() is non-blocking, you should put the thread to sleep for a small amount of time, so that the os scheduler can dispatch work to other processes. Do this by putting

time.sleep(0.03)

at the end of your while True loop.