Tornado and concurrent.futures.Executor

Question

I'm learning about async and Torando and struggling. First off is it possible to use executor class in Tornado?

The below example i'm creating a websocket and when receiving a message I want to run check() as another process in the background. This is a contrived example just for my learning sake. Neither the INSIDE or AFTER gets printed. Why do we need async specific packages like Motor if we have this executor class?

Also in all the example i've seen of Torando the @gen.coroutine are always done on classes that extend the tornado.web.RequestHandler in my example I'm using a tornado.websocket.WebSocketHandler can @gen.coroutine be used inside this class also?

Finally can anyone recommend a book or in-depth tutorial on this subject? I bought "Introduction to tornado" however it's a little outdated because it uses the tornado.gen.engine.

def check(msg):

   time.sleep(10)
   return msg

class SessionHandler(tornado.websocket.WebSocketHandler):
   def open(self):
      pass

   def on_close(self):
      pass


   # not sure if i needed this decorator or not? 
   @tornado.web.asynchronous
   def on_message(self,message):
      print("INSIDE")

      with concurrent.futures.ProcessPoolExecutor() as executor:
          f=executor.submit(check,"a")
          result = yield f
      print("AFTER")

Answer 1

In order to use yield, you must use @tornado.gen.coroutine (or @gen.engine). @tornado.web.asynchronous is unrelated to the use of yield, and is generally only used with callback-based handlers (@asynchronous only works in regular handlers, not websockets). Change the decorator and you should see your print statements run.

Why would someone write an asynchronous library like Motor instead of using an executor like this? For performance. A thread or process pool is much more expensive (mainly in terms of memory) than doing the same thing asynchronously. There's nothing wrong with using an executor when you need a library that has no asynchronous counterpart, but it's better to use an asynchronous version if it's available (and if the performance matters enough, to write an asynchronous version when you need one).

Also note that ProcessPoolExecutor can be tricky: all arguments submitted to the executor must be picklable, and this can be expensive for large objects. I would recommend a ThreadPoolExecutor in most cases where you just want to wrap a synchronous library.