Process Multiple Requests Simultaneously and return the result using Klein Module Python

Question

• Hi I am using Klein Python module for my web server.
• I need to run each request separately as a thread and also need to return the result.
• But Klein waits until the completion of single request to process another request.
• I also tried using deferToThread from twisted module. But it also process the requests only after completion of the first request.
• Similarly I also tried @inlineCallbacks method it also produce the same result.

Note: This methods works perfectly when there is nothing to return. But I need to return the result.

Here I attached a sample code snippet below,

import time
import klein
import requests
from twisted.internet import threads

def test():
    print "started"
    x = requests.get("http://google.com")
    time.sleep(10)
    return x.text

app = klein.Klein()

@app.route('/square/submit',methods = ['GET'])
def square_submit(request):
    return threads.deferToThread(test)

app.run('localhost', 8000)

Answer 1

But Klein waits until the completion of single request to process another request.

This is not true. In fact, there's absolutely nothing wrong with the code you've provided. Simply running your example server at tcp:localhost:8000 and using the following curl commands, invalidates your claim:

curl http://localhost:8000/square/submit &    # run in background
curl http://localhost:8000/square/submit

Am I correct in assuming you're testing the code in a web browser? If you are, then you're experiencing a "feature" of most modern browsers. The browser will make single request per URL at a given time. One way around this in the browser would be to add a bogus query string at the end of the URL, like so:

http://localhost:8000/squre/submit
http://localhost:8000/squre/submit?bogus=0
http://localhost:8000/squre/submit?bogus=1
http://localhost:8000/squre/submit?bogus=2

However, a very common mistake new Twisted/Klein developers tend to make is to write blocking code, thinking that Twisted will magically make it async. Example:

@app.route('/square/submit')
def square_submit():
    print("started")
    x = requests.get('https://google.com')    # blocks the reactor
    time.sleep(5)    # blocks the reactor
    return x.text

Code like this will handle requests sequentially and should be modified with async alternatives.

Answer 2

As @notorious.no suggested, the code is valid and it works. To prove it, check out this code

# app.py
from datetime import datetime
import json
import time
import random
import string
import requests
import treq
from klein import Klein
from twisted.internet import task
from twisted.internet import threads
from twisted.web.server import Site
from twisted.internet import reactor, endpoints

app = Klein()

def test(y):
    print(f"test called at {datetime.now().isoformat()} with arg {y}", )
    x = requests.get("http://www.example.com")
    time.sleep(10)

    return json.dumps([{
        "time": datetime.now().isoformat(),
        "text": x.text[:10],
        "arg": y
    }])

@app.route('/<string:y>',methods = ['GET'])
def index(request, y):
    return threads.deferToThread(test, y)

def send_requests():
    # send 3 concurrent requests
    rand_letter = random.choice(string.ascii_letters)
    for i in range(3):
        y = rand_letter + str(i)
        print(f"request send at {datetime.now().isoformat()} with arg {y}", )
        d = treq.get(f'http://localhost:8080/{y}')
        d.addCallback(treq.content)
        d.addCallback(lambda r: print("response", r.decode()))

loop = task.LoopingCall(send_requests)
loop.start(15) # repeat every 15 seconds

reactor.suggestThreadPoolSize(3)

# disable unwanted logs
# app.run("localhost", 8080)

# this way reactor logs only print calls
web_server = endpoints.serverFromString(reactor, "tcp:8080")
web_server.listen(Site(app.resource()))
reactor.run()

Install treq and klein and run it

$ python3.6 -m pip install treq klein requests
$ python3.6 app.py

The output should be

request send at 2019-12-28T13:22:27.771899 with arg S0
request send at 2019-12-28T13:22:27.779702 with arg S1
request send at 2019-12-28T13:22:27.780248 with arg S2
test called at 2019-12-28T13:22:27.785156 with arg S0
test called at 2019-12-28T13:22:27.786230 with arg S1
test called at 2019-12-28T13:22:27.786270 with arg S2
response [{"time": "2019-12-28T13:22:37.853767", "text": "<!doctype ", "arg": "S1"}]
response [{"time": "2019-12-28T13:22:37.854249", "text": "<!doctype ", "arg": "S0"}]
response [{"time": "2019-12-28T13:22:37.859076", "text": "<!doctype ", "arg": "S2"}]
...

As you can see Klein does not block the requests.

Furthermore, if you decrease thread pool size to 2

reactor.suggestThreadPoolSize(2)

Klein will execute the first 2 requests and wait until there is a free thread again.

And "async alternatives", suggested by @notorious.no are discussed here.