tests using tornado AsyncHTTPTestCase connection timeout

Question

i have problems with unit testing tornado app, pls help me. error stack trace:

Error Traceback (most recent call last): File "/Users/doc/python/lib/python3.5/site-packages/tornado/testing.py", line 432, in tearDown timeout=get_async_test_timeout()) File "/Users/doc/python-virt1/lib/python3.5/site-packages/tornado/ioloop.py", line 456, in run_sync raise TimeoutError('Operation timed out after %s seconds' % timeout) tornado.ioloop.TimeoutError: Operation timed out after 5 seconds

ERROR:tornado.application:Future exception was never retrieved: Traceback (most recent call last): File "/Users/doc/python/lib/python3.5/site-packages/tornado/gen.py", line 1021, in run yielded = self.gen.throw(*exc_info) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/types.py", line 179, in throw return self.__wrapped.throw(tp, *rest) File "/Users/doc/python-virt1/lib/python3.5/site-packages/tornado/gen.py", line 1015, in run value = future.result() File "/Users/doc/python-virt1/lib/python3.5/site-packages/tornado/concurrent.py", line 237, in result raise_exc_info(self._exc_info) File "", line 3, in raise_exc_info tornado.curl_httpclient.CurlError: HTTP 599: Empty reply from server Traceback (most recent call last):

test.py file:

from tornado.testing import gen_test
from tests.api_tests.base import AbstractApplicationTestBase


class ApiRestTest(AbstractApplicationTestBase):
    def setUp(self):
        super(ApiRestTest, self).setUp()
        self.prepareDatabase(self.config)
        self.insert_user(config=self.config)

api_test/base.py

import logging
from api import server
from commons.constants import config
from tests.base import BaseTestClass


class AbstractApplicationTestBase(BaseTestClass):
    def get_app(self):
        application = server.get_application(self.config)

        application.settings[config.APPLICATION_DB] = self.db
        application.settings[config.APPLICATION_CONFIG] = self.config
        application.settings[config.APPLICATION_AES] = self.aes
        application.settings[config.APPLICATION_FS] = self.fs
        logging.info(self.config.DEPLOY_API)

        return application

test/base.py

import logging
from datetime import datetime
import motor.motor_tornado
from motor import MotorGridFSBucket
from pymongo import MongoClient
from tornado import escape
from tornado import gen
from tornado.testing import AsyncHTTPTestCase

class BaseTestClass(AsyncHTTPTestCase):
     @classmethod
     def setUpClass(self):
         super(BaseTestClass, self).setUpClass()
         self.config = Config(Environment.TESTS.value)
         self.client = utils.http_client(self.config.PROXY_HOST, self.config.PROXY_PORT)
         self.db = motor.motor_tornado.MotorClient(self.config.MONGODB_URI)[self.config.MONGODB_NAME]
         self.fs = MotorGridFSBucket(self.db)

Answer 1

A few things I noticed.

• The main issue I see is that you're doing some IO there in your setUp method via motor and setUp cannot be a gen_test (AFAIK). If you need this type of functionality you may need to drop down to pymongo and call the database synchronously for stubbing those database fixtures.
• Are you intentionally running against a real database? Are these supposed to be true integration tests with mongodb? When I am writing these types of tests, I would typically use the Mock class and mock out all my interactions with MongoDb.
• Also, There is no cost to creating an AsyncHttpClient object, so passing that into each handler from your settings/config object is probably not best practice.

Here's an example handler test in a project of mine:

example_fixture = [{'foo': 'bar'}]
URL = r'/list'

    class BaseListHandlerTests(BaseHandlerTestCase):
        """
        Test the abstract list handler
        """
        def setUp(self):
            self.mongo_client = Mock()
            self.fixture = deepcopy(example_fixture)
            # Must be run last
            BaseHandlerTestCase.setUp(self)

        def get_app(self):
            return Application([
                (URL, BaseListHandler,
                 dict(mongo_client=self.mongo_client))
            ], **settings)

        def test_get_list_of_objects_returns_200_with_results(self):
            self.mongo_client.find.return_value = self.get_future(example_fixture)
            response = self.fetch('{}'.format(URL))
            response_json = self.to_json(response)
            self.assertListEqual(response_json.get('results'), example_fixture)
            self.assertEqual(response.code, 200)

        def test_get_list_of_objects_returns_200_with_no_results(self):
            self.mongo_client.find.return_value = self.get_future([])
            response = self.fetch('{}'.format(URL))
            self.assertEqual(response.code, 200)

        def test_get_list_of_objects_returns_500_with_exception(self):
            self.mongo_client.find.return_value = self.get_future_with_exception(Exception('FAILED!'))
            response = self.fetch('{}'.format(URL))
            self.assertEqual(response.code, 500)

The key to making this work is that my mongo_client is passed into the route object itself. So my handler initialize takes a mongo_client kwarg.

class BaseListHandler(BaseHandler):
    """
    Base list handler
    """
    mongo_client = None

    def initialize(self, mongo_client=None):
        """
        Rest Client Initialize
        Args:
            mongo_client: The client used to access documents for this handler

        Returns:

        """
        BaseHandler.initialize(self)
        self.mongo_client = mongo_client

Answer 2

AsyncHTTPTestCase creates a new IOLoop at the beginning of each test, and destroys it at the end of each test. However, you're creating a MotorClient at the beginning of the whole test class, and using the default global IOLoop instead of the IOLoop that is created specifically for each test.

I believe you need only replace setUpClass with setUp. Then you'll create your MotorClient after AsyncHTTPTestCase sets up its IOLoop. For clarity, pass the IOLoop explicitly:

client = MotorClient(io_loop=self.io_loop)
self.db = client[self.config.MONGODB_NAME]