Situation
We are working on the backend of our project consisting of two services written in Django and FastAPI. Both services use MongoDB as their database system.
In Django we use djongo==1.3.0
for ORM compatibility.
In FastAPI we use odmantic==0.3.4
.
Both of these libraries use pymongo==3.11.3
underneath.
This MongoDB SRV error has been an issue for our Django service ever since we created it, but we managed to somehow work around it by not using the latest packages, such as:
Django==2.2.20
djongo==1.3.0
pymongo==3.11.3
Recently due to security risks we had to upgrade:
urllib3 from 1.25.8 to 1.26.5
pydantic from 1.8.1 to 1.8.2
Django from 2.2.20 to 2.2.22
Those were suggested by GitHub's dependabot.
Problem
When we run any of these services locally now they break with the following base exception:
dns.resolver.NoAnswer: The DNS response does not contain an answer to the question: _mongodb._tcp.cluster0.k1eh0.mongodb.net. IN SRV
Full log for Django:
> python manage.py test
Creating test database for alias 'default'...
Traceback (most recent call last):
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/djongo/database.py", line 10, in connect
return clients[db]
KeyError: 'djongo_test'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 212, in __init__
rrset = response.find_rrset(response.answer, qname,
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/dns/message.py", line 341, in find_rrset
raise KeyError
KeyError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 220, in __init__
crrset = response.find_rrset(response.answer,
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/dns/message.py", line 341, in find_rrset
raise KeyError
KeyError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 72, in _resolve_uri
results = resolver.query('_mongodb._tcp.' + self.__fqdn, 'SRV',
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 1100, in query
return get_default_resolver().query(qname, rdtype, rdclass, tcp, source,
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 1003, in query
answer = Answer(_qname, rdtype, rdclass, response,
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 232, in __init__
raise NoAnswer(response=response)
dns.resolver.NoAnswer: The DNS response does not contain an answer to the question: _mongodb._tcp.database-test.avxhw.mongodb.net. IN SRV
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "manage.py", line 21, in <module>
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/core/management/commands/test.py", line 23, in run_from_argv
super().run_from_argv(argv)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/core/management/base.py", line 323, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/core/management/base.py", line 364, in execute
output = self.handle(*args, **options)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/core/management/commands/test.py", line 53, in handle
failures = test_runner.run_tests(test_labels)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/test/runner.py", line 629, in run_tests
old_config = self.setup_databases(aliases=databases)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/test/runner.py", line 552, in setup_databases
return _setup_databases(
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/test/utils.py", line 170, in setup_databases
connection.creation.create_test_db(
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/db/backends/base/creation.py", line 58, in create_test_db
self._create_test_db(verbosity, autoclobber, keepdb)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/db/backends/base/creation.py", line 168, in _create_test_db
with self._nodb_connection.cursor() as cursor:
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/db/backends/base/base.py", line 256, in cursor
return self._cursor()
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/db/backends/base/base.py", line 233, in _cursor
self.ensure_connection()
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/db/backends/base/base.py", line 217, in ensure_connection
self.connect()
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/django/db/backends/base/base.py", line 195, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/djongo/base.py", line 185, in get_new_connection
self.client_connection = Database.connect(db=name, **connection_params)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/djongo/database.py", line 13, in connect
clients[db] = MongoClient(**kwargs, connect=False)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/pymongo/mongo_client.py", line 639, in __init__
res = uri_parser.parse_uri(
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/pymongo/uri_parser.py", line 500, in parse_uri
nodes = dns_resolver.get_hosts()
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 102, in get_hosts
_, nodes = self._get_srv_response_and_hosts(True)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 83, in _get_srv_response_and_hosts
results = self._resolve_uri(encapsulate_errors)
File "/home/bk/inz/backend/authservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 79, in _resolve_uri
raise ConfigurationError(str(exc))
pymongo.errors.ConfigurationError: The DNS response does not contain an answer to the question: _mongodb._tcp.database-test.avxhw.mongodb.net. IN SRV
Full log for FastAPI:
> python -m src.main
Traceback (most recent call last):
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 72, in _resolve_uri
results = resolver.query('_mongodb._tcp.' + self.__fqdn, 'SRV',
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 1321, in query
return resolve(qname, rdtype, rdclass, tcp, source,
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 1305, in resolve
return get_default_resolver().resolve(qname, rdtype, rdclass, tcp, source,
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 1202, in resolve
(answer, done) = resolution.query_result(response, None)
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/dns/resolver.py", line 674, in query_result
raise NoAnswer(response=answer.response)
dns.resolver.NoAnswer: The DNS response does not contain an answer to the question: _mongodb._tcp.cluster0.k1eh0.mongodb.net. IN SRV
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/bk/inz/backend/testservice/src/main.py", line 20, in <module>
client = AsyncIOMotorClient(MONGODB_URL)
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/motor/core.py", line 156, in __init__
delegate = self.__delegate_class__(*args, **kwargs)
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/pymongo/mongo_client.py", line 639, in __init__
res = uri_parser.parse_uri(
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/pymongo/uri_parser.py", line 500, in parse_uri
nodes = dns_resolver.get_hosts()
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 102, in get_hosts
_, nodes = self._get_srv_response_and_hosts(True)
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 83, in _get_srv_response_and_hosts
results = self._resolve_uri(encapsulate_errors)
File "/home/bk/inz/backend/testservice/venv/lib/python3.8/site-packages/pymongo/srv_resolver.py", line 79, in _resolve_uri
raise ConfigurationError(str(exc))
pymongo.errors.ConfigurationError: The DNS response does not contain an answer to the question: _mongodb._tcp.cluster0.k1eh0.mongodb.net. IN SRV
In our CI setup on GitHub Actions Django service tests are still run successfully. It's clear that the problem lies within the DNS aspect.
Questions
- Are there any specific DNS servers which do not support MongoDB? I was using Google's
8.8.8.8
before and after it broke. - Why would the problem be caused by upgrading listed dependencies? Only
urllib
was upgraded for both services. - Is there any workaround?
Review https://github.com/mongodb/specifications/blob/master/source/initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.rst for how SRV URIs work, then manually issue the relevant DNS queries using a tool such as
dig
to verify your environment has working DNS.