Make .netrc authentication work with aiohttp

Question

I'd like to use a .netrc file with credentials to authenticate to an API using aiohttp. As far as I can tell this should be possible, as long as the file is in the home directory (or the relevant env variable set correctly) and trust_env=True set in the aiohttp.ClientSession.

But whatever I try, I get a 401 response. I've checked with requests, and it works just fine. I've browsed through the relevant code and it seems like it'll only pick up the credentials if a proxy is supplied. Can someone explain?

Here's an example that reproduces the issue:

First put a .netrc file in home directory:

machine httpbin.org
    login foo
    password bar

import aiohttp
import requests

url = "http://httpbin.org/basic-auth/foo/bar"

with requests.Session() as sess:
    r = sess.get(url)
    r.raise_for_status()

# no exception raised

async with aiohttp.ClientSession(trust_env=True) as session:
    r = await session.get(url)
    r.raise_for_status()

# exception raised

ClientResponseError: 401, message='UNAUTHORIZED', url=URL('http://httpbin.org/basic-auth/foo/bar')

Answer 1

From what I understand in the doc, the trust_env and .netrc credentials are only used for proxy authentication, not for regular server authentication.

For authentication to the server directly, the docs say that you have to use a BasicAuth object (as you surely know), but to use the .netrc file, one solution would be to use a custom authentication class, eg:

class NetrcAuth(aiohttp.BasicAuth):
    def __new__(cls, host):
        login, account, password = netrc.netrc().authenticators(host)
        return super().__new__(cls, login=login, password=password)

that you could then use as

from urllib.parse import urlparse

hostname = urlparse(url).hostname

async with aiohttp.ClientSession(auth=NetrcAuth(hostname)) as session:
    r = await session.get(url)
    r.raise_for_status()

Of course this is not optimal, as we would like to have the ClientSession take care of that for us, but it's maybe a step in the right direction?