Basic Auth urllib2 with HTTPPasswordMgrWithDefaultRealm and POST data

Question

I have this cURL call that works perfectly:

curl -H 'X-Requested-With: SO demo' -d 'parameter=value' https://username:[email protected]/api/work/

My conversion does not work.

import urllib2
# Create a password manager.
password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
# Add the username and password.
top_level_url = 'https://api.server.com'
password_mgr.add_password(None, top_level_url, 'username', 'password')
handler = urllib2.HTTPBasicAuthHandler(password_mgr)
# Create "opener" (OpenerDirector instance).
opener = urllib2.build_opener(handler)
# Install the opener so all calls to urllib2.urlopen use our opener.
urllib2.install_opener(opener)
# Create request.
headers = {'X-Requested-With':'SO demo.'}
uri = 'https://api.domain.com/api/work/'
data='parameter=value'
req = urllib2.Request(uri,data,headers)
# Make request to fetch url.
result = urllib2.urlopen(req)
urllib2.HTTPError: HTTP Error 401: Unauthorized

Here's what I don't get. The same server has a separate API which similar code does work on, where the only thing that has changed is the parameter and uri. Note the cURL call works on both API calls.

Second API cURL call (that works):

curl -H 'X-Requested-With: SO demo' -d 'parameter=value' https://username:[email protected]/api2/call.php

Equivalent code that works below:

import urllib2
# Create a password manager.
password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
# Add the username and password.
top_level_url = 'https://api.server.com'
password_mgr.add_password(None, top_level_url, 'username', 'password')
handler = urllib2.HTTPBasicAuthHandler(password_mgr)
# Create "opener" (OpenerDirector instance).
opener = urllib2.build_opener(handler)
# Install the opener.
# Now all calls to urllib2.urlopen use our opener.
urllib2.install_opener(opener)
# Create request.
headers = {'X-Requested-With':'SO demo.'}
uri = 'https://api.server.com/api2/call.php'
data='parameter=value'
req = urllib2.Request(uri,data,headers)
# Make request to fetch url.
result = urllib2.urlopen(req)
# Read results.
result.read()

Why does urllib2 work when the uri ends with a '.php', but not work when the uri ends with a '/'?

Answer 1

In the first request you are setting:

uri = 'https://api.domain.com/api/work/'

But if you were to do it the same as the second run, you probably meant to write it as:

uri = 'https://api.server.com/api/work/'

Answer 2

From Python urllib2 Basic Auth Problem

The problem [is] that the Python libraries, per HTTP-Standard, first send an unauthenticated request, and then only if it's answered with a 401 retry, are the correct credentials sent. If the ... servers don't do "totally standard authentication" then the libraries won't work.

This particular API does not respond with a 401 retry on the first attempt, it responds with an XML response containing the message that credentials were not sent.