getPage inside of deferred object printing?

Question

I'd like to create a callback that does a getPage on a specific url and prints that out when the operation is complete. Currently when I print d or page (see code below), I get a reference to the deferred object vs. the contents of page.

Why does the memory location of the deferred object change between print page and print d?

Eventually I'd like this program to cycle through a list of 4 of my websites, create callbacks for each of those individual connections, fire them off, and print each page as they are ready. If it's not too much to ask, can this be demonstrated?

from twisted.web.client import getPage
from twisted.internet import reactor
from twisted.internet.defer import Deferred

def connect(url):
    page = getPage(url)

print page returns <Deferred object at 0x23dcc68>.

print d returns <Deferred object at 0x7f1bacacc3b0>.

Current result (using 'http://www.example.com' as an example):

d = Deferred()

d.addCallback(connect)

reactor.callWhenRunning(d.callback, 'http://www.example.com')

reactor.callLater(4, reactor.stop)

reactor.run()

Answer 1

You should probably be using the newer, spiffier twisted.web.client.Agent rather than the older and somewhat limited getPage. Lucky for you there is a very thorough tutorial on how to use Agent, as well as some of its companion classes like ProxyAgent, RedirectAgent, CookieAgent, and ContentDecoderAgent.

First, though, you may want to familiarize yourself with the documentation on how to use Deferreds.