How do I scrape web comments that use a LiveFyre widget?

295 Views Asked by At

I'm using BeautifulSoup to scrape comments from various news articles. However, one site is using a LiveFyre widget so when I call:

test = soup.find(id="livefyre_comment_stream")
print test

I get:

<div id="livefyre_comment_stream"></div>

without all of the "stuff" in-between the div tags. I don't think I can use the LiveFyre API because this is not my site, and I'm not sure what my other options are. Any thoughts?

0

There are 0 best solutions below